Someone on Reddit suggested that maybe he went to prison. This would explain why he was in the same clothes. Perhaps he claims he doesn't remember what happened in those 30 years because he was too ashamed to admit he'd been in prison. This seems like the most likely explanation to me.