Bro no harem please. That just destroys the story. But if you're so adamant about that, than make it look like he's some kind of king figure, and that he can have a tonne of concubines as a the leader of a powerful group or something.
well once he runs in to a character from the walking dead that's easy to remember it should be obvious, I understand that it would be hard to know which zombie apocalypse he is in initially