ClasslessAscension
Memories are what makes you, you. Your personality is changed based on the experiences you have been through. So by inheriting someone else's memories, unless there is an external force at play which could be possible in this world, you will become them when there are more of their memories than yours
So choosing a class would make them forget who they really are and what they truly wanted to do. So I think that little man is right about the latter. The tower its self is making those with a class become more of the class than them. So Josh choosing classless means he is going to stay the same way no matter what.