probably

I remember reading a theory that said that the Dwemer not only physically deteriorated Snow Elves to become Falmer, but deteriorated their souls as well. The evidence is that while in the game all 'people' have black souls, the Falmer who technically should be people but fallen have lesser souls or common souls I don't remember exactly. I think it's a pretty interesting idea to use in the story if you want to.