With multiple reincarnations, I would not be suprise if he is a reincarnation of an ancient demon. Thus, Yuan could be the demon clans leader at some point in time! Since he has been also reincarnated of well known leaders of past eras!
my guess is that the first Celestial Emperor did something taboo that nobody but the Adura know about and they killed him for it. The other three families then exiled them but the second emperor actually agreed with there actions so created the legacy to give them a way out.