Idk why he feels guilty about having his priorities in check. He isn't safe there so of course he needs to focus on getting stronger. These girls who will 9/10 turn on him once his identity is exposed aren't more important than increasing his personal strength.