Sounds like his adult daughter has been exploiting him for a very long time. Moving him out to yours gives her an entire house for free.
Perhaps DD is echoing how his wife treated him, or how he treated his wife, or perhaps both were ineffectual at parenting, and DD always ruled the roost.
Whatever the precise mechanics, his habitual relationship pattern is an abusive one, which should be a dousing of icy cold water to stop you in your tracks.
This is based on your interpretations though. There could be something else entirely going on. I would simply think, and say, that it's not working for you. It doesn't feel right. And leave it at that.
A man who doesn't invite you to his home, ever.
Who doesn't share his space, ever.
Who doesn't do any of the work of hosting, ever.
Who doesn't seem to have a grounded plan for realising the romantic daydreams he has beguiled you with.
Who isn't honest with you about whatever is really going on in his house.
Who still has an adult child at home.
Who he doesn't just support, but indulges.
Who he expends vast financial resources on.
Who he doesn't seem to have parented effectively.
Who he seems to have an extremely unhealthy dynamic with.
Just don't continue, or go back, not unless he makes dramatic changes, both tangibly, and in his ability to be honest and share himself. Which will take him considerable time. While you get on with enjoying your life and meeting whoever else might happen along.