Sounds like a bloody perfect relationship to me!!
Perfect. You have your life - you can plan your day, do your thing, whatever that is, but you have the comfort and interest and support of him contacting/responding. And you have amazing sex with him when you want.
Amazing. I seriously don't see the problem.
If you want to watch a film with him, do it at his house before/after/during sex.
I get what he wants, and I think it's cool. He does want you and he is in a relationship with you. Just one where individual needs are kind of protected.
And tbh it's much nicer, e.g., for his kids and him to be together alone. Far simpler and honestly probably better for them (and you). He's being a grown-up in a way that is quite revolutionary, but personally I like it.
This is the same guy who's amazing in bed, who you love to sleep with.Who you love, actually, quite clearly. And looks like he loves you too. Wouldn't talk every day and fuck you, exclusively, if he didn't.
I think there's a lot to feel really happy with about that.