Do you go to bed early? Do you leave the house much? Do you have a child sleeping in your room? Does he get any time to himself other than late at night?
I sometimes stay up late after DH has gone to bed, as it is the only me time I get to myself, to just not have anyone wanting anything, where I can be completely alone to just watch whatever I like on tv, or doom scroll or whatever, I sometimes end up staying up really late, although I do always go to bed, I love it when DH is on lates as I can sit in bed and watch TV with a cuppa tea all by myself.. I really need that time alone to recharge, without it I feel overstimulated easily
He's told you the reason, it's because if he goes to bed with you it is just work, kids, sleep, repeat.. he's found a way of carving out some time for himself around everyone else's schedule, which he obviously craves, if you have a child in your room and/or a child in your bed I can understand why he has opted to sleep elsewhere, that may not be important to you but it obviously is to him.
It sounds like you are unhappy with things too so maybe you need to talk about what you both need and why and try and find a way to make it work.