Another perspective. We had to face this when dh and I started working from home together 24/7. He is self-employed, and I quit my job to be a SAHM. He thought nothing of coming and going without comment, saying that’s what he did at his office job (fairly senior). I first thought it was just bad manners, but actually it was annoying in other ways.
Several times a week I’d get a heart-attack from seeing him pop through a doorway or appear outside the kitchen window, me thinking he was out. Or I’d put the burglar alarm on because I thought I was leaving an empty house, only for him to set it off minutes later (not happy neighbours!).
I’ve made him adhere to a system of hello & goodbye, as well as a general idea of how the day’s looking so I can count on him for small and big things. I also really look forward to knowing when he’s out so I can have some basic alone time.
My point is, even if he had a legitimate reason for popping in and out it is important to see evidence that he recognises you and kids are part of his team. Team members communicate and support each other.
Otherwise the Team may well learn to work around them, and eventually they get cut...
Emphasise that the house is like the playing field, and the game is always on. There’s no leaving your position on the field if you’re a team player. For whatever reason. Feel free to negotiate days/evenings/half hours off with your team members like the rest of us have to. Or get benched.