Ugh, some of you lot. I'd love it so there's no reason why you should object to it...... Seriously?
OP, I feel for you. Even without all of your back story.
I LOATHE it when DH just pops in. He used to work a 1 hour commute away. I loved it.
Then worked a 15 minute commute away - hmm, nice that he was home early, and was around a bit more in the mornings (well it was after I bollocked him for disrupting our routine and told him that if he was going to be there he had to stop being in the way and to start helping - the message was received!
) but took some getting used to, he would ring me from the car saying he was on his way home and would suddenly just be there - tad annoying but got used to it.
DH now works half the week a 5 min walk away - oh how that can be annoying some times.
Both DC are at school, I work from home most days, but even if I don't, this is MY space during this time. If he popped in whenever he felt like it with no consideration as to the impact on my day I would be furious. He only pops in unannounced if he has to pick something up urgently or to check something. Otherwise he rings me up and asks whether I feel like having a cuppa/lunch whatever with him. Sometimes he comes here, sometimes I meet him at the office, corner shop.
Am I being selfish? Too right I bloody am, and I have every right to be so. It's MY space during the day. Just popping in whenever he feels like it would be MORE selfish of him than my wanting my space during the day is of me.
I also work from the office sometimes, and will start doing that more as time goes on. I always check whether it's a good day for him to have me there in the office. Because at the moment that is HIS space, and it would be bloody rude of me to just roll up and expect to do what I wanted there, whenever I wanted. He will sometimes say 'not today' and I won't. Selfish of him when I have my own space there? Yes, but again, he has every right to be selfish, it's his work!!!!
Its called respecting other people's space.
But if I was living with your background?! I would be incandescent with rage if DH tried to pull this stunt.