OP - you, I’m my opinion, are not at all unreasonable.
But what will do? Because your DH sounds like a twat concerning this.
My DH is just starting to go back to the office after two years of wfh. We have two autistic children.
On my off days (I’m PT) I have play dates, kids have freedom to have fun, I have friends in for coffee, he sometimes stops to do school run or come for a chat.
We talked about how to make it work properly, what was reasonable for both of us and made some changes.
It’s not always easy; his boss has seen me in my nightie because I forgot to get my clothes, we’ve had some renovations done which were very disruptive, Both children have made untimely appearances on Teams calls for both of us, but we both did what we could to make it easy for everyone.
You DH is performing with his booming voice and working in an open area - he’s effectively pissing all over the house to make his territory and it’s time for a standoff.
Because your only options are:
Do as he wants and go out every afternoon
Tell him you won’t and he deals with the interruptions or works elsewhere