I think you need to make him realise that first and foremost, the building is a family home and needs to function as that. So while he can wfh there are things that need to happen as a home that interfere with that - noisy housework, kids playing, people visiting etc. He needs to understand that there are things that you’ve had to change, and more importantly give up, because of his decision. And that some of those things are a problem.
So talk to him about how to accommodate at least some of those coming back.
like the Thursday circle - it’s not a weekly thing in your house as you said it moves around. So you want him to accept that every 3rd/4th/7th…Thursday, there will be noise in the house. But on the other hand, the other 2/3/6 Thursdays in that period will be quiet as you will be out with toddler. So he can schedule things needing quiet or intense concentration those weeks and either go out himself or expect to be disturbed on that 1 Thursday.
And as it is a family house with kids first, he needs to expect some noise when entertaining them or that older DCs may have a friend over at times..games and music will play etc. And also that as dc’s get older, they tend to get noisier in their play and with friends - and you may have to get noisier with them to make them do homework/chores.
So it’s worth having a proper conversation about it. Schedule a time, make it a serious conversation, and not about obstructing him but about how you can make it work for everything that needs to happen in a space that is firstly a family home. Or to consider if he needs to plan some time in the office or in a dedicated workspace that he rents for one day a week or whatever.
It’s not an accusation or a whine - it’s a proper communication about problems and how to solve them in a way that works for you both.
Because it sounds like it’s only working for one of you at the moment.