Did dh have to spend his childhood Sunday's with his paternal gp so mil is thinking 'now it's my turn?'
Either way it doesn't matter, if he didn't and mil got to do her own Sunday lunch and dh has happy memories of that then he is denying you and ds the chance of ever having those memories. Or if he did have to go them surely he remembers that it would have been nice to stay at home most some Sunday's yet he's prepared to inflict the same on his son.
I really think you should take a deep breath And see if dh will come with you to the museum. A family holiday day out. If you can miss when away on holiday then call it a holiday.
And then once you've come back and had a lovely time, tell PIL that it's your turn now to have Sunday lunch with your son and dh. For at least the next 4 years. And that they can come or not, that's their choice. If they don't come, they are denying themselves the chance to see your ds and dh and you.
And don't say that they will have to get more flexible for things like parties and playdates and homework because it is leaving the most important reason out. YOU! The fact that a child's own mother wants to spend Sunday lunch with their own young child is a significantly more valid thing than a mil demanding to spend it with just her ds and dgs.
I think she should think herself lucky especially given the way you have been treated if she gets 1 Sunday a month (and even that should be mutually convenient rather than 3rd Sunday of the month regardless of what else is on).
I'm also hoping that she does come to yours for lunch so you can tell her that, following her example, you didn't bother to cook her any lunch as she doesn't eat much... And then see what her reaction is...
Of course, you then might want to produce a meal that you've kept back so you can point out that actually you would never be so rude as to not serve food to all your guests but that you thought she might enjoy experiencing a little of her own hospitality!
But I think that you need to make a stand and say that, right here, right now, you will no longer be dictated to and all have your lives controlled by mil. Sunday lunches will be an ad hoc affair, organised individually and arranged on an approximately one time at our house, one time at your house basis. And that you will certainly not be seeing them every week or even every other week.
Stand strong and look forward to reclaiming your family Sunday's - remember you have the might of mumsnet supporting you on this!