So the nub of it is that DH is a lazy sod?
Stop enabling that and the rest might well follow.
Here's your first experiment. Next in-law visit, you are busy. No, you've been quite clear that you want to do other things on weekends, that you need family time, but if he wants to continue prioritising his parents over that EVERY WEEKEND, then you can't actually stop him. However you don't plan to continue dancing to his or their tune constantly, so it looks as if you will be spending time apart.
You would have really liked to go out as a family this weekend and relax doing x, but looks as if he will be taking the children to see his parents... again.
You get up and get yourself ready and you go off to town, meet a friend for lunch, get out and about and do some stuff. And leave him to it.
Obviously it sounds as if his parents will take over when the kids are there - but - what you DON'T do is get kids up, feed, wash, clothe, pack a bag, make sure all is set for Mr. SitOnBumWithPhone to simply install them with parents, sit back down and glue his eyes back to the screen. You don't make lunch and leave it in the fridge ready. You don't tidy up in advance. You don't drop them off/pick them up.
Sounds like he's jus the type to like the idea of big family stuff happening and keeping his parents happy, but what he likes most of all is that the arrangement completely absolves him of any responsibility. He gets to sit and regress while not only you but his own Mummy appears to run everything. He doesn't have to think or put himself out.
See how he feels about it when you aren't there to be the eyes and ears for the kids and Mummy expects HIM to make conversation.