YANBU at all OP. Mine are this sort of age and I hate weekends. My advice, fwiw:
Eldest to have "quiet time" in their room if not napping. NOT WITH THE IPAD. Colouring, Lego, whatever. Start with ten minutes and work your way up. When they have done that they can watch some TV. My five year old does an hour now - reads a book, chills etc while siblings nap. Send them up with a clock and show them where the big hand should be.
easy meals. Literally a big pot of pasta with cucumber sticks on the side, or a ready meal, or an oven pizza - whatever represents the least stress. One of you wrangle kids, the other get the meal ready and go through the eleventy billion things you normally get up for - bibs? check. Water? Check. Cutlery? Check. And so on.
Find playgrounds etc that you like/ hate the least. I regularly go to one in Shitsville south London because they have a coffee cart there. If all else fails, I'll have had a decent coffee. Likewise a museum with a Gails nearby, a museum with a nice view of the river, whatever.
Sleep train. If not now, when?
Agree a certain amount of free time with your OH and stick to it - every other weekend one of you get the afternoon to do your own thing or whatever.