You've got to be joking?
Either looking after the children is "time off", in which case it should be no problem for him to have that kind of "time off" two Saturdays per month while you do something outside the house, or it's childcare that prevents the adult in charge from pursuing their hobbies, in which case why should that fall on you every single Saturday while he goes and pursues his hobby at your expense?
That's not rhetorical. I'd want him to sit down and explain to me, step by step, which it is and why he thinks it's fair.
Better yet, get him to explain it in front of people he would like to think well of him. Would he be embarrassed to give his reasons in front of them?
Have you got friends (of his, preferably) that would give it to him straight and add, for good measure, "now before you go home and shout at DancingBinBags for 'making you look bad', let's be clear that the only reason you look bad is because of your own words and attitude, so don't think you can go bullying her in private once you get home!"?
2-5pm my arse. If anything, you've got a massive backlog of Saturdays off to be cashing in!