I'm sorry to say that he's being an absolute grade 1 old school wanker. And for the sake of your relationship you need to get angry NOW. Which clearly is not you; you are taking on practically the whole of the baby care including monitoring how he's feeling the whole time even if he is doing a short stint, and being the default carer at all times. If he's not enjoying it, he gets to hard the baby back (or if you can see the baby is not enjoying it, you take them back).
Do you see that when he decided to go to footy X nights a week, he not only decided what he would be doing those nights, he decided what you would be doing, without asking you? You'd be looking after the baby. Because with a young child, someone has to be 'on' all the time. He's just assuming he can do as much or as little as he likes, and you will do the rest, with no consultation.
And the actual baby care. Does he know anything about it? Has he read books, asked other parents, listened to podcasts, watched YouTube videos learning how to do it? Did he ask his mum, does he have young siblings he helped with? I doubt it. Because parenting is women's work, so it can't possibly be hard, can it. Controlling your feelings, using patience, adjusting your language and pace, observing your baby and watching for those cues that tell you they're tired, planning the day around their routine and preferences, making parent friends and fostering relationships so that they learn how to make friends? None of that could be HARD, could it? None of it could require his full attention? Because women do it all the time. So it must be shit easy.
He has no fucking idea. Many sex lives basically never recover from this fundamental betrayal, this discovery that your lovely partner actually has no respect for one if the hardest and most central experiences of your life.