I was in almost exactly the same position as OP. Very very similar. Unlike OP, however, I wouldn't have said he was a good partner, I DID say he was a great partner.
He was a selfish, narcissistic, sadistic absolute saggy ball bag of a man. Only to me though. To everybody else he was lovely.
The relationship started with him being a multiple marathon runner and long distance cyclist and me being sub-military level fitness. Ended with me now needing to rebuild my body as I was so unable to get workouts in on any consistent basis, over any all stretch of time, that I lost a LOT of muscle. He still runs marathons. Still bikes 100km easily.
This guy would shrug his shoulders when I used to beg and plead with him to just let me have one day in the week and one at the weekend when he wouldn't pip me to the post and get out of the house before I could/did.
So OP, if you're still reading, look at what else is going on in your relationship. Where are you in his list of priorities when it comes to YOUR needs being met? You say you had an agreement about how things are, which is fine, but you know that at any time you're allowed to change that? It's not set in stone. He should be working together with you to meet your needs, just like you're working around him now.
It's entirely possible for our bars of acceptable treatment to be eroded over time. It's not a criticism of you if that's happened, because the person we choose to spend our life with is someone we trust to have our best interests at heart. If they don't, and we have theirs at our heart, it's often very difficult to notice those bars being lowered, but by bit.
Btw I referred to my ex as sadistic: he got some kind of contentment that he was out there being billy big balls in sport and wifey was stuck at home and wishing she wasn't. It took me YEARS to notice that.