Teatank, that is entirely normal, but trust me, you won't feel that weight of disappointment every day for the rest of your son's childhood.
My chidlren's dad sees a lot less of them now, but when he does come, he has to 100% parent them, instead of before, where I did everything (refer to the choking on resentment point) and he was just present.
But I do completely understand you. My X was appalling to me, there's no way I could have stayed with him, and in fact for the children's sakes, I had to leave, but the night I left I cried for him. The weight of HIS misery swamped my own.
I've bounced back a bit since then thank God Im no longer so in tune to all his needs and demands.
Nobody is stopping a separated father from being a good Dad. Honestly. It's more convenient under the same roof yes, but quite often, a mediocre dad will just be 'present' while the mum does all the parenting.
It is your perogative to end a relationship which isn't working (for either of you by the sounds of it). He wants the convenience of having you under the same roof, but he's already made it clear how he feels. Don't offer up your life as a sacrifice to his convenience. He can be a good Dad from half a mile down the road can't he?