OP, you are not getting much support here so I wanted to add my experience which is similar to yours. I understand how you feel.
I had always wanted 4 and was always vocal in this, it was no secret.
DC3 was about 2 years old when I mentioned casually and conversationally about trying for fourth and last child. His easy-going and conversational reply was along the lines of 'im not even sure I want another, maybe we are done?'.
He was shocked by my immediate and instant response - me trying and failing to fight the tears, which came with sobs.
What was a light, easy going conversation became serious. My main issue, like yours OP, was that I didn't get chance to go thro a pregnancy and baby knowing it was my last. I still had all baby stuff saved in the garage ready for DC4. To suddenly have that taken away unexpectedly was horrible to face, given that we'always said four children.
My story has a different ending to yours tho OP. DH said he hadn't really thought about it before speaking. My conversation was out of the blue and caught him off guard. I took a long time to believe him and be persuaded because I didn't want him agreeing only cos it was what I wanted. We talked at length about it over several months.
Turns out it took over a year to conceive DC4, but she is due any day. I am 40 weeks pregnant exactly.
I feel blessed to have been able to enjoy this pregnancy as my last and feel ready to have this child lead me into the next stages of my life.