I’m not sure there is a right vs wrong here. He has a right to his feelings and you have every right to yours. I think it all comes down to whether your love for your husband is greater than your desire for a child. None of us can answer that question, only you can. You aren’t wrong to leave him, if the truth is that you feel the latter. If you really don’t want to be with him anymore and you are only staying for the possibility of having a baby- leave and try with a speed donor.
I would say, though, that perhaps understanding why he has said no more ivf might be helpful in coming to terms with it all. For instance, a pp had stated that your mental health isn’t in a great place right now- maybe he is really concerned that another round will just make this worse? And/or he feels that there is such a small chance of success that the impact that IVF is having on you both is not worth continuing to try. Or maybe he is concerned there will never be a “last go”- there will always be the desire for “just once more” just in case it works this time, so someone has to make the call to put an end on it.
I have no idea what your DH’s reasoning for delay in ttc was (maybe these were reasonable, maybe they weren’t)- but you did agree, so it’s not all down to him. Sometimes it is easier to apportion blame and ascribe nefarious intent to previous actions when we are in pain, than deal with the reality of the situation as it is. It seems unlikely your DH has just been secretly plotting for the last 16 years to prevent you having a baby- much more likely he didn’t feel you were in the right situation- financially, career, relationship stability etc- to have another child (and already having one in less than ideal circumstances may have made him more determined that the next child be more fortunate, for example). And when you both agreed to proceed, you have come across problems. Of course, as now find yourself in a very difficult situation it is easy to look back in hindsight with “buts” and “what ifs”.
I agree issuing threats/ultimatums aren’t helpful in a relationship- likely to destabilise a relationship which is already a bit rocky, by the sounds of things. Leaving a relationship which is making you unhappy is a different thing altogether. Issues ultimatums doesn’t seem healthy or fair.
I agree with a previous poster- counselling may help you work out the best route forward. You are holding resentment- he wanted to delay ttc and if he hadn’t the situation would be different (maybe, but definitely), he is denying you a baby etc. Those feelings need to be resolved one way or another- they will eat away at you and damage your relationship if not addressed. Perhaps after counselling you will want to leave, but hopefully that decision will be coming from a place of certainty and strength- not anger and desperation. Or maybe you will determine the opposite.
I would also advise against staying with him solely because ivf is unlikely to work as some sort of consolation “better to have him than nothing”- you both deserve better than that. If you love him, stay and build a life worth having together. If you don’t, leave and build the life you want- whether that includes continuing ivf alone is up to you.