So what would happen to the HARD "NO! Out of four people in this household, the only one who wants to sleep early is you. That is your choice. Here are some very good earplugs, deal with it"
If he threatens to leave his big job (clue: he will threaten, but he wont actually do it because it further boosts his BIG MAN self esteem) say "Ok darling, do what you must, I am sure that we will manage financially and if we have to move, well thats ok too". That defuses the big bomb he thinks he is carrying.
Stop handing over the power to him. Yes your life would change massively if he stopped working, but what you are forgetting is, so will his.
You will have to go back to FT and then....he is the carer of your children instead of you. He will be the one doing all the running around, the shopping, cooking, caring, etc. Do you really think that he wants to do any of that?!
He is only threateing this because you always back down. So dont. Hold the gaze, dont blink, think of it as a game of "chicken" where he is pretending to drive a truck and is actually on a toddlers tricycle.