Your home is bought by whom? Do you own it, does he own it, do you own it together? Are both your names on the deeds?
At least it sounds as though you wouldn't be out on the street if you separated, far from it, so that's great, a better situation than most unmarried sahms. But that doesn't take away from the fact that in today's world it is the expectation that both sexes have equality of opportunity.
Presumably he is attached to his job then, even though you don't really need the money? Think about this really seriously before dismissing it: what would your lives be like if YOU worked, you had the money, you gave him cash and paid the bills, he stayed at home because he doesn't want strangers looking after the kids?
After all, it's him who is anxious about the kids being looked after by a parent, but he's not saying 'so therefore I will give up my job and sit at home with all the boring stress and domestic drudgery that entails', he's saying 'I want this, but I want my job, so YOU give up your dreams and ambitions to facilitate my desire'. Why is that? Why is that so acceptable to you?
Do you by any chance come from a culture or society where that is the norm? Where are these expectations coming from?
I say that because I am taking time out from my hectic but very successful career (board level in big well known firm) to look after kids as it suited me, but I have every intention of going back, have a meticulous game plan to do so and my husband is very supportive. But I don't tell everyone about this, and I have been shocked at how many people in the small provincial community I am from are like 'yes, that's quite right, you should be at home now, your husband has a good job, so your place is now with the kids, and remember, you're so LUCKY.' I am a fairly tough cookie, but this has fairly rocked my self-esteem. Do other people in your life (your family, his family) have strong views on your situation? What did his mother, what do his sisters, sisters-in-laws do? What did your mother do, did she work?