Kanchan, I am in a similar situation to yourself, the only difference being that my DH is very happy for me to be a SAHM.
I assume that you do all the housework, cooking, shopping, paperwork, presents, everything in fact. Which means that your DH has hardly any chores to do in the evening or at weekends. I wonder how he would feel taking on a proportion of the chores if you were to go to work.
And, some of the nasty comments on here, I wonder how many of them have 4 children and pay for childcare for 4. Its hardly financially viable for breakfast/after school clubs, summer holiday clubs for 4 children. And, illnesses x4.
Many couples are lucky to have a wonderful support network from families such as grandparents. My brother and his wife work full time but its the two sets of grandparents who are doing all the running around taking them to school and collecting them and giving them something to eat.
Its not as easy as many people make it out to be. Yes, running a home is a doddle, but unless you are earning a fortune that can cover reliable childcare for 4 then not really worth it.