Personally I think if benefits are available for parents with young children to enable them to stay at home, there should be more support for those who work who want to take time off but cannot afford to- surely that's fairer? Plenty of people have their hand forced by finances, and I don't think we should be begrudging others but surely we should seek to level it out; not sure where the money would come from though.
Personally I grew up in a household with a SAHM, couldn't really afford it and we went without the basics- food was scarce, clothes solely hand me downs, couldn't afford to go on holiday, do activities, to be frank pretty hard existence. I would have loved to have had the opportunity to do crafts, have hot meals, play with other children and whatever else. I went back to work when DS was 1 and have zero regrets, financial stability and the opportunity to experience these things is important to us and something I didn't want him to miss out on like I did. He enjoys nursery, we love our time together and as I have reached a high level at work I work term time school hours and still get a decent wage. Its balanced out for me, yes the early years are important but so for me is school and beyond; I've been able to secure more time with him now he's more aware of what's going on because I carried on working.
Its different for everyone and that's okay.