Oh FFS, if looking after young children isn't a job you wouldn't pay people to do it when you're not there. Nannying, childminding, nursery work - all jobs.
As for people who say that a role is only a job if you get paid for it, childcare is effectively imputed income for SAHP meaning you don't pay for the service because you are providing it to yourself. As for NI contributions, you get NI credits over this time BECAUSE the role of caring for children is recognised as a genuine need not a lifestyle choice.
Working parents categorically do not do all the things SAHP do with less time, if they did then why would you pay £100 - £150 a day for a nanny depending on hours or £70-80 for nursery whilst at work (for those who have family help, consider yourselves very lucky). I know just as many people who sit at work googling holidays and sale shopping as SAHP who watch a couple of TV programmes with their kids each day. Likewise there are groups on both sides who are fully productive and engaged in their role at all times so there's no chilling out - it takes all sorts.
As someone who has done both (check out my username) I think there is huge value to both choices but they are YOUR choices to make. Unfortunately the general consensus seems to be to demonise SAHP as layabouts because it makes people feel better about themselves when frankly it's no one else's business. Yes you can do your own thing at home, but there is far more work associated with just being there all day with kids than shutting the door behind you and dumping them in nursery.
Being at home with one baby, health problems excepted, is a relatively easy job compared to two or more especially when they're 3/4. But there are plenty of jobs which are relatively easy, so I don't see a problem with that. Likewise when all children are at school there is more time, but that bit in the middle when you have anything between 4 year olds AND babies is flipping hard work. Personally I find it MUCH more straightforward being at work and outsourcing the childcare because I get to come back and have the fun times without all the crap and tantrums that go with parenting 24/7. I also find it more rewarding but that's because after several years at home I was craving direction, inclusion and stimulation for myself again, something you really give up as a SAHP.
So there you are, if you can (DH in your case) and don't mind putting aside the additional income and personal satisfaction (if you have any) from work for a while and you want to spend those years with your children, it's a great role but it's not that easy. If you don't, that's fine too. As long as you aren't closing any doors or doing something you think you might regret later then you can make it work.
Ultimately if you're happy, people should be happy for you and if they're not then you have bigger questions about why their opinion matters to you.
Good luck with whatever you choose.