It always varies depending on the experience of others. I know a lot of SAHMs, so they wouldn't judge me negatively if I was a SAHM because they and their families know how tiring it can be and how much it entails.
I get a reverse judgement - a lot of people think I'm somehow 'super mum' because I work, study, and am at home sometimes with a near toddler. However DH is a SAHF, and I don't think people really get it. There is an assumption that I'm the 'main' parent and that DS would want me more than DH just because I'm 'mum'. When in fact, DS spends much more time alone with DH and as such DH often knows more about what he might want or need. They have a fantastic relationship and DS is very happily settled by DH - at least 2 days and nights a week he doesn't see me at all 
I would like more free time at home, so that I could spend more time with DS and DH and not with my mind thinking about work or study in the background. I thought I wanted to be a SAHM, but now I'm not sure I'd be very good at it. I've only had to have DS completely alone for a day or more on a handful of occasions and had to get very creative as to how we spent our time! a lot of baths were had DH is just very good at being a SAHF, even if it does wear him out too.
On the other hand SIL couldn't understand how I could possibly be tired as I don't work as many hours as she does (this actually comes from a place of love, she's younger than us, still at uni, and not one of those crazy judgey SIL stories you come across on MN!). Because from her perspective, my set up looks simple - go to work a bit, study a bit, stay at home a bit, DH looks after DS and does some study also. She equally feels that DH doesn't do much. But I know that she thinks that because she just doesn't grasp the full experience of what we live and do, and at current to her the main object is to get a full time job in the city, working all hours, and anything else would be 'easy'. I can understand, because I used to think the same.
Like everyone else has said, can't please 'em all 