At a mother & toddler group recently, (I'm the only one there with 2) one woman said she was in 2 minds about having a second child, another woman said she had no intention of having another and a third agreed. No one made any negative comments, and in fact I felt like the odd one out having such a big brood!
I don't think you're alone with your negative comments. Although I now have my socially acceptable 2, I went the whole way through my pregnancy with people commenting on how close together in age they would be, how much work it would be, did I mind etc etc. It made my blood boil, happened every single day from aquaintences and strangers alike. It really put a dark cloud over my pregnancy which was very much wanted. Even now people frequently ask if they are twins. I point out one can walk and talk, and is 19 months older than the other who is lying in a pushchair gurgling, toothless, bald and very clearly, (surely?), a baby. They just then say defensively that they look like they could be twins and then say well, they are very close in age aren't they? It drives me mad and really upsets me. Will this interference go on forever?
At the moment loads of people seem to be pregnant with their second, but I know a fair few who have decided to have only one. I spent all of my childhood wishing I was an only child, and unfortunately my sibling relationship hasn't improved. My advice (!) would be to accept there's always insensitive and thoughtless people who think it's their place to offer advice/critisism no matter what your choice is, and if you've got more guts than me, have a pop back, I never do but it makes me mad every time.