i get a feeling like that sometimes when I'm at work- I'm a doctor, and 30, but prob look early 20s (unless it's been a really awful day!) and when patients say 'oh, you look too young to be a doctor', if i'm feeling a little low or underconfident for some reason, what i hear is 'oh god you can't possibly be a good doctor- i want an older better one' which, i think comes much more from me than from then.
Sometimes in the past i've been really grumpy with people and snapped back with something like 'well, i'm 30, with a toddler and a mortgage, so i'm probably old enough...' which just leaves everyone a bit unhappier! I've had much better experiences if i laugh along with them saying 'i know, and when even I think the policemen look young, we're all in trouble!!" which keeps them guessing then too!
I believe that what you see in other people is a reflection of yourself sometimes- a few years ago I made a concerted effort to always say something nice about someone before I then slagged them off said anything bad so rather than saying 'i think he's really rude' I'd say 'I'm sure he didn't mean to come across in that way, but that sounded really rude' and it really did make me see nicer things in people!!
so (stopping whittering on about me) what i'm trying to say is try and say it back to them in a positive way when you get a negative-ish comment:
them:'oh, you're young to be a mum'
you: 'thanks, I know i'm really lucky to have such a great family already'
or
you: 'thanks, having such fun kids really keeps you on your toes'
or
you:'thanks, isn't it funny what people's perceptions of a young mum have become big grin'
that way you'll start feeling less threatened by them and remind yourself how good things are.... and eventually when people watch you walk down the street, instead of worrying that they think you're too young, you'll be proud that they've noticed how well behaved/ intelligent/ observant/ kind your kids are!
Best of luck!