Grow a think skin and remember you will notice these social snubs a lot more than your DS will.
Sadly, although your sons diagnosis may not be widely known, his atypical behaviour will be apparent to other parents (IME many will steer their kids away from contact with kids who's behaviour is seen as "bad", especially if they don't know the child has a condition, and is not just being naughty), and in a year or so, the other kids will be aware of it too.
My DS is 15 now and has been through this - in the long run, it doesn't really matter about who invited him to parties at age 4, or 8,or 10.... he will eventually form a small group of real friends who won't care that he's a little bit different.
The most sensible thing we did for our boy, socially, was to tell him about his Asperger's (age about 7) and give him a list of responses to use in situations he wasn't comfortable with. His school was very against this, believing DS (or his class) should not be told he was different - the point is, he knew he was different, and so did the other kids. He was already being bullied and cold-shouldered, so it was a positive thing for him to have a few verbal responses ready to diffuse situations in the playground.
I do think it's important to get to know a few of the other mothers and make sure they know about your sons condition - they will be more likely to invite him to parties out of a sense of fairness if they don't just think he is "naughty", or "odd", or "has useless parents", (we've had all those comments and more!).