The problem is, OP, that you appear to think that "they have special needs" overrides other people's freedom to act as they want. There's a spectrum: for anyone whose child's behaviour is challenging, for whatever reason, people pass through supportive, sympathetic, tolerant, grudgingly tolerant, hostile, having nothing to do with you.
Depending on how guilty they feel, how good a friend they are with you, their own experience of special needs, how well you are seen to be mitigating the issues, how old their own children are, etc, etc, the rate at which you move along that spectrum will vary.
But eventually, if your children's behaviour is sufficiently challenging, people will put their own children ahead of yours and exclude you from their lives. There is only so much that they will put up with. And as their own children get older, they will make their opinion clearer, and it will become harder and harder to convince their parents to upset their own children for your children's benefit.
Rightly or wrongly, you appear to have moved up that spectrum with all your friends, to the point that no-one is prepared to tolerate your children. That may be entirely wrong of them, and it may mean that they are bad people. But to each of them, the effect of excluding you is small, but to you, the effect of them all excluding you is huge. You have much more incentive to fix it than they do, because they simply don't care enough to try. Terrible people, perhaps, but thinking that doesn't help you.
And even if you could appeal to the better nature of parents, it sounds like your children are reaching an age where friendships are determined by children, not well-meaning parents. It doesn't sound like other children would get a great deal out of the relationship; the argument that adversity is good for their souls is not, I'm afraid, going to convince anyone, especially a ten year old whose toys have been broken.
So you have a choice. You can either lament the fact that other parents aren't as sympathetic to your plight as you think they should be, which is going to achieve precisely nothing. Or you can try to consider what it is that you could do to stop people from thinking "oh fuck, it's MoL, let's pretend we're out". Because it sounds like you've used up your allocation of sympathy, tolerance and guilt.