People are bloody despicable. I'm really glad it's not you doing that OP, not going to lie I'd have handed your arse to you if it were!
My child has SEN, although you likely wouldn't know it to look at her or speak to her, she's one of those who would've flown under the radar years ago. Her classmates and the parents know though because she has her own TA and I'm quite open about it.
I did a party for her, that was at a noisy place. Because of her SEN I hired the whole place so it'd only be her and her classmates and she'd feel comfortable. Because she has several children in the class with SEN I also arranged a quiet space for them to retreat to (with their parents, they're little) complete with books, ear defenders, colouring etc if it all got too much and said to the SEN mums to feel free to bring the children's tablets etc if they wanted them and not worry about it or coming to find me beforehand to say goodbye if they needed to slip off early. I catered for the ones who only eat plain bread or nuggets or one type of crisps.
I cannot imagine not including children with additional needs it's absolutely horrible. That was automatic to me, I wanted all of the children to feel able to come knowing they could join in and have fun and step away if it got too overwhelming. I trusted the parents to manage their children's behaviour appropriately if they became overwhelmed or behaved in such a way that might have upset or hurt others, and they did. Most SEN parents are way more alert to their children's triggers, and on hot pins at things like parties. I know I am. I don't sit and chat and ignore the kids while they play I am constantly watching so that I can intervene if needed.
I would lose all respect for someone who deliberately excluded children for having a disability. It's an awful example to set to the other children. Our school would also refuse to hand out invitations where it was clear just a couple of particular children were being left out (SEN or not).
I wouldn't tell the Mums specifically, to answer your question, no. They're probably sadly used to this sort of bullshit by now but it's still hurtful. It stings when your child is left out for something they have no control over. What I will tell you, for if you or anyone else reading this is planning a party, making a little bit of effort to be inclusive goes a really long way and means a lot. It might not be your problem, or your child's problem. But there for the grace of god and all that. Inclusivity and ablism is all of our problem.
Every single one of the mums of the SEN children spoke to or messaged me to thank me for making an effort to make sure their children could come and join in comfortably. It meant a lot to them and happily since I've done that other parents have done similar. Because they'd have felt awkward before, saying to people 'I know your child finds X tough, we've got a room off to the side if they need it'. It is awkward, if you're not used to this world.
I can honestly say that our class is really inclusive, the children are all accepting of their classmates with additional needs or differences. It's lovely to see and I hope it continues as they get older. You only get that when parents lead by example!