As kindly and gently as possible, you are thinking of this situation in only a highly emotional way and taking any rationale out of it (which I completely understand by the way).
You are assuming that everyone commenting on your post has no experience whatsoever with something like this.
My children have ADHD, my FIL is extremely intolerant and would never cope with my kids (he has proved that many, many times over the years, he's an arse when we are there let alone leaving the kids without my hubby and I) and that wouldn't be good for my kids so they spend no time alone at all with him and very little with my MIL. In his mind my kids are undisciplined brats who probably need a good flogging to straighten them out. In reality I am constantly correcting, parenting and disciplining my kids.
I understand how much non neurotypical kids already miss out on and how much that hurts, how much harder life is for them in general.
Some people need help to understand our kids and some just never will get our kids. If you can take this opportunity to help your kids Grandparents get to know and understand your son, how great would that be, they are probably genuinely scared of how they will cope so go with them but take the back seat sometimes so they can learn.
I really think you should try opening up to them about how much this hurts you to see them want to love your daughter but not your son.
Yes, without a doubt, your son deserves the love and care of his family, without a doubt in my mind, hopefully an honest conversation and some support from you and your husband can help build that relationship.
I've given up on my FIL, I just never turn my back on him and my kids and we invite my MIL to our house separately, it hurts knowing that my kids will never have loving Grandparents the way other kids do but we have tried so hard to change it for years before giving up, he is one who doesn't want to understand.
Don't give up just yet, have that conversation and see what comes of it.