Op I can understand your hurt, but you need to look at this objectively.
Your sons needs are high, you claim they are not but from your posts in May this isn't the case. Not wiping and jumping on soft furnishings, not being able to be left near unsecured cupboards and melting down if you're not around aren't easy to manage, especially for older people.
You seem to really be fixating on the 'normal' comments and I think this is down to you being autistic yourself and feeling triggered by this, wondering if that's what they thought about you growing up, that you weren't 'normal'
Regardless of their attitude or comments towards your son, at the end of the day it's unreasonable to hold your DD back from experiences with her grandparents. They feel able to cope with her, she will enjoy the time outside of the immediate family environment.
You also seem to be conflating siblings of ND children needing s break and not loving their siblings. Both can be true, my brother was diagnosed with Asperger's years ago (when that was a separate diagnosis) and was high functioning, no toileting issues or communication issues but it was amazing for me to have breaks growing up. I still love him, but his existence made my childhood very different and not in a good way either. When younger I resented him a lot and needed that time for me, to go to Pizza Hut (he could never go as the smell of melted cheese triggered him) or to go bowling (he could never handle all the noise and taking turns)
I still have a limited diet as an adult due to him, growing up as DM didn't want to have to cook 3 different meals I was only fed things he would eat, which wasn't a wide range and it has led to me having still restricted eating now, as those prime 'trying new foods and textures' phases went past me, who lived on nuggets, chips and bread until secondary when I started cooking for myself.
My parents made a lot of effort to try and limit the impact of his needs on me, it wasn't enough and never would have been. You don't seem able to understand just because you give her 121 attention, at some point that won't be enough. Your needs are clouding your judgement on this and it's very clear from your posts.
You are minimising his needs, you've already made your decision which is the wrong one for your DD.