My DC (10) has ASD. Our home is the 'safe space' and we try to keep change to a minimum /talk them through potential visits.
They have a 4 and 3 year old cousin who often appear in the garden with a parent or grandparent without warning. If the children behave in a way on the garden equipment that I know is stressful to my DC, I ask them politely not to do it. Both the parent and grandparent think I am unreasonable for this and should be more inclined to let the younger ones do as they please. Equipment in our garden has been funded for meeting my DC sensory needs.
I don't think they understand the level of distress that these behaviours can cause my DC (self harm in the past).
YABU - stop asking them.
YANBU - Look out for your DC