You can't get away from labels; it's how human society works. We all look at each other for expected behaviour and notice when it fails to materialise.
My dd can either be labelled "disabled" or "lazy and rude" when she doesn't get up for the headteacher - but there is no way HT is just going to shrug and say "heigh ho, just another child who doesn't do as she's told, well never mind".
Her friend on the autistic spectrum could either be labelled "boy with SN" or "boy who makes unkind and inappropriate remarks when he is old enough to know better"- the choice of not noticing his unusual behaviour isn't really there.
The mother of the 12yo who has a meltdown in the shopping centre probably prefers the label "SN" to any of the other possible choices.
The gentleman with Tourette's who used to go around my local area shouting swearwords wouldn't really have been any better off if locals hadn't realised he had SN and been able to explain to their children that he isn't really a rude and nasty man because he shouted a rude word at you.
Society views disabled people negatively. But that doesn't mean there is a high tolerance for people who just fail to comply with society's rules for unexplained reasons.
If you have SN bad enough to need a label, it won't magically disappear because you don't use the label. And people who are abused for looking different won't magically start looking the same as everybody else just because you are no longer using the word "disabled". (and the little shitheads who abuse them will continue to do so until the law does something about it)
Dd's paed told me repeatedly that she mustn't be allowed to use a wheelchair because "that will make her think of herself as disabled". I always wished for the courage to reply "so you don't think she might just notice anyway as she is crawling up the pavement on her hands and knees?"