I have a son who is borderline SEN. Really this translates to the fact that if you are teaching him you need special abilities (tolerance and patience being the key). His dyslexia does disable him, but only a little bit, his slightly odd mannerisms probably disable him more in terms of making friends. He is certainly not "normal" or "average" I certainly think he is special however. In the right environment he is happy and will I suspect contribute to the world (we have always seen him growing into a mad professor type). I don't however think the normal environment disables him.
I have close family who are profoundly disabled, they have high care needs and will always need a great deal of support. They are also special, but I think that the term "SN" is probably not sufficient on its own for their needs. I have a cousin with AT, who is leading as independent life as possible (with a great deal of support), but has the threat of early mortality hovering over him all the time. He is also highly intelligent. My niece has DS and autism, and in non verbal (doesn't use signs either, so has very very limited communication) My nephew has epilepsy and Aspergers type behaviour (there is a term that describes this but I have forgotten what it's called), he swings from being very unwell with his epilepsy to having very challenging behaviour with his Aspergers. I found it very unhelpful when school suggested that ds might be AS, because my understanding of autism is from the extreme end, so I thought it was a nonsensical thing to say. My mother is also registered disabled, with severe and chronic arthritis. She continues to live a fairly extraordinary and very active life. Grouping all these individuals into one group isn't very useful I guess, and my mother would be surprised to be called SN (although she does need help and extra consideration at times).
I think that part of the underpinning of this debate is that a very important part of the disabled movement was about ensuring that people didn't assume that those with physical disabilities were not automatically considered to have learning disabilities (or be "stupid"), particularly in an age when those with LDs were locked away. So blanket definitions are in that context problematic.
I tend to think of SN applying more to children, and to be more at the extra help and then will be OK in the mainstream world end of the spectrum of disabilities. I do think it is a much more positive or at least neutral term than retarded, which is innately negative IMO.