I was born with a disability and sometimes use a wheelchair. My disability is very visible but like you my friends and family are used to it and don’t think about it.
Ive been mocked and insulted about it many times, I’ve had people taking photos since camera phones came out, even if I was out with my young son many years back.
It gets to a point where you’ve had enough, sometimes our skin can only be so thick or our self esteem can only take so much. Something we never asked for or chose to be is used against us daily. It’s not just about you getting into a fight for him it’ll also be thinking about how awkward and uncomfortable it must make you feel too. He’s likely to be protecting you as well as himself.
Society and adults especially has gone to shit with disabled people. I had on the whole a far better time as a disabled child, with hardly any problems at school than I ever do as a grown adult.
It won’t just be when he goes out socialising, it’s also when you’re doing the most mundane things like shopping, or picking up some much needed medication. I always liken it to being super famous, without the money or the fans, but that level of people looking at you, commenting about you, nudging their pal, taking a sneaky pic.
Let him pick the places he’s more comfortable with, if you’ve noticed he’s not coming out offer to do a night in with you all, give him time to repair his confidence and self esteem again.