Thinking about this a bit more, I think my concern, that is in the intersection of concerns about (some applications of) the social model of disability and concerns about gender ideology, is to do with the use of labels to deny reality and compel certain behaviour from others.
I'm not so much thinking about physical disability - it's more obvious when we look at students, say, who have "reasonable adjustments" because they have disability labels like ADHD, dyslexia and/or autism. Some, not all, of those students seem to take the view that, given the label, other people have a duty to bend reality into what it would be if they had ideal abilities in the areas where actually they have deficits. So it's not enough that they get extensions, permission to use proof-readers, whatever; it is an affront, and must be someone else's fault, if they ever do less well than a peer and can see that as being caused by the disability. (E.g., you gave me an extension on piece of work X but that meant I had less time to work on the next piece, Y - you failed to issue me with my time-turner, in effect.) The key thing is that it's the label that enables the attitude. If the student didn't have a disability severe enough to get the label, but was just a bit crap at spelling/organising themselves/communicating, whatever, they and everyone else would take it as perfectly natural that they didn't do quite as well as a peer who was better at those things. So there are two satisfactory states: skills which are good enough to do well, and skills which are bad enough to get a label, and in between, there is a middle ground which is less desirable than either. I emphasise that I'm not saying every student with a disability label views it this way, but I've met plenty for whom this seems explanatory.
Rather similar, it seems to me, is the adoption of trans as a label. It's saying, I'm not just a bit uncomfortable with my body, like other girls - it's not just that I hate my breasts or wish I didn't have periods or hate my broken voice or my thick penis - it's that I am trans, and it's your job to make me feel fine and turn the world into a world where I am comfortable. Again, we have two satisfactory states: the (largely imaginary) one where someone has a gender identity that accords with their sex assigned at birth, and feels comfortable in their skin, and the one where someone is uncomfortable enough to deserve the "trans" label. In between, there's a middle ground that is less desirable than either - not being trans, not special, but just not very comfortable in your skin or with your place in society.
In both cases, it's great to accept the wide variety of humans and use looking at that properly to try to adjust the world to better suit more people, where we can - but using labels to deny reality is really a hiding to nothing, especially where there is no Platonic reality to whether someone "deserves" the label.