I have to say, I think part of the reason this disconnect has happened at all is that the gender critical stance, this idea that we could have a society without gender, is really pie in the sky, it can never happen and probably would not be very nice if it did.
I think so too. This is a really important point. 'Gender abolition' is very close, philosophically, to 'Self ID', since both aspire towards a society in which, in the words of Shulamith Firestone, 'genital differences between human beings would no longer matter'. This is an impossible goal, not least because the differences between the sexes are not limited to genitals, or reproductive organs.
Both positions are rooted in an extreme social constructionism that sees all social relations between the sexes as fundamentally nothing to do with biology and always arbitrary and constructed.
I sometimes see gender-critical feminists say things like, 'I don't want to be treated as a woman, I want to be treated as a person'. But you can't be a person without a sex - there is no such thing as a non-sexed human being. My embodiment as female is not incidental or irrelevant to who I am; it's an intrinsic part of my personhood. I shouldn't have to aspire to a world in which people don't notice that I am a woman (and how would such a world come about, absent extreme transhumanist interventions?), in order to receive fair treatment. But this is the desired goal of both genderists and 'gender abolitionists'. I don't think most people want to live in such a world.
I think if we acknowledged that gender is always going to exist, and to some extent most people like it to, we could much more easily talk about ways in which it might be problematic and deal with those. There will always be masculinity and femininity because we will always know and think of ourselves in all the ways that our sexual bodies interface with the world and our experience and relation to the other. But we don't want those to be prisons for people.
This is why it's so much clearer to talk about 'sex roles' rather than 'gender'. If we abandon the (IMO false) claim that all behavioural differences between men and women are socially constructed, we can more easily talk about which sex roles might be harmful, which might be cultural artefacts that are better off abandoned, which might be partially rooted in biology but still amenable to change, etc