I'd be interested to understand what you mean by gender affirming healthcare.
I understand gender affirming hormones and surgeries to mean giving cross-sex hormones and cosmetic surgical interventions to enable someone to suppress or remove their sex characteristics and gain a simalcrum of the sex characteristics of the opposite sex.
Wider gender affirming healthcare might mean using their preferred names and pronouns in a clinical setting. Possibly including changing their 'gender marker' on their health record. Or it might be psychosocial support aimed at helping them live as a person of the opposite sex, or voice training to pass better.
Am I missing some healthcare that you believe falls under the gender-affirming umbrella? Is any of it the above well-evidenced? As PP has pointed out, Cass states clearly that affirming is NOT a neutral act in children. Is there any reason to think that in adults it would be different? Particularly the kind of young fragile adults that form the majority of the current cohort.
One of the fundamental problems with starting healthcare from a gender affirming basis for me is that is shuts down any talk of WHY. The aetiology of gender dysphoria, that Cass outlines very well - what is a patient's personal story that means that they are gender incongruent/dysphoric and wish to change their secondary sex characteristics? As multiple studies have shown, the current cohort of patients are genderally young/same-sex attracted/neuordiverse/traumatised or several of these. By not being able to start from an open neutral stance, but being obliged to 'affirm', the possibility of addressing some of the underlying issues - and who knows? potentially resolving the dysphoria - is shut down. This to me is the real tragedy of gender affirmation. So yes, I agree with OP, all gender affirming care should go. That does NOT mean all care should go.
To be completely and baldly clear - I am not saying that we should, as clinicians, say 'trans is not a thing'. It is possible, I would hope, to hold a sympathetic and neutral space for open discussion and the options of multiple ways of addressing the distress. Which should be evidenced.