Critique of a study claiming MH benefits of blockers/hormones

[polarisation]

Researchers Found Puberty Blockers And Hormones Didn’t Improve Trans Kids’ Mental Health At Their Clinic. Then They Published A Study Claiming The Opposite.

jessesingal.substack.com/p/researchers-found-puberty-blockers

Statistics heavy, but shows how this study claims dramatic improvement in teenagers treated with blockers and hormones, where actually the study's own data shows that they did not improve at all - they only maintained their original level of poor mental health. The claim of improvement was based on comparison with the apparent decline in mental health experienced by the dwindling control group (80% dropped out by the end of the study with only 6 remaining at the end). No explanation is given for why so many might have dropped out, and the methodology used for categorising the severity of the MH problems of the participants, and the statistical analysis method chosen, are questionable too.

As a quick summary, the main author was interviewed on "Science Friday" and made the following claim, which the host described as "huge"

DIANA TORDOFF: Absolutely. Thank you for having us here today. Our study was conducted at Seattle Children’s Hospital’s Gender Clinic. And our goal was to prospectively follow youth during their first year of receiving care to better understand their experiences, their well-being, as well as barriers they faced in accessing care. So we enrolled 104 trans youth, who are aged 13 to 20. And we found that youth who received puberty blockers or gender-affirming hormones were 60% less likely to be depressed and 73% less likely to have suicidal thoughts, compared to youth who did not receive these medications.

But according to this critique:

Among the kids who went on hormones, there isn’t genuine statistical improvement here from baseline to the final wave of data collection. At baseline, 59% of the treatment-naive kids experienced moderate to severe depression. Twelve months later, 56% of the kids on GAM experienced moderate to severe depression. At baseline, 45% of the treatment-naive kids experienced self-harm or suicidal thoughts. Twelve months later, 37% of the kids on GAM did. These are not meaningful differences: The kids in the study arrived with what appear to be alarmingly high rates of mental health problems, many of them went on blockers or hormones, and they exited the study with what appear to be alarmingly high rates of mental health problems.

Just wanted to share in case anyone sees these stats being quoted in the wild!

Thanks for this.

So compared to a "control" group who didn't recieve treatment they did better they think because they lost most of the control group

Abs of course the control group would have known they hadn't been given the treatment which would be pretty hard for a depressed teenager to cope with

Indeed the ethics seem a bit off kilter also

These poor kids. People with common sense know that for a child to want to take these extreme measures with their bodies, there must be some psychological issues. None of them are in fact ‘trapped in the wrong body’ (there’s no evidence of the existence of objective gender incongruence).

How is it ethical to give neither the control group nor the intervention group any treatment for their severe depression and self harm? At minimum the control should have been given the best available medication/psychotherapy for this. It should never be a question of 'this is better than nothing' when treatments are available. That is highly dubious practice, no wonder the controls dropped out.

Haven't the Tavistock always said it's not ethical to carry out a comparison study like this? If the non-medicated group are experiencing the same symptoms as the medicated group then they should not have medication withheld ( putting aside opinions on the type of medication we're referring to here).

Thanks for this. I read it on Jesse Singal's Twitter and meant to post it but forgot.

I am no statistician but it's clear that medical professionals were riding roughshod over ethics to try and get the outcome they wanted. Trying to justify why they want to medicalise and sterilise children. It makes me feel sick

After a swift read through, I'm flabbergasted and I don't know why. It's not as if this is an area with probity and integrity but I had expected better of the reviewers and the journal network of JAMA.

This is not the place for this discussion but the reviewer system for journals needs a huge overhaul. It would take time that so many people don't have to review some parts of the paper and when the authors won't share the data then there's nothing a reviewer can do but note that as a problem and state that it wasn't possible to validate that section or its results.

Thanks for sharing. Studies like this are so helpful when you are trying to refute fairytales and feelz.

Haven't the Tavistock always said it's not ethical to carry out a comparison study like this? If the non-medicated group are experiencing the same symptoms as the medicated group then they should not have medication withheld

The non-medicated group didn't have their medication withheld for the study. They had it withheld for other reasons. Quite possibly, because of their mental health issues.

I do not know how Jesse Singal manages to keep doing this, honestly. I read that article and then I skimmed the study and I felt like banging my head against the wall.

If I was tasked with patiently unknitting all the bullshit and deception stitch by stitch, like he keeps doing with all of these shitty and unethical abuses of the scientific process perpretrated in the service of the grand cause of sterilising depressed, autistic and homosexual kids, I think I would turn to drink.

there's nothing a reviewer can do but note that as a problem and state that it wasn't possible to validate that section or its results

The reviewers might also have been researchers in areas related to transgender medicine, so would have just waved it through with a loud round of applause. All the research in that area is always like this.

I read about this,too. The actual findings were that medical intervention made no difference to levels of depression or suicidal ideation in the group put on either puberty blockers or hormones. The levels (self-reported) stayed the same at a high level.

What happened to the control group (which didn't receive medical intervention) is what drove the results. Their self-reported mental health got worse over the twelve month study.

The problem with trying to understand that is that the dropoff rate among the control group was giant, and ultimately only a small handful of individuals participated in the final round of the survey.

What should have happened is recontacting everyone in the control group to find out how their mental health self-evaluation developed. Those who stop participating in the study may have done so because their problems got less serious. Or perhaps not, but we do not know.

The actually relevant result is that the self-reported mental health problems did not change with medication.

@Absurdle

there's nothing a reviewer can do but note that as a problem and state that it wasn't possible to validate that section or its results

The reviewers might also have been researchers in areas related to transgender medicine, so would have just waved it through with a loud round of applause. All the research in that area is always like this.

I agree. There is an article that has been mentioned a couple of times re. the genetics of gender dysphoria - there is no way it was reviewed by a geneticist. It has loads of unaddressed questions that a geneticist would have picked up immediately.

What I think it is actually very telling is the title of the original article: "Mental Health Outcomes in Transgender and Nonbinary Youths Receiving Gender-Affirming Care"

They know they cannot claim there is any improvement...

This is the article that is discussed here:

jamanetwork.com/journals/jamanetworkopen/fullarticle/2789423

The group which was given medication and the group which was not consists of youth who contacted a particular gender clinic. There is no information about the reasons why some were prescribed medication and others were not. It's really unclear to me if the comparison group has a greater attrition rate because the children leave the gender clinic or if they are given medication, too, and so join the experimental group (not the proper name for that group but the group on medication).

Note that the attrition rates are not mentioned in the main body of the article. They are in the supplemental information table eTable 3.

I strongly recommend having a look at that table. At the twelve-month survey the numbers in the control group (no medications) is six individuals who. The attrition rate is higher for the control group than the experimental group.

Sorry for the typos above.

Here is why that supplemental table e3 and also e2 above it are so interesting:
At the beginning of the study 7 children were on medication, 97 not. At the three-month point 44 were on medication and 41 were not. Some seem to have dropped out altogether at that point?
at the six-month point 59 were on medication, 24 were not, and at the twelve month point 57 were on medication and 7 not.

Confusing.

The outcome prevalence measures are also extremely confusing. The first contact, the three months point and the six months point show higher depression and suicidal ideation levels in the control group (non-medicated), but the six months point doesn't, really.

The outcome differences become huge in the twelve month point but the group not on medication is now so tiny as to make that comparison something driven by perhaps one or two individuals?

I should re-read the article.

So there isn't an intervention arm and a control arm? The control arm is those waiting to start and the intervention arm at 12 months has:
7 on medication for 12 months
Approx 37 on medication for 9 to nearly 12 months
Approx 15 on medication for 6 to neatly 9 months?
That seems back to front. It is normally not really relevant how long the study ran for (to a point) but rather how long each participant took part. So you would normally say there were 7 participants with 12 months followup in the intervention arm and 7 in the control arm. Except you probably wouldn't because that sounds very under powered.