Except it goes way beyond "a few bad actors".
Referring to your rapist as "she", or be found in contempt of court, was written into the official guidelines for the conduct of court cases. It has been removed now, after campaigning, but it was official policy, not one or two " bad actors".
Official NHS policies and prison service policies, likewise, were influenced by transactivists and it was following those policies that led to men in women's prisons and men and women in opposite-sex wards, and the lack of proper procedures at the Tavistock.
Ideas like being born in the wrong body" have been actively pushed in schools, and that is only now being corrected.
"No debate" on trans issues has been pushed in universities, to the detriment of academic freedom, and has cost people their jobs.
Stonewall and similar organisations have peddled the line that women legally have to include transwomen in their spaces. This was not the law, and women had to go to the Supreme Court to get this spelled out. Despite this, many organisations are still dragging their heels over giving women their rights. Some organisations have now disaffiliated from Stonewall, but not long ago there were dozens of public bodies and private companies subscribing to them and listening to their version of the law.
You seem to have a naive image of a tiny, powerless minority. In fact, they have had the ear of the authorities for a long time.