As for your examples and your insistence that we give up some personal freedom to live in harmony with others, the groups you reference were in a position of privilege over the other groups.
The disadvantaged groups there did not seek to take freedoms or rights away from other vulnerable groups but to create equality and equity of opportunity to redress discrimination or oppression by the privileged group against the vulnerable group.
Females as a class do not oppress males who identify as trans. We neither extract resources from them, whether paid or unpaid, as in the reproductive, domestic and emotional labour females are expected to provide in our patriarchal society nor are they subject to the same systemic oppression from birth that females are.
Females as a class also do not discriminate against males who identify as trans. We are neither dominant nor privileged enough as a class to have controlled the levers of power that created institutionalised discrimination.
And trans-identifying people do - or largely did - face institutionalised discrimination and oppression via the laws of our country which denied them equal treatment, allowing your career to be curtailed for instance or bullying and harassment to happen or housing to be denied.
Those laws were the laws of a patriarchal society, made by males to suit males and enforced by males. Not females.
In any case, both the Gender Recognition Act 2004 and the Equality Act 2010 changed this. The former is one of the most liberal gender recognition laws in the world, allowing those who identify as trans to acquire a legal sex that is different from their biological sex without requiring either hormonal treatment nor surgery. It comes with a 95% success rate for applicants and the right to reapply after a six months waiting period for those who fail. And that status change is protected by incredibly stringent privacy provisions.
The latter gives you the legal right to be treated equal under all circumstances unless this conflicts with the rights of other protected groups (which we endeavour to resolve in an often complicated and sometimes contentious balancing act).
So, if you believe that persons of goodwill strive to make progress for all vulnerable groups, you are clearly failing, because you advocate for one vulnerable group to gain rights by dismantling the rights of another.