Also, affirming trans identies does not exist in a vacuum. It has consequences for other people as well.
If society agrees something in a man's mind means he's really a woman, it is unavoidably also agreeing that it is something in a woman's mind rather than the reality of her body that makes her a woman. And that unavoidably reduces the scope of womanhood to just a subset of our actual mental cabilities and possibilities. Many women do not see ourselves as simply a mental subset of humanity and it is not at all fair to force this identity on to us.
When society agrees that such men should, for their own comfort or mental health, be accomodated with women rather than men in single sex spaces, opportunities and language, it is unavoidably also agreeing that the women in those spaces must accept sharing them with men regardless of their own comfort or mental health, regardless of whether those women do in fact have the same belief that womanhood is a mixed sex mental attribute which they also see in themselves or not.
Many women do not consider a male bodied person to have a body that is somehow metaphysically like ours and therefore actually a female body because of how he feels inside, and it not at all fair to force his touch onto us when we ask for female care.
Distress and mental health issues are so often caused or aggravated by circumstances. There are many things society could do to alleviate this for individuals that we do not do because of the unfair demands or consequences for others.
The incels would be happier if women were forced to date/fuck them. The homeless would be happier if homeowners were forced to give up their spare room to house a stranger. The alcoholic would be happier if someone else fed and housed them then left alone to drink. Millions of abusive men would be happier if their wives just smiled and obeyed them in all things like a Stepford robot.
We don't sacrifice one group to the unreasonable beliefs of another in thise cases and we should not be allowing sexist or disordered ideas about what type of mind is "right" for men and men to sacrifice female identity, autonomy and rights.
Once you look at the whole picture rather than just considering the male half of the picture, you realise it isn't a cost free accommodation at all, but a huge and unreasonable demand on women to accomodate the needs of some men.