If the lurid predictions were true of men pretending to be trans and entering women's spaces in huge numbers, or shortlists for women being dominated by men then of course this would be a concern.
It doesn't matter if men are "pretending to be trans" or actually believe they can change sex, they are all male and should stay out of women's spaces and services. I do not differentiate between those 2 types of people - "genuine" or "pretending" is irrelevant, they are men. Just men.
How many women need to be impacted in order for this to be an issue? It seems like you are saying that as long as women are losing out in small numbers then it justifies allowing men to self-ID as women. If it is the case that either way, a small number of people will be negatively impacted by either the passing or rejection of self-ID, why can't that small number be the males who choose to identify as their idea of a woman?
Decisions have consequences. When a man makes the decision to present to the world as his idea of a woman, he should accept that one of the consequences of that decision may be he might not feel comfortable continuing to use men's single-sex services and facilities, but it does mean he has a right to women's facilities and services.
Remember, these men who feel entitled to use women's spaces and facilities have chosen the identity that they feel entitles them to that access. Women to not decide to be women, we are born women.
But nowhere where trans inclusion has been pursued has this happened
Please ask the women who are incarcerated with Barbie Kardashian in Ireland if women objecting to self-ID is scaremongering. Or the young women who missed out on their places in sporting competitions because of the likes of Lia Thomas and Laurel Hubbard. Can you put a number on what you feel is an acceptable amount of women who lose out or suffer as a consequence of male entitlement dressed up as a stereotype of what a woman is?