Women absolutely aren't trying to exclude anyone from sports at any level - not men, not Trans people.
We just want to keep a category which is only for women, so that women can compete meaningfully. Why should women dedicate their life to training in their sport - to be the best they can possibly be - just to have someone cheat and take away any chance they have to win on the day? It's exactly the same as doping, which is banned.
A doped male athlete is the fastest of all. Are you suggesting we remove all restrictions and have a single open category where only doped male athletes have a chance of winning? Personally, I think that would be a bit shit.
Open category + protected women's category (no doping allowed for anyone) is fine by me, and gives access to everyone. Oh, and both categories should also have prize money, training opportunities etc - that's about fair access and opportunity.
It's true that professional Trans athletes may need to decide whether to delay taking hormones, and then may choose to retire earlier than they would like from top-level professional sports - balancing up how hormones will affect their chances competing. Much like professional women athletes may need to decide whether to delay having children, and then choose to retire earlier than they would like from top-level professional sports - balancing up how pregnancy will affect their chances competing (both during the pregnancy and also the permanent changes it makes to your body). Being a professional athlete impacts every part of your life.
At recreational levels, athletes should still compete in the fair categories. Less might depend on results, but it's still important to the competitors that recreational sports is fair. All athletes must accept that their life choices will affect their performance - whether that's taking hormones, getting pregnant, or having some drinks with friends - and that's ok.