There are many different types of people with disabilities who need to use accessible toilets. Causing them to wait while "any and all genders" exercise their right to use a newly gender inclusive toilet can have a variety of harmful outcomes for many if them.
Some may find they wet or soil themselves, some may become extremely anxious and stressed at the thought they may wet or soil themselves or that a catheter/stone bag may burst, some may be using crutches/sticks/other walking equipment and suffer increased pain from having to stand and wait.
My elderly Father uses a wheelchair and has dementia, he is doubly incontinent so potentially we will be waiting outside the only toilet we can use, with increasing foul odours coming from a pad which needs changing and he will be losing dignity from that and from my having to repeatedly tell him in front if other people that we're waiting to use the loo and change it. Of course all this may be true while we wait for another person with a disability to exit, but if we're waiting for someone with no medical need to use a disabled facility it compounds the frustration, embarrassment and general stress of the experience.
Disability makes every part of life harder, why are we expected to sacrifice the few concessions society makes in order to appease people who just want public recognition and validation of their "unique gender identity"???
That 60 seconds you may pop in to use it could well be over 5 repetitions of "why are we waiting, whats that smell, I feel uncomfortable" for us. So yes, I do think it is a selfish attitude and yes it does make a difference. Not to mention one person doing it normalises it for others around them.