It's quite possible, in my opinion. Public bodies and businesses aren't actually interested in 'inclusivity' because that takes genuine effort, time and money to understand the needs of different groups and find sensible solutions.
It's why at the moment simply replacing the Female label on doors with Gender Neutral is enough of a token gesture to win them woke points. It's the easiest and cheapest and laziest option. It gives the impression of being progressive and inclusive while doing no such thing, especially for women.
What should happen is maintaining decent toilet provision for women and disabled people alongside unisex toilets.
The thing is, unisex toilets don't have to be just for trans people. If queues are long and someone is in a rush, they can use the unisex toilet. Lots of woke and 'cis' people claim to not be bothered by unisex toilets, so they will also be happy to use them.
The unisex toilets can also be designed with baby changing facilities so that parents of either sex can take small children to the loo. At the moment, parents might struggle to take their children to a single sex loo as they get older, but a larger self-contained stall will help parents, especially those with multiple children, to use the facilities.
If you are a bit old or infirm and your carer is of the opposite sex, at the moment they might not be able to accompany you to a single sex toilet, but they can help you use a unisex one etc.
I'm sure there's loads of ways to design an extra stall that is unisex and can be used by different people for various purposes without infringing on single sex or disabled facilities. It's not something that only trans people would use.
It's deliberate laziness on the part of public bodies and businesses to claim that it's a tricky issue and deliberate strategy from TRAs to pretend like third spaces are not feasible.
I would highly recommend disabled charities and support groups (and disabled people) to voice opposition on any ideas to convert disabled loos into ones for trans people.
We all need better toilet facilities and it's so hard to get businesses and architects to properly plan for such basic needs.