Can a man claim he has the right to be in the women's toilet or changing room?

Can a man claim he has the right to be in the women's toilet or changing room?
Lets flip it. If a business offers a toilet or changing room and labels it 'women', can customers expect it to be single sex and for women only? The EA suggests we can expect some facilities to be single sex.
I think the onus should be on the business to label their toilets and changing rooms accurately.
And if they want to make them mixed sex they should explain how they will keep their female customers safe from flashing, voyeurism, filming and hidden cameras.

If, say he was accompanying his disabled wife and there were no disabled facilities then that'd be okay but I'd expect an ' excuse me do you mind' opener.

Context is everything

To deter these males from violating the boundaries of females against our consent we are going to have to demand that single sex spaces and services be made mandatory.

When it comes to toilets, changing rooms etc that boundary exists at the door on which there is a sign that indicates females only. Any male who crosses that threshold is violating that boundary against our consent.

Those males who claim they are protected by "gender reassignment" also claim that this protection gives them the right to enter women's spaces. The law does not say this; which is made clear by the existence of the single sex exceptions wherein it is stated that those who claim the protected characteristic can be excluded on the grounds of privacy, dignity and safety for females.

Those single sex exceptions were not intended to be ignored, they were intended to uphold the social contract that understands that people need privacy, dignity and safety. That social contract is being dishonoured. This is why we must now demand the mandatory application of single sex spaces and services.

I’ve found that many shops have the kids attached to the men’s department. It’s bloody annoying.

Seems v odd that the men's changing room was attached to the kids' department but I think you should have gone to the women's changing room, sorry @Smartiepants79

I have been thinking about this recently as twice in the last month or so I have ended up in a men’s changing room with my children as that was the one attached to the children’s department.
Both had fully locking, proper door changing rooms but it did make me question whether I should be in there.
The alternative was a trip down the escalator to the other side of the shop to find the women’s.
Staff didn’t seem to be bothered but I’m still unsure??

Understood, but it makes no difference, since the issue is sex. The act also makes that clear because even a GRC doesn't change things.

I think if a space is described as female single sex by the owners but they allow or encourage opposite sex use, then they might be liable for any sexual assaults or voyeurism etc which occurred especially if the woman/girl didn't know in advance. And possibly a claim for (indirect?) sex discrimination or discrimination based on religion? But I repeat, IANAL.

IANAL, but I think that if the space has been designated a female single sex space, a TW does not have any legal right to be there. No male people would be entitled to go there. But if the owners of the space had not done that (and in most cases they don't have to, the default position is that anyone can go anywhere unless the owners choose to discriminate on the basis of sex which they are permitted to do in pursuit of a legitimate aim (privacy, safety and dignity are given as examples of legitimate aims).
So no, TW and other males do not have the right to be in spaces set aside for the opposite sex.
Wrt the social contract, it has unfortunately been broken due to the demands of TRAs based on erroneous Stonewall law. So there may have to be changes in the law to make SS spaces mandatory and/or penalties for owners in order to nudge us back to the previous honour system. Yet another social good destroyed by transactivism.

The EA states that places may segregate such spaces by sex. But in places that do not do so, does that entitle people to enter the spaces of the opposite sex?

I think it does (entitle them). Single-sex spaces aren't mandated in law - they're determined by the organisation providing the facility. So a gym chain, for example, could decide to have mixed-sex changing rooms, a hospital could have mixed sex wards and a theatre could have mixed-sex toilets. They legally have the right to make them single-sex, but they are not obliged to make them single-sex.

In Scotland, he just needs to say he identifies as female.