I accept that you are referring specifically to what the law says, or does not say, about men using spaces designated for use by biological women.
But the law is not the only thing that keeps society running as friction-free as possible: there is no law that says 'It is a criminal offence to walk in a dead straight line down a pavement, causing other pedestrians to take evasive action so as not to be mown down by you', but normally we take the line of least resistance when walking through a crowded space.
There is no law that says that on the day before the last posting date for Christmas, a local business may not completely clog up the only local postbox with all their cards, leaving no space for anybody else's cards - yes that's a real-life example😒
There's no law that says an able-bodied person can't use the accessible toilet just because they want to, not because they are disabled and need to.
There is no law that says you must not tell random 4 year olds to get real, Santa Claus doesn't exist.
Another 'there's no law' social convention is that when a space is designated as being for the sole use of one sex, members of the other sex respect that.
'The good men stay out so the bad men stand out'.
Up to very recently, that worked very well - of course sometimes men entered the women's toilet, but it was usually by mistake - hence the 'Excuse me, this is the ladies!' challenge - and occasionally a man entered a women's toilet to do harm to women.
In both cases, the men knew they shouldn't be there, they were there through mistake or malice.
What's different now is a bizarre obsession with men deliberately and knowingly using spaces designated solely for women.
Saying 'there's no law against it' may be accurate, I don't know enough about UK law to agree or disagree, but being legal doesn't make something pro-social.