Accusation - a statement saying that someone has done something morally wrong, illegal, or unkind
Insinuation - a sly way of saying something, usually something insulting
The two are different. And as we don't know what the cleaner said, it could be accusation, (oy! you're a bloke!get out of the ladies!) insinuation ( you look like a man to me, you're in the wrong place) or polite correction (excuse me, do you know this is the ladies?)
But as other posters have said, if men would just keep out of our spaces, we wouldn't need this discussion.