Man's perspective as someone who lived and worked in France in his early 20s with English girlfriend who was less fluent in French: from your description, you're both being unreasonable, but you probably more so.
In the open-plan office I worked in, it was commonplace on arrival in the mornings to go around and greet people I barely knew and often it was the case that girls would kiss me on both cheeks when doing so.
As much as I might like to claim this was because I was so devilishly handsome that they couldn't resist getting close to me, it wasn't: it was cultural.
Not all girls did it, and by the same token, not all the other boys shook hands either, but a large proportion did and this was despite me not knowing them.
Going topless in France is much less of a big deal and as others have suggested, it does seem to be as much about the girl in particular and your own insecurities / internalised misogyny.
However, having said that, I do think your husband could and should have been more receptive and understanding of your concern when discussing later. However, you surely must bear in mind that he did go out of his way to invite you along too?
I think you'd have had much more reason to be concerned if he hadn't. Unfortunately, he's now possibly got more reason not to invite you along to anything social with colleagues in future.
Oh and the her not speaking English thing: that's totally acceptable and you being totally unreasonable, unless you expressly said to her (apologetically) that you don't really speak French yet.
And even then, it's still not necessarily on her to accommodate you entirely. Assuming that she's capable of speaking English to a level she feels comfortable.
Wherever it is you are in France, I hope you and your husband are able to embrace all aspects of life and the culture... just not literally physically embracing other people!
As others have suggested, try to immerse yourself in the French language as much as possible: TV, radio, etc. It really is a great way to improve your comprehension as much as anything, especially alongside some formal learning. Courage!
(Courage or bon courage being the French way of saying good luck for something that requires effort... although bonnne chance would also be understood)