Okay, this may sound harsh but to be honest, I'm a bit furious on your behalf right now. I'd back way up and start with your statement that his regularly flirting with other women is okay and just part of his personality.
My feeling is that you're being majorly gaslit right there. This is further shown when he continues with this same MO when he's caught at more than flirting. He's telling you that YOU that did something wrong. And you're halfway believing it.
When someone says "Oh, I'm just a big old flirt," I instantly dislike them and, as a married person, I get rid of them. Because what it translates to, to me, is "I get a thrill out of crossing boundaries, with no respect for my or other people's marriages. The usual social rules don't apply to me. I seek constant sexual validation from strangers. I put out sexual signals wherever I go." Single people flirting with other single people, fine. Otherwise, ick, no.
After all, what is "flirting" but putting out sexual hints? That's exactly what it is and he does it so much that it's "part of who he is." That's a problem. Whether those sexual hints are taken further or not is up in the air at that point but he is definitely, deliberately putting those signals out there.
Who needs it? Your partner is supposed to have your back and protect your heart and your relationship. Constantly giving sexual hints to other women is the opposite of that.
And, as is to be expected when someone takes step one on a journey, that invites consideration of step two, then step three. These little "whoopsies" of his, well, he asked and he received.
You've also had multiple other people tell you they found his behavior with them inappropriate but apparently, you felt that they just didn't understand his extra special personality. You may want to rethink that.
I suggest marriage counseling or finding someone with a better personality. Best wishes to you.