I think you're absolutely within your rights "to make a scene" about this! You've been with him for 23 years, married for 21 years, have four children by someone whose behaviour is now making you realise/think/believe that your entire relationship with him... was a fabrication. If he had any inkling that he was gay (or is he trying for the bisexual angle, I wonder?), then he should not have bound himself to a woman in matrimony. He should not have brought children into this world, whilst lying to their mother/himself about his sexuality - because that's not fair on, or to your children. How are they going to react when their father admits what he's done, or is he just going to spring a male partner on them at some point in the future ("Surprise!")?
When my grandmother left my grandfather for another woman (yes, I have the grandparents in the right order), in the 1950s, he reacted pretty badly by all accounts. For one thing, he "kept" their oldest child (my father) who was 4 at the time, for almost 3 years. My grandmother was permitted to take their younger two sons (aged 2 and a few months old at the time) with her. She wasn't allowed to see my father at all, until the point where my grandfather's mother talked some sense into him - my father, who was a very young child at the time, was distraught at and confused by the disappearance of his beloved mother and younger brothers from his life, and remains profoundly affected by the separation to this day. I don't think he even realised that my grandmother was romantically involved with another woman until he moved hundreds of miles away to live with them - and, lost his father/paternal extended family for 50 years as a direct result of this, because my grandfather was devastated by the fact that his marriage and the births/raising of his 3 sons were a sham. He refused to divorce my grandmother until my father was 15, and didn't pay any maintenance - and this was a pattern which repeated in his subsequent families. He left his girlfriends/wives and their children before they could leave him.
So know that you're not the only one this has ever happened to (which I'm sure you already do), and hold on to the fact that you're absolutely justified in making "a scene" - you're grieving, after all, the loss of the man you believed him to be.
It does get better. But, like most things, it takes time. I'm not sure my grandfather ever recovered - but back then? This wasn't a subject he could seek support in enduring, outside of his immediate family. You, at least, will be able to confide in and take comfort from your friends, which you're going to need. Especially as you need to be strong enough to support your children through this... 