I think you have a right to happiness and your children need to learn to respect that, especially the older one. Yes, it wasn't the best thing to do to have an affair, but as you said, you planned to leave your husband anyway and, if they don't know that, I think they should be told.
It's very difficult to see your mother with a man who is not your dad, but that may well have been the case even if you'd left your husband prior to meeting this man.
I think it would be different if they were much younger and still very dependent on you, but at the ages they are, I think they need to realise their own lives are / will soon be moving on, and there's no reason for you to sacrifice yourself.
It sounds like you keep your relationship well apart from your children anyway - maybe only the 16 year old is directly impacted? You probably need to continue that for a few years, but ultimately why should you be alone just because your grown up children aren't happy about you having a relationship?
In time they will understand more and their negative feelings will likely decrease.
As for the idea that men come and go, well, no they don't necessarily. Not everyone finds love easily.
I think some posters are being overly harsh. Yes your children come first, but you haven't pushed them away deliberately or stopped being their mum. They can't expect to be the centre of your world forever, but they will still be able to have a relationship with you, if they wish to.
Sounds like your ex might have poisoned them against you - if it were me, I would stand up for myself and tell them my side of things (without trashing your ex, of course).