It sounds as though your biggest fear is letting the DCs down by introducing the chaos of divorce (as you know it) into their lives.
But perhaps, as you've also said that any split would be amicable, things would be very different for your DCs in the case of divorce, than they were for you as a child. Because you and your DH will make sure you handle it decently, surely?
You care about your DCs and how they feel growing up. So, perhaps you could think about the values of integrity, honesty and respect as being helpful and decent things to pass on to them.
Also think about what you want to pass on to them about what loving, fulfilling relationships look and feel like.
How can you do this if your relationship with your DH is a lie, and your other relationship is a secret?
The path you are on is littered with deceit, secrecy, subterfuge, compromise. You are unable to have a full relationship with your DH, but you also cannot have a full relationship with the other man. Would you be happy for any of your DCs to grow up and live like this?
Do you not want and honest and open relationship with your DCs, both now and when they are adults? 
And putting yourself aside for a moment... what about your DH's life? and the other man's life? For you to do things your way, both men have to fall into line, and stay in line, forever... how likely is that? At some point one of them might want things to be different.