The thing is, posters have mentioned feelings and it's easy to get swept up in those. I suspect you already are.
But in the cold light of day, feelings simply aren't enough in a situation like this.
You may think you're in love with him and it will work out, but that's a one dimensional view when going into a step parent situation. He may have feelings for you, but he will also (hopefully) love his children more. So in the emotional pecking order you're already No 4, which is as it should be. But at your age why should you be? It's an entirely different situation if you have kids and you're coming into this on an equal footing of understanding the setup, but you're not.
Every decision made going forward that will effect your life will be shaped by the emotional pecking order and if it isn't, then he's not a good dad.
You also realistically don't know what his ex is like or how she will react. She could make your life an utter misery by controlling your relationship through the kids if he's not strong enough to stand up to her. Just go and read the step parent board.
I've been a step parent for nearly 20 years and it's a hard path. Find someone to build a new path with, don't compromise your life.
Also, I'm guessing if you met him at work, he's senior to you? think about that power dynamic carefully.