I got together with my DH, when my kids were 11 & 9 years old. I'm going to be honest, it hasn't always been plain sailing, because they are not his children, and he doesn't have the patience with them that I have. Or the guilt of splitting with their real Dad.
That said, he has always understood that they are my priority, and would never have suggested that they live elsewhere! Over the years he has done many things for them that a Dad would do, like giving lifts, doing the school run, cooking for everyone, attending school plays etc. He has also been a sounding board for them, an advice giver and a voice of reason through the teenage years.
They are both mid 20's now, and they have a great relationship. One of them lives on another continent, and we visited over the summer and had a lovely holiday together.
Your children will never be his children. He will never love them like you or your Ex does. But he should be understanding that they are a priority for you and that he needs to have them in his life if he wants you fully in it. You can't spend your life straddling two camps. How would he feel, if you told him that in order to be with you, he had to stop seeing his sister/mother/father/friends? Or that you were not prepared to meet any of those people.
One of my friends was actually dating a man like this. The children in the scenario were his though, and he refused for her to meet them, ever after a few years. It didn't last. He just wouldn't merge the two camps, and she had to end it.