Ok I can understand with your updates why you wouldn't necessarily want to call this bluff. Him having money, for example, does make it easier for him to use a court to harass you.
However. I really think there is very very little he can actually do here - or really wants to. Leave contact out of it for a moment. What he really hates is losing control. He lost that ages ago of course, but this new piece of information showing that yes you HAVE moved on has riled him up again. So he's trying to flex his muscles.
However- all he can do is try and intimidate you. He cannot DO anything. You can ignore the email. With a previous non-mol too, he's on totally shaky ground by even contacting you like this. He knows it, too, which is why this is all he's done - it's as far as he dares push it. He won't try and see you or come to the house. He's just trying, very hard, to scare you - because that's all he has.
So first thing to say is - you could ignore this, and all would very likely be fine and he would get bored. I can see however that you need to shut the door on it as it is getting to you. So here you could respond in a way which lets him know, very firmly, that if he starts up his harrassment again you will nail him:
Hi X. I'm afraid I don't understand your reasoning for needing to know any details of my private life, especially as you have actually remarried yourself. If you wish to put your arguments to a judge, I will be happy to provide all details of your previous harrassment of me as required should your solicitor contact me. Please do not contact me again directly on this matter as it is inappropriate and threatening. Regards, Astrantia.
Now as regards the children and contact. The thing that jumps out at me is that, despite having the money, one thing he's chosen not to do is to use the courts to harrass you for contact. This is the typical way in which abusive men pursue their exes when all else fails - because it's the one area in which they have the 'right' to continue to force dialogue - he has the right to see his children. Yet he hasn't. You've hardly seen him.
So - bottom line is, he doesn't want to see the children. He doesn't want the bother or the expense, and he doesn't have family behind him pushing him to do so either. Oh he threatens it, like now, when he's got nothing else to pull out of the bag - but he doesn't actually want to. So, while I understand you not wanting to poke the beast, don't forget that right now he has the life he wants and it doesn't include his kids. So while I understand you wanting to stay quiet on contact issues, I think you should take heart from this and know that these are almost certainly empty threats.
If I were you, I would respond to the demands regarding your private life as above, don't mention the children, and that in itself MIGHT get him to back off as he knows full well there is NO WAY he can pull a court case up to force you to let him grill you about your partners. Put a history of harrassment and a non-mol on top and he wouldn't dare go near a solicitor for that. So you tackle his demands head on and let him know that you will NOT be harrassed.
If he then drags it around to the children, you can blandly reply that he has of course every right to go back to court and he will be pleased to know that you've kept records of all contact broken etc. so fine.
But I don't think he will go there because more contact is clearly not what he wants- he wants to use it as a means of forcing you to cow to him. So let him know first that that isn't going to happen, and I bet his 'interest' in going to court for contact wanes.