He is poking the bear and you are growling to prove you are unhinged to his audience.
If you walked away from him, you are unfinished business and he is out to show that he was wonderful and he put up with you and you walked as you were trying to manipulate him.
He showed his true colours when he chose money over his kids. You have moved on in your life but you are clearly and understandably, bitter about your experience and he still has power over you and he is using it to his advantage.
He is purposefully winding you up and you are taking the bait. And I totally get why you want to tell his partner your side of things, I really do....but now isn't the time. He will just turn it around on you and use it to confirm how deranged you are. It will not end well.
Do not let him contact your kids by phone to make arrangements as others have suggested. He will weaponize it and will make arrangement convenient for him and are outrageous from your perspective and will sell it to the kids and make you out to be the issue. Do not do that to your kids, do not put them in the middle.
Use a parenting app, and keep records. He is a parent and he can't pick and choose between DDs good and bad days to take her. His wife should understand this. You need to keep your cool because at the moment, when you kick off, you are masking his bad parenting and diverting attention away from how he created the situation. I know this is hard but I'm sure you have heard the old saying about being nice or smiling at someone who is annoyed, because it winds them up....this needs to be your tact. He will pop eventually and in the meantime his wife/the kids may begin to see the pattern. But to be honest, wifey may never see what you want her to see because that would be admitting she fell for a wrong 'un.
You need to concentrate on your kids and their wellbeing and at some point they will be old enough to start joining the dots and you don't want to blur this by reacting to his bad behaviour and allowing him to continue distracting people from that by your behaviour. The best thing you can do is kill him with kindness, it will drive him nuts.
Keep cool, smile engagingly at him, get yourself a voodoo doll of him to stick pins in if it helps but be Mrs Super Cool and Nice. He will show his true colours and your kids will realise for themselves what he is like, that doesn't mean they will turn away from him or hate him, they may still love him but will understand the nature of the beast and will appreciate you all the more. You don't need to tell people what he is like, because once you stop masking his actions, it will become self evident but it will take a while.