What you are going through sounds really hard, Shorti. I'm sorry you're having such a difficult time.
I think you would benefit from some counselling, and even though you are now single, I think Relate could still give you counselling on your own, so do contact them.
Well done for being consistent and going out at 10 am so that when he was late, he missed you. The fact that he got cross was his problem, and you need to concentrate on remaining calm on these occasions, not taking blame when you have done nothing wrong. If you continue to be consistent about when he can come, he will soon learn that you mean what you say.
If you had lingered and waited for him to turn up late, he might not have turned up at all for all you knew, so you did the right thing there.
Just remember, you can decide what your behaviour is going to be, but you can't choose his behaviour. All you can do is organise your life so that his unreliability has as little impact on your lives as possible. And yes, you do need to leave the way open for him to continue the relationship with your dc.
If he gets cross with you again, try to be distant with him: don't let him feel he has any power to make you feel bad. Give him a chance to calm down; insist he speaks to you with respect or YOU will hang up on HIM.