Fransmom this is clearly taking it's toll on you both. You think you are being clear but he either doesn't hear what you are saying, or doesn't want to hear. Sometimes if you 'nag' someone enough, they just switch off. You had a go at him and basically said you thought he was useless and that your daughter wouldn't want anything to do with him when she was older and realised what he was really like. That's a pretty devestating thing to be told, even if it was by someone who was hurting and trying to hurt you too. Saying such things might make you feel better, and may even be justified, but they aren't helping you or your daughter.
Think about what is best for her, hard as it is to see beyond your own feelings at times your daughter loves you and she loves her dad.
Go to mediation. Try and find a way forward on this. Having a third party there can help you work through things and get your point across so that he hears what you are actually saying as well as you being able to hear what he is saying too.
Some people really don't understand that children need routine and stability. You have to make him see that this is important and that it is having a detrimental affect on your (yours and his) child to have contaact distrupted like this. Listen to his concerns and feelings, express your own, come to a compromise and a written agreement if you possibly can.
Agree the dates and times. Agree a time in advance that any changes need to be agreed in writing by (2 or 4 weeks in advance), except in the case of emergency (we're talking hospitalisation, severe illness, car accident, not mates birthday party here).
Get the agreement in writing. Do not tell your daughter he is coming until he arrives, then she won't be disappointed when he doesn't come.
Tell him you are going out and therefore he needs to collect her at the correct time or you will have to take her out with you and he will miss his contact. Then if he is not there within 20 mins of the agreed time, go out. Even if he sends a message saying he will be half an hour late then stick with it and say sorry but I made it clear I had to go out. Do not change to the following day or different time, this should have been agreed 2/4 weeks in advance if he wanted to swap. If he agreed the time then he should stick to it. Until you show him he is going to miss out if he can't be reliable he will keep doing this.
Make it clear that spur of the moment late return is not acceptable, any chages should be agreed 2/4 weeks in advance, save for dire emergency and unforseeable events like a puncture on the drive back.
If he is late bringing her back make it clear this is not acceptable and you expect him to leave (however late he is) earlier next time to ensure he is not delayed on the journey.
Make it clear to him that if he can not stick to the agreement you will take it to court where the agreement will be by court order and he will be in breach of the court order if he returns her late like this.
If he is significantly late again (we're talking 40 mins plus rather than 5 or 10) then you should ring the police and report this as DV.
You have to be strong and deal with this. Going to court to get the contact officially agreed, and reporting the late return to police as DV are things that you can do once you've exhausted negotiation and mediation. Don't be afraid of doing these things. Your daughter deserves a sane mother and a reliable father and to know that when people make promises (which is what he is doing by saying he will see her on a certain day at a specific time) that she can trust them to do what they promise.
And as for the communicating via email then maybe that is the best way to go right now when you are both upset and angry and saying things you don't really mean. If email is hard for you because you have to go to the library then use texts too. But avoid conflict until you ahve a third party mediator there to help you both get to the bottom of what you are both trying to express and say.
Keep strong, your daughter deserves time with her dad, but it can't be at his whim and cancelled and changed without mutual agreement. That is potentially damaging for her and is causing issues for you which again reflect back on your daughter. Get this sorted.
Best of luck
Gilly