The way I deal with my ex is this, it may help you, I don't know, but I try to see things this way -
I see my job re: contact as giving him the opportunity to be a good dad. If he does do that, brilliant. If he doesn't, I don't chase. In practice this means if he asks for something (time with the kids, phone calls, to give them stuff from him etc) I do my very best to facilitate it. If he doesn't, I don't go offering my soul or chasing to extraordinary lengths in the hope it will make him step up. It's made me feel much more calm to know that my job is to be a good mum, and his job is to be a good dad. I can't make him do his job, so I focus solely on doing mine, while allowing him the chance (when requested) to do his.
I'll never, ever stop him from being a good dad. But it is up to him to want to be one. Perhaps, you could just make a decision in your own mind not to care about what your ex might do and just get on with your job (for now). In other words, say nothing. Offer nothing. Do nothing... Don't say yes or no to contact right now at all. In the future if he asks to be a good dad, you can work out how you will facilitate that for your DD and him.
Right now he's being a prize tit so you can safely ignore him until that changes, without being in the wrong. But if you make noises about not allowing him to be a dad, you may discover he becomes a bull in a china shop about his 'rights' without taking time to reflect on his responsibilities. If you give him time to think (whether it be days, weeks, months or sadly even years) then he may realise that being a dad isn't a choice, it's a job he's signed up for. And by not doing it, he'll only have himself to blame in the future.
I know it's hard, but this too shall pass and you won't feel so raw forever xx