OP, I could have written so many of your posts.
Your xp doesn't appear to be interested in consistent contact to benefit your DD. You have offered him that and he didn't want it.
My xp and I split when my DD was 6 months. I took the matter to court, gave xp the contact he asked for in court. He stuck to it for as long as he had access to his mothers car, then he stopped.
There was absolutely nothing I could do to make him see DD. He could have done, but he didn't bother. He put so many obstacles in the way, he was too busy, he was working, he couldn't get to mine to pick DD up, he didn't have the things he needed to look after her for contact.
I agreed to drive DD to him and pick her up, I agreed to supply him with everything she needed, I agreed to change the days/hours to suit. It wasn't enough. He ended up saying 'Oh it's just too much bother.' 
I continued trying and trying. He continued playing headfuck games with me, so I returned the matter to court again. The judge knew he had let DD down consistently for about 7 months at this point, yet not a word was said by the judge to tell xp that it wasn't acceptable, not a word. The only person who had an opinion on xp's behaviour was my barrister, who asked me, in all seriousness, 'Has he always been such a clown?' XP was merely offered different days/times that suited him. He chose every other weekend, all weekend. It was seen to be usual for a 13 month old to have contact overnight with a father she barely knew. Court order granted.
Contact resumed for approx a year, then XP just stopped bothering again. He had split with gf and met another woman. He didn't contact me to arrange seeing DD for 10 months.
By this point, DD was almost 3.
Contact resumed sporadically, and has remained sporadic until this day.
I never wanted to deprive DD of her dad when we split. That's why I bent myself into a bloody pretzel shape to facilitate contact. DD bonded with xp and I thought she missed him. It must have been strange for her to suddenly wake up and he wasn't there at 6 months old.
I didn't realise that at 6 months, her memory would be short, and by facilitating contact with a man who isn't interested in DD very much, (although he could fool anyone he loves her to bits) all I actually did was create a situation where my DD knows her daddy well enough to crave his company, well enough to cry for him, she does remember daddy for a very very long time now (think 6+ months at least), talks about him every day and waits by the window for him. 
I have merely postponed the pain for DD, until she was old enough to understand how much it hurts even more if that makes sense.
If I had my time again, I would let him make the effort, knowing he would probably never take me to court (too expensive) and wait until he had proved himself to be interested in building a relationship with his DD that was consistent and regular, regardless of what else he has on.
I definitely would not take him to court myself. It will not make him stick to contact if he doesn't want to.