Couple of things:
Agree with posters saying the laugh with teachers thing wasn't ideal. Understand why other posters are sticking up for it, but I'll give you an example where the tables are turned to put into context.
I'm divorced Dad and kind of have opposite problem to OP - I can and would like to see more of DS (ideally 50 / 50) but my ex doesn't want me to for various reasons. As a result I feel she is a little controlling over him and his time (not here to talk about that, know there are arguments from her side, just putting things in context). DS goes to nursery and has recently transferred from nappies into pants. As part of this process the girls at nursery were asking me (and ex) for ages if we could move him on to pants. I was asking my ex but she seemed a bit reluctant. So one time when the girls were saying "we really think it is time", I replied to them by saying "yeah, I really don't disagree, but you know what ex is like, have talked to her but she'll be making that decision when she wants to not when someone else suggests it". Reaction - knowing looks and some giggles.
About 30 seconds after I walked out of the nursery room I knew that had been wrong. In essence I'd let my personal feelings and frustrations spill out in front of carers who look after our DS and whether she was there or not, being derogatory about her to DS's carers was wrong - however much I feel what was actually said wasn't all that unreasonable.
And that's what's happened here. Yes OP's ex is clearly a toss pot in many ways, but he is still the child's fahter and encouraging influential people in his development to think less of him, however justified, isn't going to be helping any quest to try and get him to realise his responsibilities. Much more likely to drive him away because if he does ever turn up at the school he may sense those feelings from the teachers and not exactly feel encouraged - and let me tell you as a single Dad those traditional areas of "mums" (nurserys, schools) can be intimidating enough as it is.
Plus isn't the golden rule of a parental split to try and keep the children away from the personal feelings because if they pick up on them it gets damaging for them.
So, lots of words, but do feel quite strongly here that just saying "he's a toss pot he deserves it" is too simplistic.
Having said all that, getting back to OP's original problem, which is unreliability and lack of responsibility I'd consider some form of escalation if it's causing problems. Essentially he's messing you around OP and that's not fair, so don't think anyone could argue with you escalating to a situation where expectations are clearly defined and set out.
Think it would be good to go through steps as part of that escalation to give him every chance, i.e:
- If you keep being unreliable I'd like to define your contact more formally, 2. You are continuing to be unrealiable after I've aksed you not to be so now would like to formalise contact through mediation
- Court if mediation isn't forthcoming
In that process it's possible he'll either reveal he really isn't that bothered (sad, but good to know and then you can stop kicking yourself) or he'll realise he does need to step up and stop messing about.
Essentially he's messing you around OP and that's not fair, so don't think anyone could argue with you escalating to a situation where expectations are clearly defined and set out.
Some will say avoid legal processes because it's "bad" and causes "conflict" (which it often does), but if nothings changing what are the other options other than keep on putting up with erratic behaviour - and that's not fair on you or your child. And responsible parents will always put conflict behind them because ultimately they'll realise it's damaging for the child (i.e. if he does feel upset eventally he should calm down and realise it's not about "him" - that's the theory anyway!!!)