I've had worked in a contact centre before and I know how very very difficult it can be so you have my sympathy, specially with two buses to get there even!
However, my view is that you should make the effort for your child. As you say yourself, you know it is important for a child to know both parents.
I know you might not like to hear this because of the incredibly difficult time you have had with your ex and his family but I'll say it anyway: It IS really important to your child that you give him the opportunity to have contact with his dad. His relationship with his dad has NOTHING to do with your relationship problems - unless you and your ex make him aware of the problems he won't even know about them at his age! So if your child becomes aware it's because the adults choose to involve him - very very unfair on a child, because he will only get one mum and one dad and his full emotional development will be best served by getting love from them both, regardless of what happened in their own relationship. We need to know where we came from for our own sense of self worth, self esteem - and as your child gets older it will be more and more important to have involvement from his dad.
Perhaps think of the contact at the contact centre as your chance to have a clean slate and give your child this opportunity. if your ex screws up or doesn't turn up then that's his choice and you have no control over that - but at least you'll know you have done your best for your child.
And I have seen parents make remarks to their child about their ex and please never do this to your child, it's so so cruel to involve them in this way. The key is to see their relationship with your ex as completely seperate to yours - UNLESS your ex is abusive to your child obviously! Though in a contact centre it should be OK.
Sorry to bang on but I have seen very damaged kids in contact centres because of hatred between the parents which has been transmitted to the child. I know you want the best for your son and I know you have no control over what your ex does, so I'm not saying it is all down to you - quite the opposite - as long as you approach it positively you've done all you can. If it goes well, I hope I can re-assure you that it IS worth it for your son and his future.