"unless I'm willing to forgive him there's no point."
Ahahahaha, love the way he's making this your problem!! He's not going to change. I know it hurts like hell, but you're making the right choice.
The thing about apologising for something this big is that it is completely meaningless unless there is an intent behind it to change and make reparation. You've spent years waiting for this to happen, and nothing has shifted. In fact, his behaviour now is of a piece with his whole problem - passing the buck, putting the blame on someone, anyone, other than him.
What you say about working together towards goals is vital. Like you, I was in a relationship with a man-child for a long time. He was all kinds of fun as a friend, but when it came to building a life together, he was comprehensively useless. And this problem got worse the more 'adult' steps we took - it was kind of disguised while we were younged by our comparative lack of responsibilities, but it became more and more evident as time goes on. The crunch came when we bought a house together, and he was just useless at getting anything done. He just wanted to play, all the time. Having a marriage, a child, a home with a man like that is a nightmare.
It was absolutely terrifying to end things after so long, but it cleared a path for me to meet the man who is now my DH - who is the complete opposite: responsible, kind, practical and able to handle just about anything (he actually enjoys getting stuff done to such an extent that I'm the one reminding him he needs to relax and have some fun too!).
I am sure something similar will happen for you. And maybe in the long run, you can be friends with this ex. But not now. Right now, keep up the no contact, stay strong. Weekends can be the hardest time when you are newly single, so do try to keep busy. Sending love.