Hi, I actually don't think you should be too hard on yourself. This was new relationship not a long term one where you had both agreed together that he could focus his energies outside the relationship for a while. When you "dumped" (harsh word) him previously you made that decision based on the facts, circumstances and your feelings at the time and if you were finding him "unavailable" then I can understand how early on in a relationship that might make you feel dissatisfied and break it off.
Men who are unavailable can be very attractive - are you sure it is not that fact which is making you "pine" for him now?
I don't know how badly this is hurting you but you don't seem to have much choice but to do what he wants for the moment. I'm not one for believing that you should put your life on hold. Just get on with your life and if it is meant to be it will work out and if it doesn't then by the time you realise this you will probably have moved on anyway.
TBH this sounds like a relationship that is just Not Going To Work. For whatever reasons, both you and he are only really interested when the other one has run off crying and swearing, never again - and then when you get back 'together' it's the other one's turn to start acting up. Now some people might tell you to stick together to avoid spoiling two households, but I think you would both be better off drawing a line under the mess and moving on.
I'm not clear, though, about whether he's the father of any DC you have? If so, you are going to have to work on setting boundaries; if not then just cut him out of your life. Have one last shag if you must, and cry all over each other, and write some poetry about the One That Got Away or something, and then leave it be. You'll both feel a lot better.