Back story:
Recently got together with someone from the past - we'd become very close a few years ago, I fell in love with him, he ended up with someone else. They got married, I didn't see him for years, but 6 months ago their marriage broke down (she had had several affiars). Mutual friends noticed that we were talking about each other, asking after each other and so on a lot, and contrived a meeting.
At said meeting, we talked all night, ended up deciding to make a go of it. He said that he'd known at the time, all those years ago, that he was making the wrong decision. We were both utterly delighted to have found each other again and were initially inseperable.
After that first flush, reality kicked in and we saw slightly less of each other. But still several times a week, and had a great time. Also started telling friends, all of whom were very happy for us both and very supportive.
Well, for some reason, I started feeling very insecure. Told myself I was being silly, but just became full of doubt and nil confidence. I started to find it hard to believe that he could possibly want me, but I couldn't understand what had triggered this.
Then earlier this week, he suddenly came out with "Right, this is never going to work, we should stop seeing each other". I was 
and burst into tears.
He then went on to apologise profusely, said that he didnt mean it, was upset about something else, and realised that although he has some fears about the future (what if it doesn't work out between us? Am I the person he wants to have children with? He is 42 and really really wants kids. I am 37, so not much time if we do go down that route)
We talked and talked. I excplained how I'd been feeling too (the lack of confidence and insecurity). He said that he definitely did want to continue with the relationship. I initially said I didn't, because I couldn't cope with the fear of him suddenly rejecting me, but then changed my mind in the morning. He said he was really, really happy that I had.
But, but, but... I'm still feeling awful. This feels like the last chance saloon for me. I've previously had lots of crap relationships with ego-maniacs and narcissists. I always seem to end up with people who are unavailable, don't want me, or are using me to prop up their egos and so on. I'd recognise this, and this new chap couldn't be more different. He's kind, gentle, very affectionate, sweet, funny, intelligent, gorgeously sexy, great with DD, and very, very caring and supportive.
I feel that if this doesn't work out, I will just give up on relationships altogether. It's all just too stressful and heartbreaking.
What can I do to get over this hump? And can anyone shed any light on what he's thinking? Because I'm too full on confusion and self doubt to see clearly.