Blossom keep an open mind but at arms length...
Gotta unless you're very patient (I'm not!) sexual anxieties could be anything from premature ejac to his ex criticised him in bed... neither are nice for him or you but for him to bring this up means he's either sending out a warning (and expecting you to understand) or he's using this as an excuse (I'm guessing) as to how/why not to or to get close to you...
You could meet up but I'd have to say I'd want these anxieties out in the open! Don't be sad by the way... 
My story so far... have had interest (wonder if any men read my profile...) quite a few older men saying I'm stunning etc which is nice to know have replied to one or two...