OP, what do you want? Are you interested in marriage (or equivalent) and children? I'm going to assume yes or probably, and I also don't know your age, which matters in terms of urgency and specific advice to give you.
What I would say is that one of your previous comments, "he was scared of getting that close to someone because he doesn't want to get hurt again" would be a red flag for me. Relationships that are even in the orbit of love and commitment would entail risk, by definition. This doesn't speak well to his courage, or his interest. He should be wanting to take you off the market and be willing to be hurt and accept the opportunity cost in order to do that, so that you can't meet someone else. If he was keen on keeping this relationship in the medium or long-term, I would expect him to do that. He may change his mind in the future, of course, but why doesn't he think you are worth that already? I suggest it's worth looking into that very closely. I agree with the other PP in that he seems to be keeping his options open because he thinks he might be able to do better?
You say this man is very good looking? Perhaps you are putting up with this situation to make up for a real or perceived difference in sexual market value?
It's worth noting, on average, women have the tendency to develop feelings for men they are sexually active with after some amount of time, regardless of the quality of the partner, and this is driven by biology and biochemistry, and is to some degree independent of the quality or sexual market value of the man. In other words, you can develop feelings for a partner who is not a good partner, provider, husband, father, etc. These are pre-civilisation, gene based triggers which promote reproduction. This is not the same for men, because the risk and opportunity cost of pregnancy and childraising is higher for women than men. This is partly why we developed values and culture including marriage contracts, no sex for before marriage, chaperones, more parent involvement etc. before we had the welfare state to care for mothers without partners providing the resources.
This is all quite obvious, I know - I'm just working through socratically to reach the conclusion, which is, that it's worth understanding what is going on with your hormones and biochemistry, so that you can make the best decisions possible and optionally override your "feelings" if you need to, and seperate them from the more holistic, intellectual decision of picking the best possible partner.
If you're only looking to have fun, you have one set of factors to evaluate, and it's less important, if you're trying to find a partner for life, I recommend giving your future children the vote and imagining what they want for their ideal dad. The latter approach usually delivers the greatest amount of happiness and fulfilment. Sex with lots of people is overrated, and has a very real cost. Lots of sex with the one person is so much more satisfying, if you pick well.
This ended up being an unexpectedly long reply, and I'm fully aware I've not focussed on your clear mention that you were fine with this being a casual thing, but I wanted to offer the wisdom that maybe you actually don't want a casual thing? I don't think it's ever too early to find the perfect partner, and wasting time with time wasters isn't worth it, in my opinion, becasue your choices will diminish over time.
Anyway, that's my offering 