you can’t have everything
Is really good advice OP. What are you looking for in a relationship? There are no right or wrong answers here, it depends on you and what you want in your life.
Is sexual attraction important? Some people have no interest in sex and are happy in more of a friendship relationship, than sexual.
If sexual attraction is important, then you have a problem if you don't fancy him.
In order for sexual attraction to arise (as it were) there needs to be something else there for that intimacy to grow. A good, decent, kind, man with a great sense of humour and sense of adventure, may mean you click on many different levels that lead to sexual attraction developing over time.
I'm attracted to humour. If I find someone funny, even if I don't find them immediately attractive, I'm half way there. If they are into anything creative eg art, writing, music - I'm another step forwards. Throw in kindness and I'm batting my lashes.
If there is nothing else OP and you don't find him attractive, then it's a lost cause. You're better off alone that lonely in a relationship with someone you wouldn't touch with a bargepole.
In answer to your question, yes attraction can grow if there are other elements there.