I once ended up dating someone who I really didn't fancy at all when I first met him, saw him looking at me with a group of others - my first thought was " you've no chance mate".
After being friends for 6 months, I ended up seeing beyond that and liking him for his personality, and even fancying him I think.
But, being a partner is different to being a friend, and in the end I lost respect for him because of some of his behaviour, so after that was gone, there was nothing else to glue us together, as I stopped seeing the attraction.
Last BF, instant attraction, he's perfect for me looks-wise, but still I had to end it over questionable things he did, so the rest was not there. Still fancy him, but avoiding him now as been on/off a couple of times, just because of strong chemistry I think, though we would make for good friends if he wasn't so damn attractive to me.
So, the ideal would be a combo of both, but, if you like a person for who they are, and they are able to maintain your respect, it's totally ok to go with that as, ultimately, respecting them is what counts and is enough on its own to hold it together. If you still respect each other years down the line, that's a precious thing.