Wow. I can’t exactly believe this, nor can I get my head around the fact that so far, a majority agree (as of this point, the vote was 70% YANBU).
I mean, relationships are difficult. Finding someone you’re truly compatible with is hard. You could find someone that makes you feel great, has a laugh with you, meets your emotional needs, cares about what you have to say, doesn’t appear to be abusive or controlling or otherwise a bad bet (so far), and you’d strike him off because he’s... too attractive? Yeah, sounds a great plan, that. Likewise those who say they wouldn’t even give someone who is “too” attractive a chance. I mean, when I was dating, I admit I had some definite deal breakers but, “Sorry, mate, but your face doesn’t look enough like a slapped arse so I can always guarantee I’ll be the more attractive one,” wasn’t one of them 
So let’s say you make this decision to always end up the most attractive one in your relationships. And then, God forbid and I’m knocking on wood, something happens, and you end up with very severe facial disfigurements. I hate to break to it to you, but the gorgeous bloke might be the one who stays and still tells you that you’re beautiful every day after that, whereas the “handsome but average” bloke you were sure was safe will decide he can’t handle it.
How do I know? I’m living it. I did OLD way back in the dinosaur days and used a service that chose who to match you with, as opposed to just letting you choose for yourself. When I first saw my DH, I may have thought the same as you did... then gave my head a wobble. Yes, other women do find him attractive, but they’ve never been disrespectful to me. He and I did find it a bit funny, though, that neither of us are jewelry people, but we decided to get him a wedding ring after a year or so, because it helped stop women who would start up a conversation with the intention of asking him to coffee and then be either mortified or think he was lying when he said he was married. (I’ve heard the tales that a wedding ring is a ‘come on’ for some women, but we’ve not had any issues since). But I’m living ALL of the scenario I mentioned. I got extremely sick, and our life has not worked out as we planned. My body is definitely not as attractive as it was when we got together, and while I hope that I may get back there, the truth is, I may not. I really think every average to average-attractive man I dated before him would have run for the hills screaming left long before now.
So if you haven’t started dating yet, and you haven’t gotten that close, I suppose it’s your choice. But if he’s promising in every other way, I think it’s terribly short-sighted to bin someone off for this reason. Of course, it also means by your logic, that all the men you’re fine with dating should refuse to date you, because they want to be the more attractive one in the relationship 