For me, location is paramount.
I wouldn't buy anything in a large town or city. I love my little market town (population 3,000) and that's big enough for me.
Wouldn't get anything on a busy road, or without its own parking for at least 3 cars.
In terms of the house itself, I wouldn't look at a house that had no downstairs loo, or one that had a downstairs bathroom.
This is a strange one, but I wouldn't buy a red brick house. Lucky enough to live in the Cotswolds so all the homes here are made of Cotswold stone. And having previously lived in a street full of red houses, the sun shines so much more nicely off the yellow Cotswold stone. It just feels a sunnier / happier place to live!
Wouldn't buy a Victorian terrace either. They have all the things I hate in houses, and often dark rooms with small windows. If I had the money, I'd happily buy one of the lovely Georgian houses in our village, or the ones nearby, but not Victorian. Too dark inside and usually made of red brick, in depressing row upon row with no parking.