Interviews are two way things. I learnt this in a job interview where they were really disorganised and I thought, "I don't care what the outcome of this is," (even though I was quite desperate for a job at the time,) "I really don't want to work for someone who can't even sort themselves out when they're interviewing people, and can't be bothered to put over a good impression of the company."
It's why interviewers usually ask, "Do you have any questions for us?" Most employers recognise that there are things about working hours, flexibility, location, benefits or whatever that can give the job the edge over other ones in the area, or that there could be a candidate looking for the opportunity to develop skills in a particular area of the job, something like that.
I've turned down job offers in the past. One was because I had two offers at the same time, and the offer I accepted was better. The other was more recent, and the role wasn't sufficiently different nor sufficiently better paid than the role I was already in, to make up for the much longer commute, the office that had no natural light and a couple of other things. If I'd been unemployed, I'd have jumped at it, but they didn't make a good enough offer for me to resign from the role I was in.
It's fine to turn down job offers, and it shouldn't affect your reputation unless you do it a lot, with every offer you get. Most of us don't make enough applications which get to the point of interview, far less job offer, for that to happen.
I'm currently waiting to hear if I get a second interview for a role I've applied for. I got on well with the interviewers, I could definitely do the job, they seem pretty keen on me. It's similar to what I do now, and my main reason for applying would be to get away from one of my current managers, who is a bullying, incompetent arse. It's not because I particularly want the other job. It would be a good job. If I get an offer, it would depend partly on money. But the thing that most puts me off is whether I really want to commute into London. It wouldn't be completely unbearable, it's certainly not unfeasible, but that part of it definitely wouldn't improve my quality of life. I'm going to take the process as far as I can - I may not get a job offer at the end of it anyway. But they might be able to sell it to me (I'm working on my list of questions about things which I'm not sure about), and even if they don't - interview practice is always useful.