The above response is absolutely perfect
However if it's just about the interview, I've often seen people (especially women) talk themselves out of going for promotions because of imposter syndrome, interview anxiety and because they think people will judge them for going for it.
If you are completely honest, is it the job or is it anxiety?
If its not the job and its the worry of failing. You already don't have the job, if you go for it and fail you'll come out in the exact same position as if you withdraw
However there's a chance that you won't and you'll get it! There's a chance you'll exceed your expectations
I've interviewed lots of members of my team for promotions. Lots of them have got it. Some haven't
The ones who haven't, no one thinks badly off. We dont think they were ridiculous for trying. Some have then got interview coaching, tips or even help with being given development experience so the next time a job comes up and they really really want it, they can really thrive.
I wouldn't think badly for you if you withdrew but I always think it's worth trying.
Again different if it's genuinely that the job is a poor fit