I'm very similar. I passed at 20 and was 37 before I drove again. I always hated driving and it took 4 attempts to pass but I did it.
There is a big long story but basically my friend hit a pedestrian and killed them in their car around the time I passed. It wasn't my friends fault (inquest etc determined) but that was irrelevant really, the pedestrian was dead and my friend was devastated. This all got in my head I think and I couldn't drive. I honestly thought I never would.
Fast forward to three years ago and I had just had my second child, I realised that driving would make my life so much easier so I cautiously thought about giving it a try. Eventually I bought a car and figured I would just have to take the plunge.
I was so incredibly anxious for about the first year, I used to literally shake and have the runs on days I needed to drive, even after the driving part of the day was over. It was pretty crippling. But I kept going, I forced myself to do it at least every other day but kept my goals small - the local supermarket, the kids nursery and my work were pretty much it. I would park further away and walk to make sure I got a space I felt comfortable with. Parking was and still is a bit point of fear for me.
It has now been three years since I got my car. I still don't love driving and have still never driven on a motorway, but I drive regularly and no longer suffer the anxiety that I did. It has been very gradual though, but I honestly never thought I would ever drive again so this is huge progress for me and I can accept that this may be as good as it gets.
My top tips:
Force yourself to go, every day. You will hate it but I promise it gets easier.
Plan your routes, look at google maps in advance so you know what lane to be in and where the roundabouts and lights are. I still do this when I'm going somewhere new.
Pick three routes you can cope with and stick with those for now. Do them over and over until it becomes second nature. It will, honestly.
Park as far away as you need to to get a space you are ok with. I park at the furthest side of the car park still and it's fine, it really doesn't matter.
Listen to music if it helps, it really helps me feel less anxious.
Make your car your own. I was like you, my DH picked it and I didn't even want to look at it online or pick the colour or anything. I refused to call it 'my' car for ages and ages and never put anything in it at all. Now it's like a skip on wheels but it feels like mine and I'm more comfortable with it.
Finally, you don't have to love it and you don't have to suddenly go on a 500 mile road trip. I barely drive more than 20 miles from my house to be honest, but I do it and I never thought I would. Give yourself time and space to get over little hurdles one at a time.
Sorry that's really long but I hope it helps!