I could have written your post. Mine is 9 now and if he had his way, we'd go to bed, in my bed, together every night.
So, I let him do it once a week. The other nights, and this started when he was about 6, I'd spend half an hour or so with him, get him all relaxed and send him to bed.
Sounds simple. It wasn't! We redecorated his room, bought a new, bigger bed. Bought lots of various lights, make sure he has a drink at the ready, make sure he's into a good book, and we discovered he likes his door completely closed so he can't see the 'scary' top of the stairs, or hear us because then he knows he is missing out.
We talked about it a lot during the daytime. We bought a book called what to do when you're scared of your bed or something. We talked about the fact he could come into our bed in the morning for a cuddle. He still moans about it, but as he's gotten older, he's got better at it. Especially when I show ho that he can read, or draw, or listen to something while he's in bed. As long as he's lay down still.
I also like the bedtime meditations by Christianne Kerr on Spotify. There are a couple of meditations she's done for kids and they're great. Worth a try!
It's about gentle perseverance. I found that if we did anything too quick, or made him too upset about it, he would regress. So slowly, slowly. Gently.
As an aside, as soon as he could talk he asked me why we put him in a jail bed. He meant his cot! He always hated his cot, and to hear a 2 and a half year old call it a jail bed (because of the bars!) Made so much sense! Bless him, he though we were putting him in jail! No wonder his sleep associations are off!
Good luck, not sure if any of that will help you.