OP - I had to reply as I was just like your son when I was his age - really terrified of the supernatural and in a bit of a world of my own. I think oversensitive children can appear naughty to other people, but they live in a bit of a world of their own. His behaviour is not an attack on you - and probably more than other children he needs complete consistency - and more than that - security. So - a lot of discipline doesn't really suit these children - they need to be talked to, held, comforted, have everything explained to them - without being made to feel they are wierd. You need to come across as together and comforting and very clear about your messages - and they have to be loving but firm.
It's an odd thing to say, but it can be hard work being a child when you are like this - it is hard when you are over sensitive/ big imagination - being very small but having big thoughts !
I remember finding life easier the older I got - people talk to you more like an adult, your friends are more mature - so, just because he finds it tough now, doesn't mean he will always be like this.
One last thought I totally sympathise with your son about wanting to switch his mind off - I still have that problem! What about meditation techniques?
My mum bought me a relaxation tape when I was about that age - it was a life saver, I listened to it in the evening before bed - very simple stuff, aimed at kids, imagine you are in a lovely field, trees everywhere etc etc....but I can still remember how calming I found it. I had a quick google and there's lots of stuff for children.
www.learningmeditation.com/children.htm
I used to lie in bed imagining the devil was going to get me in the night!...but that really helped...
Another calming idea could be yoga? Childrens yoga may include meditation techniques that he could use at home...or you could do together..
Exercises with dad sound a great idea - but marching up and down the road does not sound fun - why doesnt your DP sit down with your son and together they can create an exercise plan, that they can go and do together. That way it's not a punishment - exercise should be enjoyable anyway, not equated with punishments.
another thought - if he feels cleverer than his friends - is there perhaps a weekend class he could go to for particularly bright kids? Not to push him too hard, but for the sake of enjoyment. A science club or a reading club? It can be hard if you read a lot of have lots of thoughts but others around you don't. Have you spoken to the teachers about the gifted and talented track?
There is an interesting argument in Oliver James book 'how not to fuck them up' - where it says that if a child is insecure, you need to go back to the beginning and really baby them - let them act like a little child, give them all your attention, cuddle them a lot - make them feel safe again.
anyway, you aren't a bad mum - he is a kid with particular needs and you will learn how to work best with them - at least you are trying ! thats what matter.