I want to know the shopping trip story too, OP
. I reckon its because you were posted on a US military base in the UK which is considered US "soil". But you decided to leave it to go shopping, had a dramatic waters-breaking-in-Marks-and-Spencer moment, and we're taken to a UK hospital before 1983(?). So your DS was automatically a UK citizen as he was born in the UK...
I know nothing about the tax implications, though, sorry. But as pp have said, I understood UK tax was based only on residence (with various exclusions for double taxation, non domiciled people and so on) which is different to the US. But your tax advisor would be able to help.
I'm dual UK/US too, but from the other direction. My dad is a US citizen, but I was born in the UK and have never lived in the US. I've never earned enough to be liable for tax in the US and have no intention to live there. DS can't claim US citizenship through me because I have never lived there, nor through his grandad because he's now lived in the UK for too long! I have no right to vote in US elections as I have no legitimate ties to a state. I looked into renouncing US citizenship (around the Time of Trump, as it happens) and as heron said, I had to prove I had all my US tax returns in order 
and pay some obscene amount of money, which I decided was more than my principles were worth. (Dad also looked into becoming a UK citizen at the same time, as he has lived and worked here for 45 years...but couldn't face the test!). But also, citizenship is a bit weird, and I feel "tied" to the American part of my family, and dual citizenship is part of my "heritage", so I would feel a bit sad to give it up, even though it provides me with no tangible benefit and a bit of financial stress. I appreciate you're in a different situation though, and your DS may not feel that link to the UK as part of his heritage at the moment.