I can sort of understand the change from H to A. Especially with an accent and the person doing the writing thinking that James was maybe "dropping the H" in his speech.
I could well imagine that with a name that may be pronounced something like "Urnderson" the person who wrote that down may have presumed that he wasn't speaking properly and so inserted the letter "H" himself.
On Ancestry there are more than a dozen James Anderson the son of James Anderson who were born around the time of his stated birth in the censuses (as I'm sure you're very aware).
Likely the only way you are going to resolve this question is with DNA. Have you done a DNA test with Ancestry or some other website? If not, then I would encourage you to do so.
Even more, I would strongly urge you to ask your parents (if they are still alive) to do DNA tests. The reason I say this is that they are a generation older than you and so have more DNA from older ancestors than you do and so you can link back further.
From there, you can trace these other people's family trees and see where you are related. This will very likely give some clear indications as to which particular James Anderson was your gggrandfather.
.
My apologies if you already have all this information and I'm just repeating what you already know.
"...Family tradition has always been that he was a “Scottish sea captain”.
This might well be true. There was a James Anderson born in 1821 in Leith (I believe that is the year that your James Anderson/Henderson was born?) who was a ship's mate (sort of second in command) for very many years.
There are a couple of records of him in the National Archives. You may need to sign up for a free account to view these records.
If you have a look at page 109 of this document (third entry from the top):
https://discovery.nationalarchives.gov.uk/details/r/C408335
It shows that James Anderson, born in Leith in 1821 was certified as a Master's Mate on 18 Feb 1856 at Sunderland and he then sailed on the voyages mentioned between the years 1857 and 1866. (I don't know what all the abbreviations mean but it does seem to give the ships' names and dates that they sailed).
Then, on page 124 of this document, it records the same information for the years 1877-1886.
https://discovery.nationalarchives.gov.uk/details/r/C408334
I can find no record of him at sea between 1867 and 1876 but he does appear on the 1871 census living with Elizabeth and is described as a Mariner.
Both of the above records state that his Mate's Certificate dates from 1856. But when you go and look at that certificate, on the front it says "Renewed". And on the back, written in James' own hand writing (or, at least, he did sign it himself) is the following:
"I James Anderson of the Township of Bishopswearmouth in the said Borough Seaman
do solemnly and sincerely declare that on the Twentieth day of December one thousand eight hundred and fifty four in travelling from Sunderland to Liverpool I was robbed of my certificate as mate along with other articles"
So, going back from there, it appears that he first got his Mate's Certificate on 17 March 1851 with 13 years of experience of being at sea.
Now that we can attach images again. Here is his claim for (and the actual certificate) his Mate's Certificate attached below. Both the 1851 original and the 1856 replacement.
The original certificate gives his date of birth as 25 May 1821. It shows that he went to sea for the first time when he was 16 and a half. He worked his way up from Apprentice to Seaman to 2nd Mate and then to Mate.
It was interesting to see what different routes he had been on. He started off with "Coasting" (just sailing locally along the coast) and then sailed on various different ships to America and the Mediterranean.
What I found interesting is that he also sailed to Archangel, Russia.
However, my big problem with this is that this James Anderson was sailing with ships out of Sunderland between 1837 and 1847 and then North Shields from 1847 to 1849.
But he was Mate of a ship sailing out of Shoreham, on the south coast, for six months in 1849 before returning to ships sailing from Sunderland.
I really don't know how to reconcile this with him meeting Elizabeth. Where was she when they met? I would guess that they went back to her home town to marry, but just how did they meet in the first place?
If it is even this James Anderson.
Anyway, just something to add to things.
.
Although there was also a different James Anderson born in 1820 in Fife who was a ship's master (ie captain) and another born in Banff in 1819 who was also a ship's master, and also a James Anderson born in 1823 in Montrose, Forfar who was a ship's mate. There was also another James Anderson born in 1821 in Whitby who was a ship's mate.
There was even another James Anderson born in 1824 also in Leith who also become a ship's mate.
But, if you are absolutely sure of his date and place of birth then the others are not the correct James' (this just highlights how easy it is to get the wrong person).