You poor thing. I really feel for you.
Right, just to dissect a bit. Firstly, I don't know the legalities of changing a name that's already registered where you are (you said you were in a different country).
Secondly, it seems to me that Martha is the 'special' name, the one that pulls at your heartstrings. This is what happened to me, I 'felt' something about name X, but it was one of three on a shortlist, and when it came to deciding my OH gave me a (not very good) reason for not choosing X, so I went with something else. He chose to tell me the reason a couple of days after giving birth, by the way, when I was traumatised and bonkers. The name we went with was a bit more 'out there' and I could never say it with confidence. I dreaded people asking me her name. I too just called her 'the baby'.
In the end DP just wanted to do what would make me less anxious, also there were no other children to consider.
I don't think the argument about it being unfair on your DDs washes, to be honest, the older one will get used to the change and DD2 doesn't know anything about herself yet! Presuming your DH actually likes the name Martha I'm sure his annoyance is due to being worried about what people will think. You just have to front it out if you change it, I'm afraid.
So, should you change it? FWIW, you seem to have an emotional attachment to Martha, and that's what you thought of when she arrived. Anna is a beautiful name, it was on our long list, and Anna Parsons is lovely. Martha Parsons to me is more memorable and more impactful, precisely because of the rhyming thing.
In the end the only way I could move on was to change it. I sometimes do think about the reason DP gave, but for me that's less important than the way the name makes me feel. Hope that helps.