*Again, it's you who's being sneery and othering to call it "very heavily accented English".
Everyone has an accent. Not just Glaswegians and Indians. It's not the accent that is problematic, it's your husband's inability to understand it.
I expect you'd say my lower middle-class Home Counties English accent is less heavily accented than a Mumbai English or Gorbals English accent, but it isn't. It's all a question of perspective. Your perspective is that your accent is "normal" and everyone else's deviates more or less from it. That's where the prejudice comes from*
I agree. My accent is RP with a slight note of Croydon.
I listened to a programme on R4 about accents some years ago. The accent expert explained that people from Croydon have such an absence of local pronounciations that it is very distinctive, because it's almost a non-accent.
I love the variety and diversity of local regional accents. I'm fascinated by the fact that 2 Irish friends have markedly different accents, even though they grew up barely 20 miles from one another, and how my friend from the Wirral has an accent so different from that of my Liverpool-raised colleague. (I had 2 Liverpool colleagues at one time, and there was considerable variety between their accents. They attributed this to the fact that one supported Liverpool and the other, Everton. 
I wondered if this was shorthand for differences in social class.)
I find it sad that SE accents, especially the Sussex and Hampshire ones, have all but died out (I know one person with a Sussex accent, and he's in his early 80s) and that many dialect words are falling into disuse. Soon, everyone east of the M3 will be speaking RP or "estuary English".
I think we should celebrate the richness of accent and dialect. And if people struggle to understand some of them, they probably need to listen better.