I doubt very much it's a medical issue. It would happen at work if it was that.
You need him to explain what he means by 'You don't listen to me'.
When people say that they can mean one of two things:
1 The actual words they say to describe something
2 The words they don't say (because they can't put their emotions into words) BUT hope the other person will read between the lines.
A really common example (often with men) is that when women say how they are feeling, a man will offer a practical solution. Often, that's not what's needed. The person who is upset/angry/ frustrated knows the practical or logical answer. What they need is a hug, a shoulder to cry on, some empathy.
So when someone says you don't 'listen' they don't just mean you can't hear their words.
They often mean your response falls short of what they need. And that is usually empathy.
An example....your DH may moan about a colleague or work, but is afraid to say he wants to throw in his career and do something else that might cause financial insecurity.
That's just one small example.
What you need to do is start asking him how he feels. Not just what he says but what's going on inside his head.
He may be sick of you, He may be having an affair. Who knows. But until you sit down and talk properly you won't know.