It's not about falling over themselves to justify it.
He started there when wages were far more generous. That's the main reason he is paid more.
This happens to both men and women. In a massive uk company I worked for, the men and women who had been there 20 plus years, didnt have to work bank holidays, were paid substantially more, even more than their managers if the managers had joined after them. Over the years the wages have been less generous, perks have been too. People who have been there 20 plus have the best one, 10 plus isnt as good but better than people who joined in the last year.
The company has no obligation to pay them less because the pp is his manager and a woman. When he reduced his duties, they didnt look at reducing his wages.
He and the OP are in different positions. Other peoples wages do not automatically mean yours should be a certain amount. Especially when the situations are so different.
The OP says herself that wages arent as generous as they were when he started.
If a man joined the company when the pp did and was in the exact same situation, I believe he would be still getting paid less than this man.
If OP wants a pay rise, she needs to concentrate on yourself, not saying 'well this man who worked here from the year dot gets more so I want more'. That's not how you negotiate a pay rise.