My husband and I have separate hand towels. Different colours. Mine remain the colour that they started out after being used for a couple of days. His do not. His end up with dirty patches on them because he's an idle fucker who can't be arsed to wash, rinse nor dry them properly. His hands are covered in flaky dead skin and, sometimes, bloody splits because he can't stop picking at them.
Not all hand washing is equal in cleanliness after the fact.
We have a third hand towel out for use only before food handling with the understanding of extra care with hand washing because my immune system is compromised due to serious health conditions.
I'd dry my hands on my clothes before I'd touch his regular hand towel.
I've got my weird notions and he has his. We don't bash each other over our idiosyncrasies.
EVERYONE is weird about something, usually many somethings.
Sometimes we can trace their origin, such as childhood or trauma, sometimes not. It is normal to be weird in your own way.
Don't let internet arseholes call you names. You do not owe them any apologies nor should you accept their despicable shaming of a stranger. Calling someone rank, disgusting, mental and making extremely personal comments to a person who has said they have mental health issues, is beyond giving an opinion that has been sought.
If you don't like him wiping his arms on the hand towel, you don’t like it. Just that. Talk to him. Tell him you know it's irrational but ask him for his understanding. Get him his own colour towels and mind your own business over how he uses them.