It certainly sounds as though it's a sensory thing.
www.falkirk.gov.uk/services/social-care/disabilities/docs/young-people/Making%20Sense%20of%20Sensory%20Behaviour.pdf?
Some kids with sensory processing difficulties can perceive touch as severe pain, therefore if you're trying to be very gentle when you touch him, which is a natural, kind, nurturing thing to do, he perceives that as pain and screams.
Watch and observe HOW your DH handles him, I'm guessing it's with a much firmer grip and your DS perceives that as deep pressure, which gives him reassurance.
You've said your DS screams when he has physical contact with you and he's done it from birth, so it's obviously not an at least initially visually led response.
If your DS' sensory perceptions are different, then it's possible his sense of smell as well as touch is very heightened and he just reacts negatively to how you smell. You've already said you've not used anything scented.
I know this sounds crackers, but as a parent of a now adult with sensory issues, you soon learn that crackers is sometimes your ordinary. Could you try wearing your DH's clothes before getting close to your son, also rub your DH's clothes over any of your exposed skin like face hands and arms, so all of you smells only of DH before going into your son's room?
Get close enough to him without touching him, but close enough for him to realise you smell of DH and see how he reacts and if it makes any difference for the better.
Video his reactions if you can, as HandsFaceSpace wisely said upthread, Remember: when you say it, it's anecdote. When they see it, it's "evidence