"he is a nice guy but has had previous alcoholism, and has mental health issues and tbh I don't know that I trust him to be with my son alone"
this segment of a sentence in your op is why I and I suspect others believed you were wary because of the mh/alcohol issues.
"the background story is that my cousin was sexually abused by our grandfather a long long time ago (he was 10 years older than the youngest in the family) and told everyone not to trust this man everyone thought he was BU but he wasn't and it turns out my aunt (cousins mum) was a victim of his sexual assault when she was a kid but didn't say anything so I'm always on high alert after this and so was my mum untill she met her bf" so this is the real reason?
Understandable to a degree but not all men/grandfathers are the same. And I'm speaking as a survivor! I would never leave my daughter alone with my father (whom I know to be abusive) but her other grandad is lovely, I don't get on particularly well with him (political views differ) but he adores my daughter and I'm as sure as anyone can ever be that he would never harm her. He was until a few years ago looking after her on occasion since she was 2 (dependant on his health) I actually trust him more than her dad! He's totally incompetent!
Your son is 5 as he gets older - And I'm not talking too far in the future - he's going to be wanting to do things that may involve men you don't know very very well. Play dates, sleepovers, clubs and outings. You can't limit his life, his development because of this fear.
"It's just at times like this I don't understand why she didn't tell her child or he nephews and nieces about his assault and continued to let us go there" because some victims experience genuine psychological denial.
Of course your decision and nobody can make you do something you're uncomfortable with but part of parenting is balancing risk vs reward. It's tempting to duct tape them in bubble wrap I know but it's not the answer. In fact kids that have been very protected are MORE vulnerable to abuse because they don't develop their own instincts as well.
Is your sons father on the scene? What does he think?
Whatever you decide you are basically going to be saying to your mum that you don't trust her to make the right decisions for her grandson. Not a reason not to talk to her but be aware that's how she's likely to feel when you do.
You're also making life very hard for yourself if there's only 3 people you trust to care for him.