It wasn't just the royals that were fooled by him. Heaps of people were. I never saw anything other than what he wanted to see.
I know now on MN everyone says that "they always found him creepy", but look at some of the threads before he died. There was the odd one or two, but more often not.
And have you never been fooled by someone? There's someone who likes to push themselves into the public eye locally; people are marmite about him.
Some people say he's so kind, looks out for everyone, so sweet of him for doing these things. He's very good at controlling the narrative. There's a few people actively hero-worship him.
He's very good at making connections with the right people who will push him forward with him saying "oh really, little me, I'm only trying to help, I'm not sure I can... but as a massive favour because I can see there's no one else, then I will...".
And he won't let them see his real side. He assesses people into people that can push him up, then goes out of his way to woo them. People who can't help, or would stand up to him, he either ignores or discredits them.
From personal experience I can tell you that he's a practiced manipulator, who will lie to save himself, and throw others under the bus without pausing to think. He's an expert on coercive control. As far as I know there are no pervy allegations about him, but he's defended people who do (and in at least one situation been convicted by the courts).
People who see that side of him, if they stand up and say anything are shouted down by the other people.
I suspect when he is gone there will be plenty coming out about it, and half the hero-worshippers will be queueing up to say how they really suspected it. If they did, then they are guilty of enabling him - but I don't think most do. He hides it well to the people he thinks matters.