It would be such a terrible thing to say that I did wonder if something was misheard or misinterpreted down the phone. I have to say, though, not only does Spencer seem very credible, Richard Kay says it chimes with his recollections at the time. It's also notable that although the Palace put out a statement, said statement did not deny that he said it.
If he did say it, then that is beyond appalling and he deserves to have it come out. It would be despicable enough to say it to a third-party, such as a courtier, say, or privately to his own family, but to say it to the deceased's BROTHER! 😱
If Charles did say it, I reckon Diana would be doing a happy dance wherever she is.