Our family supposedly has a bit of a scandal in our past. Looks-wise there is a resemblance. One elderly, now deceased relative told me very factually about her past and also about the rumour - taking it as absolute truth. Many other relatives very vehemently denied it.
Two of our relatives did DNA testing and there is a distant cousins link to a descendent of the disputed ancestor. I believe it shows a genetic link of some kind and that there is some basis for the rumour.
The different names on the birth certificate and marriage certificate, movement to a different region and sudden, unexplained wealth do suggest the rumour to be true. Subsequent children were given more typical names. In photos they don't look like full siblings.
Regardless of any proof, my elderly relative told me and I believe her. I always have. I wouldn't dismiss Prince Simon's claims entirely.