Theres 3 situations.
Driver paying partial attention. Either due to another hazard or just not paying full attention. Someone in hi-vis is more likey to get the attention of this driver, than someone dressed in black. So Hi-Vis is worth wearing.
Driver paying complete attention. Someone dressed in black still not be able to be seen, depending on conditions. Someone one hi-vis is far more likey to be seen. You cant avoid something if you can't see and have no clue its there. So worth wearing hi-vis.
Driver no paying attention due to being on their phone, maybe just a dick or whatever. Person dressed in black won't be seen. Person in hi-vis has a slightly higher chance of catching this persons attention and being avoided. So still worth wearing.
Anyone who doesn't see a horse in daylight and or moans that the rider didn't have hi-vis on is a complete idiot. They clearly are not capable of driving daily and so arrogant to think its someone else fault. However, we will live in a world where you need to be caught doing something ro recieve consequences for it. So unless the police catch them or they are caught on camera and its enough to follow up, nothing can be done.
Car drivers are not perfect. Hi-visit wear won't stop every accident. The onus is on car drivers to pay attention and follow the high way code. Hi-vis is part of being safer on the roads.
Its impossible to say how many accidents have not happened due to people wearing hi-vis, because you can't count something that has not happened. Even the suggestion of studying wether people give extra space to Hi vis wearers, wouldn't work. People would know their driving was being studied and so act more cautiously. Also you would need people willing to be hit, because they would need a group wearing clothes where the drivers can not see them, to compare.
I can't see how hi-vis does not help at all and I don't believe (apart from some dicks on facebook) that most people think hi-vis is the be all and end all and solves all the road problems.