"If statistically significant is the meaning of significant, is there an actual percentage? And who decides that?"
Just to add, this isn't quite the issue here, since the case is taking the actual results (1 out of 9 off ventilator, additional 4 out of 9 reduced time -- plus 4 out of 9 no change I assume), and from that trying to work out some reasonable estimate of the chance of a good outcome for Charlie.
Lots of unknowns or only vaguely known in there.
First error, I think, is forgetting about the small size of the sample. Nine patients -- and were they carefully chosen (i.e. he only treated the ones who had the best chance of a good outcome) or not? Saying 10% rather than 1 in 9, or 56% instead of 5 out of 9 makes it seem like the 56% chance of some effect is a really reliable, accurate figure. Whereas it is just the most likely value for the chance of some improvement as regards ventilation. But actual value could vary quite a bit. For example, the 10% (which was actually 11.1% to start with), could, if you look at the graphs, actually be somewhere in between 5% to 20%.
If you look at it as saying -- 50-50 chance of no improvement to lung function at all, then even if that is good, you have to factor into that:
- Passing blood-brain barrier
- Improving brain cell function
- GOSH being wrong previously as regards catastrophic irreversible brain damage having occurred.
I think many people have just got the 10% chance of a 'cure', and well, some figures of over 50% thrown about, sounds pretty good, yes? Because most of us are just lousy at accurately assessing probabilities. Which is why I stick to the 1 in 9, or 5 in 9, people do have an intuitive grasp of what that means, you can visualise that.
I keep thinking of the possibility that you could have improved lung function, but no improvement as regards Charlie ability to experience anything. Or you could have some improvement of lung function, plus he could experience pain and react to that pain and the seizures he might be having. Which would look like a worse situation that his current one frankly. The chance of an improvement as regards both lung function and his experience of the world seems to just get smaller and smaller................