MadamMinacious, I had a quick search and here is a gentle introduction into how sampling works: www.umsl.edu/~lindquists/sample.html
A little under half way down, it mentions non-probability sampling methods. This was the method used by Stonewall.
In statistics there are two main sources of error, which is called bias (yes, another instance where a common word has a precise definition inside a field). I did a search for the language that I was taught, which I can't find, but there are two types of bias:
- random bias
- non-random bias.
Random bias is due to the fact that we have used a sample, to estimate to the population. Because any bias in the sample is random, it is presumed to have little effect on the results. This is why a lot of good studies present confidence intervals around the estimates - this shows that we think that the real result lies inside that interval, and the report shows the single figure that people are interested in. For example we might find that 35% (33-38%) of our young girls had outpatient treatment for tonsillitis. The figure of 35% is the one that people will use, and the confidence interval in brackets shows that actually, the real percentage could be as low as 33% or as high as 38%. And because our study was so good, we like the fact that the confidence interval (95% in this case, the normal confidence interval used) is quite tight around our estimate.
Non-random bias is the problem. Non-random bias cannot be adjusted for in statistical analysis because it is introduced by poor sampling design. The population is ill-defined. A sample frame wasn't used. The actual people answering were just anyone who decided they were motivated to fill in the survey (and maybe fill it in multiple times, to make a political point). Who answered the survey? Well, who knows. Were the people completing it actually the ones that were wanted? Who knows. Were the people completing it actually representative of the population (e.g. transgender) with respect to sex, age, disability status, ethnicity, country of birth, country of residence, personal income, household income, employment status, household type, duration of transgender identity etc etc. Who knows.
So what does the "data" mean? Well, nobody knows.
Sadly this is the point at which people tend to ask statisticians to come in and fix the data.
In the immortal words of Ronald Fisher: "To consult the statistician after an experiment is finished is often merely to ask him to conduct a post mortem examination. He can perhaps say what the experiment died of."