Scammer/imposter research participants?

[kualitate]

Has anyone else noticed an increase in scammer/imposter participants for qualitative research projects?

I'm currently recruiting for a qualitative study involving interviews and offering a voucher incentive for taking part. Shortly after posting details of the project on social media, I was inundated with responses from scammers - I had about 40 people sign up within the space of 5 mins!! The email addresses were all a similar format and the language was also a bit over-familiar and they were very keen on scheduling in an interview asap.

I scheduled an online interview with one who seemed very convincing and eager, but it was obvious straight away that they were a scammer once the interview began.

I've looked it up and there seems to be a large number of papers being published by other qualitative researchers who have experienced the same thing and saying it's become a real problem since the pandemic and the shift to online interviewing. Nobody I've spoken to in real life has encountered this and I'm wondering how widespread it is and what can be done about it?

Recruitment has always my least favourite thing in general as it's so time-consuming and difficult to get people to sign up for interviews, but I've never had this before and it feels awful to be honest. I'm actually jaded about the future of research as I always thought qualitative data would be less vulnerable to this sort of thing than quantitative surveys...

Are you offering any kind of incentive to participants?

There’s been pieces in the media about this recently - last week I think. Some of it is bots.

I haven't had it, but some colleagues have. They think that respondents were putting the questions into GenAi and reading out the responses or dropping then into chat - some wanted cameras off. I don't know if there's an easy answer when doing online interviews, other than going via agencies that pre-vet candidates (I don't know if this is a thing, but a colleague suggested it was) and/or terminating interviews. But the participant info then needs to make it clear this could happen.

When you say scammers do you mean they lie in order to screen into the interview? Then give nonsense answers?! I'm trying to work out what you mean by scammers.

I used to do qual and never experienced anything like this but this was ten plus years ago.

yes, there is lots of this, and papers documenting and other papers giving advice on how to avoid the scammers etc.

we have also had scammers who claim that their Amazon voucher payment didn't work and they need another voucher. And repeatedly complain to the ethics committee until we give one to shut them up. We have no way to verify if they spend the voucher or not, so they end up with 3 vouchers. When we got 3 or 4 of these in a row, we stopped doing voucher payments.

we mostly recruit via Prolific which makes an effort to weed out the scammers, but there are still some.

I have a couple of masters students looking into recruiting online and this is very disheartening. If anyone happens to have a link handy to some research or similar that I could pass on please post. We also use prolific - sorry to hear it's got in there too. I agree, recruitment is challenging - always makes me anxious. I'm especially anxious about one of the proposed projects for this reason - but co-supervisor is quite keen.

Bots answering surveys and being part of focus groups, badly affected one of my projects about 3-4 yrs ago. About a year later I was chatting to Qual researchers who had obvious fake participants (people impersonating the target group). And were very annoyed about it.

You don't need to offer an incentive to get fakers. The fakers will chance any and every opportunity for a possible reward; if nothing else, the bots can be programmed and the fraudsters just plain learn, from what they encounter and how they might fail, to be better fakes next time.

LLMs are being programmed to write fake articles and submit them to journals. Editors say these fakes are obvious but... they won't be obvious forever. the LLMs are learning how to generate better and better fake articles. Of course we all have had the annoying and pointless experience that is receiving fake peer review.

But sure, go on, tell me why it's so marvelous that chatGPT is free and governments should pay huge sums to subsidise access to AI for the masses and put up with all the environmental damage AI requires. "Look at all the marvelous things AI enables."

Thank you. I'm glad I'm out of that industry as AI and bots wasn't a thing really when I was running qual projects. I left the sector in early 2018.

Yep, this is a real problem and always has been to some extent. Even if people aren't just pure scammers/bots, basing research on people motivated to take part by incentives is always going to skew things.

Here are some papers

https://www.tandfonline.com/doi/full/10.1080/13645579.2024.2410176

Ridge, D., Bullock, L., Causer, H., Fisher, T., Hider, S., Kingstone, T., ... & Southam, J. (2023). ‘Imposter participants’ in online qualitative research, a new and increasing threat to data integrity?. Health expectations: an international journal of public participation in health care and health policy, 26(3), 941.

Pellicano, E., Adams, D., Crane, L., Hollingue, C., Allen, C., Almendinger, K., ... & Wheeley, E. (2024). A possible threat to data integrity for online qualitative autism research. Autism, 28(3), 786-792.

Apparently the scamming practices have also infiltrated market research so it's not just academic research being targeted. I was really surprised it's happening so much in academic qualitative research - it's really picked up after the pandemic and the shift to online interviews becoming the norm. The big signs to look out for are people refusing their cameras as they will often be feeding your questions into AI and then read them out - genuinely makes me so frustrated!! It also raises all sorts of ethical problems like what if you're running a focus group on the experiences of parents of autistic children and a scammer gets in.

Scrapping incentives might be the only way forward but it feels crappy that genuine participants wouldn't be compensated/thanked for their time and efforts.

I'm just surprised there isn't a wider conversation about the impacts of all this on research and publishing as a whole as you say MedSchoolRat - it feels like the quality of research is being downgraded and seriously compromised yet no one is really taking it seriously?

I have to admit that the whole experience has really made me question and be a bit more sceptical of survey findings in general that I see reported on the news etc.

This is very true but I think we're in a whole real era of fakery right now sadly given the advances of tech/AI

A friend says apparently people have been selling/renting their Prolific accounts to scammers so sadly it can't be trusted to vet people in a way you'd hope :(

https://theconversation.com/imposter-participants-challenge-research-integrity-in-the-digital-age-246126

Market research was affected long before academia, actually. Market researchers are more savvy & have had more stringent checks for longer... it's academics who are newer to realising the faker problems.

Oh this is very disheartening. Thanks for the extra info especially @parietal and @kualitate for the links. Doing away with incentives is not really an option - recruitment is tough enough with them. Quite small incentives work well, but if you have 3 or 4 figures of pps - it really mounts up. For a smaller masters/3rd year project, even double figures can be outside the budget.