Anyone have any recommended tools to use to detect AI generated student essays

[gallopingissuchfun]

Sick of reading assignments that are clearly styled by AI and at odds with students’ previous critical abilities. University’s TurnitIn software doesn’t detect AI text. What tools are out there to prove it was AI generated?

Someone said to me it’s like 20/30 years ago if we told people to stop using Google/ask Jeeves because that might be “cheating”. I’m not 100% sure about that comparison but will we look back in 20 years time and think such a comparison was correct? That the thought of having to think up your own argument or critical analysis was insane!

Exactly. Instead of fighting AI, maybe we need to rethink how things are done. I don't work in academia but the logic applies across fields. If AI can do X, Y and Z, then what exactly is the part that humans are valuable for?

Take social media as an example. Before AI, you'd spend a good 15 min, if not more, to write a decent caption and a visual that's worth looking at. Now you don't have to. So what do you do with that time? You can, for example, do research, The point of social media is to identify trends before they blow up, or even better, start your own. AI certainly can't do that because that takes critical thinking and AI has none (which is the whole argument here right? being able to form judgement).

The old way of getting students to think critically was to get them to write essays (and I've handwritten my fair share!). AI can now do that better than us, so maybe it's time we find a different approach, with or without AI.

The point of university is not to train workers for future dystopian roles of "ai prompter", its hopefully to teach critical thinking.
Writing is a good way to gather your thoughts together and develop critical thinking.
Using AI risks outsourcing thinking, in the same way that using Sat nav reduces your ability to read maps and navigate.
Many of my students have started handing in AI prompted garbage, the issue is they don't know its garbage, they haven't developed the critical thinking skills to know the difference between good and bad writing because they aren't practicing it enough.

I've just finished reading some essays that are complete word salad, even my weakest students 10 years ago wouldn't have submitted such garbage.

I'm watching the quality of my students writing and critical thinking decline in real time.

This isn't about being an AI luddite, I understand why workplaces may use them. But a university is a place where people work out how to think, AI can upset that process and learning how to use AI at Uni won't make them better workers in the future. Becoming good critical thinkers with the ability to form their own thoughts and arguments will help them to navigate a challenging workplace and if they do end up working with AI - they have the ability to apply some quality control.

Your perspective SophInLondon perhaps comes from someone that was educated without the temptation of AI. If you went to Uni 5 or more years ago, you gained critical thinking skills that you can now used in the workplace and use AI successfully. But the issue with the current generation of students is they have fully checked out and lean far to hard on AI, in some cases they scrape passes (aI can be tricky to detect and prove) and they will come out into the world with degrees.. but limited abilites.

Personally I think Unis need to have a robust response to how AI is used and keep it on the fringes for as long as possible.

DS told me about a boy stcschool aged 15 who fed all his essays into AI and his handwriting too so it woukd write new assignments in his style and his handwriting. Enterprising but worrying

Sorry about the typos - fat fingers

I read that Sweden is banning all technology from infant/junior school. Textbooks, paper and pencil. Maybe they’ll have the last laugh when their students actually can pay attention, have knowledge and can think critically.

Problem with just saying we need to not be luddites is that students can now upload the assignment task, upload the module guide, upload the lecture slides and recordings. Upload 40 related journal article (from the module info as we are told they can't possibly find their own readings) and press a button to write a 2000 critical essay on x in the style of an academic who works at x department.

So we get students who haven't attended, haven't watched, haven't listened, haven't written but crucially, haven't thought!

Someone is trying to recruit me to teach.
I may not be able to change the assessment in next 12m, but after that I'm thinking hard about how I'd like to change assessment.

the point of the course is that they learn something (not just LLM loading).
How can we best test what is in their heads that is not how to prompt an LLM.

It does amaze me how many kids must think they will get a higher paid job despite this, moreover that a 50k debt for life is worth it when they are just not engaging at all. They clearly have no concept that one day if they manage to pass all of the interview vetting etc, they might actually be expected to know things and solve new problems. Or that having a bit of paper with a degree on it is just a debt receipt if you can't pass an interview because you decided not to know core concepts.

Many of my students have started handing in AI prompted garbage, the issue is they don't know its garbage the , they haven't developed the critical thinking skills to know the difference between good and bad writing because they aren't practicing it enough.

@Spronkles42 this describes the papers I’ve just graded. Worst results in years. The adults enthusiastically promoting it can use it as a tool. For those who already possess domain knowledge, it’s a fantastic tool. This, however, does not apply to our (my) students who have succumbed to cognitive surrender. Dunning Kruger in action.

Graduate employers beginning to query why graduates with an amazing CV and excellent grades are failing to perform in interviews. It’s going to get embarrassing. In person exams solve some of the issues but writing an essay meant students had to maintain an extended argument whilst maintaining logical coherence. That just doesn’t hold anymore.