Anyone have any recommended tools to use to detect AI generated student essays

[gallopingissuchfun]

Sick of reading assignments that are clearly styled by AI and at odds with students’ previous critical abilities. University’s TurnitIn software doesn’t detect AI text. What tools are out there to prove it was AI generated?

Well I for one have learned that exams at universities are no longer a thing. Who knew?

I’ve just finished a round of marking essays. I’m guessing about 80+% GenAI. No checking by students to verify the information, which was incorrect, no challenging ai content because they don’t know enough how to, all of the writing in an infuriating social media style of exaggerations, sweeping hyperbole, wild and wide eyed rhetoric. Lowest average marks in years.

Def cognitive surrender. We’re doomed.

Why can't they handwrite their final essays in an exam setting?
Genuine question.

Many of our assessments are in person exams with more to come.

Shame in that pedagogically they’re not the best format, but it’s the quickest way to address cheating. Though students now cheating with smart watches lol. A colleague allows students to use ai through the year, but final exam is in person. All students did poorly because of not learning the skills themselves.

Plus, whilst exams test knowledge, they're rarely effective at demonstrating analysis and critical evaluation, skills needed now.

I did a history degree with some elements of English Literature, and our final exams tested analysis and critical evaluation very rigorously. Surely it just depends on the type of questions set?

I write essays every day in my job. My job isn't in academia.

Unfortunately we can only detect about 67% of AI generated coursework so you will be open to litigation if you claim that work has been done with AI. The only thing to do is to create assessments that make it very difficult for the student to use AI or assessments that do use AI but require students to show independent critical thinking skills

Having pretty much eradicated exams, we've now gone right back the other way and are replacing essay- and project-based assessment with exams.

@WhatHappenedToYourFurnitureCuz thats interesting. What is your job and who reads the essays?

The clients who request them.

@WhatHappenedToYourFurnitureCuz so are you are writing uni essays on behalf of students so they can cheat on assessments because thats what is implied by your lack of info? i genuinely cant think of any other use for an essay with the exception of essays which are published in a book as a means to explore a subject. E. G. Essays on the slave trade etc. But those are pretty unusual.

Yes you can get it to write in a way that will fool this type of software.

No, nothing like that. It's strange you can't imagine any use for essay writing beyond university!

Also no, that wasn't at all implied by my post or my not wanting to share my job title on a forum.

That's a good idea.

Think tanks, journalists, web content providers, researchers who need to translate medical, social, ecological, scientific, economic, political data into easy-to-follow material for their clients, co-workers, stakeholders etc all write essays or long form that is similar in structure to an essay.

this is genius - critical thinking is rapidly becoming a rare commodity

But the problem is you can get ai to do the second step as well. Pit copilot against ChatGPT 🤷🏻‍♀️

the critical thinking bit is knowing why this might be a bad idea 😁

Well quite. Sadly I don’t think that will make any difference though 🫣🤣

On my course - 35% of the students would have an anxiety attack and burst into tears... some would refuse outright.

Many would submit unredable garbage - out pass rate would tank.

Then many of the students would put 1000% more effort into fighting the existence of exams then they ever would in terms of work.

Student satisfaction surveys would tank, students made to do hard stuff will complain. Student satisfaction surveys are an important marketing tool. Management only care about recruiting max numbers... if you question pedagogy they pretend to care but they don't. It matter's not how much AI makes course work essays impossible to mark. At a bottom feeder uni (most of them at this point), students don't like exams, so they won't be asked to do them.

To be honest the AI coursework is the tip of the iceberg.... the marketising of HE has destroyed it. Anyone that went to university in the 80s or 90s won't recognise it now

@Tonissister thanks for that. I guess in my head those mostly arent actual essays though? Technical documents or explanatory documents but not actual essays on a topic. Same skill set though. I wasnt really saying that essay writing as a skill isnt important, just that actual essay writting is often not the actual skill that the students need and that there could be other ways to assess critical thinking, comprehension of a subject and communication skills.

Unfortunately, gen AI can be used to do this too. It is a good skill to learn though

I’m in the middle of marking season and am more surprised to read a submission that wasn’t written with the use of AI

I was at a meeting recently where another academic excitedly told us how he was experimenting with using AI to mark coursework assignments/essays. Uploading the marking rubrics, guidance, learning outcomes apparently worked very well to provide detailed feedback but the actual grading was off and needed manual adjustment

Im an old dinosaur with one foot out of door but the prospect of a system where AI marked AI was deeply depressing

One of our SLT has had the idea of AI proofing an assignment by getting the students to build a chat bot and then write a report on the chat bot. But they’ll just get AI to do it all for them. I build chat bots for work and I tell AI what I want and it builds it for me.

I'm getting sick of trying to explain to SLT that AI is a real problem in the classroom, and getting 'helpful' suggestions about how to overcome it by people who haven't taught in the classroom for years if not decades. My head of department is constantly saying "get them to critique what it produces". They will just ask it to critique itself or use Claude to critique Gemini. It's also an exercise that gets tired rather quickly. The only way I can stop it is to completely ban all devices from the classroom.