Can a uni reliably say someone has used ai to create their work?

[Unissss]

i personally don’t see how tbh

Some commercial products claim to be able to deliver an accurate probability rating and %, tested successfully against real life AI and human examples, but there is no way to guarantee accuracy AND it will vary a lot depending on the nature, length, and complexity of the passage. A one paragraph general communication may very well "read as AI" because the human who (independently) produced it has learned to write that way from reading that type of writing.

Alll AI content was originally created through the self-expression of one or more humans, and "nothing human is alien". As AI use becomes more widespread and normalised, cultural creep will result in more and more humans spontaneously writing "like an AI" because that is how they have formally learned to write and where their examples of good writing or "typical" or "appropriate" writing have come from.

My sister was applying for a job which said that AI could not be used for the application. I read her application thought it was a bit long winded and got ChatGPT to tighten it up. I then added a couple of bits and pieces. She then ran it through some AI detection software which correctly identified that AI had been used in part. This was idle curiosity on my part and she stuck with her version but I was interested to see if AI could be detected when it hadn’t created the work from scratch.

My DH works at cutting edge of AI. He told me it is almost impossible to prove. Unless you literally copy & paste an entire essay. If you put it in your own words they cant.

Even if they suspect, its just that, a suspicion. They would have to prove it and its very difficult to do.

I am not suggesting you use AI btw, regardless of getting caught or not. I am just saying what he told me abd he works in a pretty niche area, he definitely knows a huge amount about AI.

No. Not for essays. I am a teacher and we check work using the same AI detection programme that universities use. It isn't accurate - it just picks up certain things that AI does often - and actually detects very well written and gramatically correct papers as AI!

I wrote a paper recently as I am doing my masters degree alongside working, and had to 'dumb it down' to get it through the AI checker!

This OP wouldn't have put it in her own words though, if, as she claims, it was her spelling and grammar she was wanting to check. She wouldn't have then messed with it.

Yes, she might find herself in difficulty...

People talk about AI as if it’s one single app. The different models produce different types of output and some are remarkably good.

The short answer is no, someone can’t tell whether a piece of work is AI simply by looking at it or running through an automated checker. Those things are shockingly bad at telling between AI and human-generated work.

AI has evolved and while it still often produces waffly, shitty content, with the right prompts and some light editing, it can be very good. Certainly good enough.

The difficulty is that the tells of AI often crop up in human writing naturally - because AI is based on human writers and continues to be trained by human writers. Human writers like me; after 16 years as a copywriter, my job is now making AI better.

But a diligent uni will carry out other checks - is the work different to your usual style? Is it referencing non-UK or non-course texts? Are there any copy/paste markers in the text? And of course, can you talk knowledgeably about what you’ve written?

You say you used AI for spelling and grammar - did you also allow it to change the wording? AI often REALLY likes changing the wording rather than just carrying out a simple SPAG check, and that might have been your undoing.

In my institution, you’d go before an academic panel. They’d decide if it was poor academic practice or deliberate intention to deceive. If found intentional, you’d receive a mark of zeto, be able to resit and have a mark capped at 40. But if they didn’t think it was deliberate, you’d ‘just’ lose marks.

Turnitin claims to have 98/99% accuracy now.

Providing them with the unchecked version, as a pp said, is the best way. If there is such a version....

As it will show the date as being pre submission.

They would have to prove it and its very difficult to do.

We have a viva. If the student can reasonably explain their work, they’d likely accept poor practice, but I cannot tell you just how many students are unable to talk meaningfully about an assignment generated by AI. So, yes, they can tell. It’s more about knowledge and understanding than specific AI use per se.

edited for typo

A uni prof I know says 'Yes we can'.

When I did my degree at a Swedish university 20 years ago every single essay and research paper was orally examined at a viva seminar with an opponent (another student) appointed to lead the questioning

year 1 and 2 short essays (1,500-2,500 word) got 10-15 minutes
year 2 stage 1 dissertation - 6,000-8,000 words) approx 30 minutes
year 3 - bachelor dissertation - 10,000-15,000 words - 60-90 minutes

It was pretty easy to find out who had not written their own work.

Although there was this guy who plagiarised his dissertation from a small, niche academic journal ... his bad luck that the original author was the examiner for his dissertation having recently arrived back in the department after a year abroad,

the detection is far from perfect. Anecdotally, I’ve noticed a pattern of people who have autism being accused of using AI. We tend to use formal speech patterns.

Lecturers usually have a very good idea that a student has used AI, but it's much harder to prove it.

Even if they suspect, its just that, a suspicion. They would have to prove it and its very difficult to do.

That's irrelevant. It doesn't really matter whether it's easy to spot or not. It doesn't work like a criminal prosecution. The onus is always on the student to provide proof that they didn't use it, not on the uni/lecturer to prove they did. It's pretty simple if you genuinely didn't use it - you can show your essay plans, your drafts and edits, or demonstrate your understanding in a chat with your lecturer.

This is what I have done. All humanities departments are dealing with this. If students genuinely think we can’t tell when they’ve used AI they are taking a huge risk. Some colleagues are less interested in pursuing cases; many of us, however, are very focused on it. My faculty has already required students to attend a viva and defend their work when early investigations suggest they have not written it. Academic misconduct cases are judged on the balance of probabilities, not beyond reasonable doubt.

No, not reliably. Detection tools look for patterns, but those patterns are not unique or consistent enough to prove anything with certainty.

In future, however, be careful not to ask AI to "tidy up" anything, and just use a spellchecker.

Probably.

And why would they? ... use AI? It's crap. and just 65% accurate.

This was already a problem before AI, albeit lesser. Essay writing 'services' cost, but some students preferred that to making an effort.

My technique is not to ask them to explain the essay topic but to take one of the more convoluted, word salady or nonsensical sentences and ask the student (always asked in a butter-wouldn't-melt voice, no accusation to be heard!) to explain what they meant by that. The impression they gave of a fish out of water was a good giveaway.

No. I have a friend who is brilliant, who has been failed twice because the checker said her work was AI. It is not.

ETA: not at university. Distance learning with no lecturers who know her personally. No viva offered.

University lecturer here...

No, not 100%.

Yes 99%

You can't "prove it" no matter what any university claims... but you can get to the point where you are beyond reasonable doubt.

AI checkers are unreliable but give a good starting point.
Comparisons with previous work
Oral defenses/questioning (getting students to show they understand what they submitted)
Recognition of common AI writing styles/patterns
etc.

In my experience, most of the time, the actual proof is a student's confession. The professor goes with his "evidence" and then the student admits what they have done, and that acts as the proof.

Yes, that can absolutely cause it.

If you were my student, I would ask you for your original draft. You should have the draft that you gave to AI. If you are telling the truth your first draft should be 100% human written and clearly so.

That can prove you just used it for spelling/grammar checks, and satisfy your lecturer.

If you don't have a human written first draft? I'd presume you are lying.

In the future -

1, Save your drafts if you are going to use AI for checking,
2, Check your class/university policy very clearly for permitted uses. If they don't have a policy, ask your professor/lecturer
3, When you give AI your writing, be very very clear say something like this:

"Please fix the spelling, grammar, and punctuation of this piece of writing. Do not alter the word choice, style or structure"

That usually results in it just making basic fixes.

If you don't tell it not to, it will often make bigger changes to your writing and style, which can then be viewed as AI-written.

I think more assessments will become exams or interviews rather than written essays.

I do face-to-face university interviews with students who are trying to pass a professional qualification. There’s nowhere to hide, AI’s of no use to them and they have to know their stuff.

AIs are useful for producing a basic plan or draft with ideas for avenues of exploration. An intelligent student will use this as a suggestion for forther reading and thinking. They may make use of the basic plan, but the finished document is a completely fresh one, expressed in their own language and tone. It will be written from scratch and not copied and pasted. They will understand their subject through engagement with the literature and ideas. This is not cheating.

She should not accept this. Universities actually have a really difficult time in failing students for AI use, because proving it is so hard.

There is a trend in universities to now prohibit their professors from using AI checking software or failing without "proof" because of how many legal issues it has been causing, with students in some countries suing their university because of it.

Distance learning is irrelevant, if she is actually innocent, tell her to write up her case including any and all of her drafts/plans/work in progress etc. send it to the head/director/dean of the department/college and I wouldn't be surprised if she saw a positive outcome.

That being said, there is always the chance that she did use it in some capacity, and isn't completely honest.