Can a uni reliably say someone has used ai to create their work?

[Unissss]

i personally don’t see how tbh

It's worth pointing out 'running it through an AI checker' is almost meaningless. The only decent one is Pangram. The others are useless. It's partly because the tech is moving so fast and the hype is so massive people are doing things like asking AI to verify a video (it can't, but it will confidently guess without telling you it's just a guess) or an image (the same) without realising the limitations of the tech.

AI writing is full of em-dashes (still!), "it's not x, it's y" contrastive phrasing (see also 'Not X. Not Y. Just Z'), lists of three, short sentences, mini section breaks (here's the kicker/and honestly?) and other quirks that are really, really annoying once you've seen them the first million times.

But we're already at the point where most of the tells can be removed if you just ask Gemini/Claude/ChatGPT to refine the style in a particular way.

We used to try and do “AI breach-proof” assignment formats, but over the last year or so Ai has improved so much it’s pretty much impossible - it can write your computer code, interpret graphs and solve maths-type problems. It’s academic referencing has also improved.

I’m a university lecturer of 20 years, so lots of experience before and after the introduction of ai. I would say that yes, there are clear signs of ai use that we can identify - but it is mostly not possible to 100% ‘prove it’. The only way I have been able to do so is where I have identified hallucinated sources in the bibliography that could only have come from ai.

My experience so far is that there are different levels of use - mostly where a student is using ai to entirely generate an essay, it doesn’t get them a significantly better mark than their own work would do, and often when they lean too heavily into ai they do far worse than they would have done on their own. Some use it to smooth grammar and expression but that often creates a flat and lifeless piece of writing, or gives a mediocre essay the polish of an excellent one, which is annoying because it takes much longer to mark and unpack the reasons why this is 55, when there is a gloss of 75 to the presentation. There are also a lot of students for whom English is not a first language using it to translate work done in their native tongue into English, which has its own set of issues. My advice to university students would be that you need to write your own essays and learn that craft to a reasonable standard before ai is going to be consistently useful in enhancing your work beyond what you are capable of on your own

I recently did a course on coding in python and if they suspected you used AI you had to go in and explain your code like how it works. For coding it’s kind of obvious like if you’re using packages we didn’t get taught or doing things a different way than we were shown.

It depends on the LLM used and how it's been prompted. If you submit a crude first response output from a free LLM and don't fine tune it at all, then it will be fairly obviously AI to most people.

However, if you know what you're doing, you absolutely can replicate your own style of writing and it will be imperceptible.

We were moving towards phasing out exams at my university due to the unsustainable amount of resources needed to implement exam support adjustments for students with academic support plans.

Now exams will be the only way we can truly assess a student due to AI and the increasing use of essay farms.

Why not just use Word's normal spelling and grammar checker?

Do you say 'please' to an AI? I definitely don't. Maybe that's rude?

My university has done this from this semester. Online exams are all now in-person.
From now on.

not sure on software but when reading them, there is a certain style or way of words so to speak that ai uses at least when ive used chatgpt, grok seems different word style etc personally id use combination of the two if needed a certain part of info, but then using a dictonairy to reword most or all the words to be my own words. etc

For proof reading, why not use MS Word? running the odd sentence through AI is probably okay, but I wouldn't be running my whole dissertation (or indeed a paper prior to submitting for publishing) through AI.

It's also worth noting that PhDs are slightly different as you have a viva at the end of it where your knowledge will be verified.

Some of the things people have light lighted though are not AI, just stylistic choices. For example, capitalising words in titles is a normal thing isn't it? Some people do use Oxford commas (although infrequently in the UK).

Obviously an em dash is a massive tell but kids aren't stupid and they would surely remove these

It depends on the extent to which it's used. If a student asks AI to write an essay I am guaranteed to spot it and they will have an assessment offense on record. If they've used AI a little bit for inspiration and cleaning up their text and adding some extra citations, I can usually spot it but won't act on it beyond informally telling them to stop it. Have faith in your own writing and stop using AI tools, they just make your work sound generic.

I would agree that I'd think you wouldn't want to be putting big chunks of your unpublished PhD thesis into AI, because it will harvest the content. You wouldn't want your data or analysis getting out there, pre-publication, for example; or ending up (mis) quoted in UG work....

Agree. I am astonished what people are prepared to feed into AI

Generative AI doesn't work like that. Plus there are settings so that you can tell it not to learn from your work.

Agree. But it's a quick fix.

It's improved a little bit with this. The style 2 years ago was horrific. Flowery and over wordy. Using 10 words when 1 would suffice. It has improved a little since then. I assume that eventually it'll be sophisticated enough to be a lot less detectable to the untrained individual/AI checker.

Oral exams are way too labour-intensive.

Yes, I can tell when my students use AI. I have marked essays for years and years without AI existing and I can tell the difference now it has been introduced. As I read hundreds of essays, I can tell all the signs instantly; the phrasing, the way they reference, the same theories and ideas being repeated consistently in a specific order. And yes, I know ways to catch the student out. AI checkers, however, are useless - I have written my own work and it claims it is AI.

🤣 DD has just written an essay (without using AI), on whether AI can be considered to have moral patiency!

Back to grades derived from: exams and essays written on a random day in-class and verbal presentations in seminars, and one or two longer pieces of written work. And serious questions if the quality of the work in the long essay or dissertation is different from that in the other three.

Back to the 1970s. Back to handwriting.

It's obvious when it's been used, especially ChatGPT! You can often 'tell' just from reading an essay. (Or, a product description in an online store, a social media post etc)

There are other obvious tells, too, like citations for library books that are not in the university library. Or the wrong referencing style was used. Or sometimes the book/journal doesn't even exist.

If the university is suspicious, they'll ask the student to come in for a plagiarism meeting and ask questions about the work, explain their choices, where they sourced the books, etc.

Many/Most lecturers will have years of experience of non-AI assisted writing and the probable development of students based on those who research as directed. The research recommendations come from a field that has covered ground over the lecturer's career, and most probably, the lecturer will also have sat in a number of conferences and interview waiting rooms, going for the jobs within that field, with a number of the same faces and their specialisms as they have worked their way up. It's a combination of current, active community plus the history of the research you could see in various 'slices' of that community at different times. AI's version of putting that into written form will present differently, as well as its linguistic tells, which people have described.

I have had one experience (pre-AI) where a lecturer who had been 18 years within a department had correctly identified plagiarism in a dissertation that originated from another student's dissertation, with a few years passing between the 2. The first in that history had been obscurely stored in hard copy in the university library. The tutor was able to check that it was there, and that material had been plagiarised. It is a very rare situation, granted, but an example of forensic depths in cumulative experience.

I can see the next step is questioning the student, so fair enough. But I think you should be careful of the difference between "knowing" and "suspecting"

My tutor clearly thought she could "tell" my work was AI. But it absolutely wasn't. I just have quite a formal /crisp writing style after years as a lawyer

And as for books - in my original undergrad degree my parents sourced piles of books for me from elsewhere (sometimes at quite a cost) as my (then undiagnosed) physical disability was meaning I struggled to get to the library. And I needed to work in my room where it was warm so never felt able to work in the library. I don't think I actually ever used a book from the library. Again, it doesn't mean you can "tell" it only means you can "suspect".