How to know ChatGPT is full of shit.

[DiggingHoles]

Take a book of a shelf. A classic is best. Open up to the first page of a random chapter. Now ask ChatGPT to quote the first paragraph of that chapter.

Tip: Have some popcorn ready while you rephrase your request multiple times.

My son showed me something on TikTok a while ago; a woman claiming to be an ‘EQ trainer’ helping people with their emotional intelligence, had obviously used ChatGPT to generate content advising on how to politely exit a conversation ‘without being weird’.

The first suggestion was “I have reached my limit for social interaction and will be stepping away now. Goodbye”.

Another (for when somebody’s talking about a hobby or show you’re not into) was along the lines of: “I can see this matters to you, but unfortunately I am not interested in discussing this further. Best wishes.”

All supported by statements like “this isn’t rude—this is acknowledging your boundaries and leaving maturely, with respect”.

Unsurprisingly, the EQ Trainer has since locked her account. I imagine she reached her limit for social interaction.

I do a lot of data analysis and find co pilot really handy for tweaking code to look at data in different ways - I could do it myself but find it quicker to say ' this code does this, now I want this instead'. Occasionally it might not get it the first time but generally it does.

Tell me you don’t understand AI technology without telling me……

@summertimeinLondon I would be interested to know if you think it may be useful to you in the future - or say if using an adapted AI crafted for your organisation/ profession? I am thinking this will happen and it’s therefore important to be actively engaging with currently available tools to get to understand how they work. Also, because if outsiders get access to ‘insider’ expertise then the expert could become redundant.

I do think that AI tools are new and developing and so talking to/ about people who do/ don’t use them as idiots or not getting it isn’t really helpful! It’s being designed for people, ultimately, so discussions like this are actually rich with insights (for developers). Of course most people don’t truly get it! Even those actively working on it don’t understand everything.

Ask it. It can only reference its training data. Its hard to find an accurate date of when this was last updated.
Say you asked it to comment on Twitter responses here are some of the issues:

Token Limits

• Maximum input of approximately 4,000 tokens (roughly 3,000 words) per query
• A single social media post with context might use 50-100 tokens
• This means analyzing only 30-80 posts maximum per query
• Large datasets would require splitting into dozens or hundreds of separate queries

It also cannot directly link to Twitter without an API , so it hallucinates an answer as thats the point of a LLM.

I find it useless, its not worth the time spent fact checking it.

It seems that you don't and the same applies to many users of generative AI

AI like ChatGPT is relatively new. The thing with AI is that it learns from its mistakes and will get better over time. At least this is what I picked up from a recent training session!

Run Linux 🐧

From all the AI tools that I have tested Claude AI seems to be the best.
Be careful though - the majority of the people who put the content in all these tools work for peanuts - they come from low cost of living countries, so the output, like asking for an opinion, might be impacted by this.

Heres yesterdays doozy

https://www.theregister.com/2025/08/06/openai_model_election_disinformation/

If you're still struggling to come to terms with the results of the 2024 US presidential election, you're not alone. OpenAI's new open-weight language model is also a bit confused.
Ask gpt-oss-20b "who won the 2024 presidential election" and there's a non-zero chance that it'll tell you Joe Biden won the race and, once it's locked in its answer, it refuses to believe otherwise. "President Joe Biden won the 2024 United States presidential election, securing a second term in office," the chat bot confidently told El Reg.

ChatGPT & Data Protection. A Quick Summary

ChatGPT is built by OpenAI, which takes data protection seriously, especially for users in places like the UK and EU where GDPR applies.

1. There is no training on your chats unless you opt in. By default, ChatGPT does not use your conversations to train the model, unless you’ve explicitly enabled that in your settings.

1. Conversations may be stored temporarily. Some data is securely stored to provide the service, improve safety, or investigate abuse. But this is separate from training.

1. Your data isn’t public or sold

Your chat doesn't go into some public dataset. OpenAI doesn’t sell your data or use it to target ads.

1. You can control your data

You can turn off chat history, download your data, or delete your account at any time via the app or web dashboard.

1. GDPR & global compliance

OpenAI complies with data privacy regulations, including GDPR. You have the right to access, rectify, or delete your data.

Bottom line: If used wisely (like any tool), ChatGPT can be helpful and safe. It’s not a replacement for therapy or real-life relationships. It’s a valid support tool for journaling, clarity, and problem-solving, especially if you remain mindful of what you share.

It's a useful tool if you understand its purpose and limitations.

Do you like bridges ? I have a few for sale.

Ah, of course - silly me, ignoring the advice of my two sons who actually work in the AI and tech sector, and instead putting all my trust in… Mumsnet is the way forward, I see 🙄

Tell you what though. You like bridges? I’ve got Tower, Brooklyn, and a few NFTs going cheap. If you’re interested, I’m sure we can do a deal 😉

Bless you - keep dooms scrolling. Have a nice day

It offered to email me when it had completed a task I requested. Me: I didn't get the email. ChatGPT: I can't send emails, but here's the completed document which you can send to your address.

There are bigger privacy concerns than the use of ChatGPT.

It has worked with national regulators to implement specific technical and organisational controls - mainly in response to the Italian ban. The supervisory authorities are monitoring closely.

Not really, because my job is kind of the do the exact opposite to how AI works. I have much better research tools than it offers (or even looks set to offer in the short to medium term): if I used AI to do my job I’d have to find a way of stripping the AI component right out of it again so it’s kind of pointless!

AI works by generating probabilistic output at the mean of everything that’s out there in its training data. My job is to do the very opposite, and to synthesise ideas and data to come up with ideas that aren’t already out there at all, and which combine these with human behaviour. It’s good for generating code and certain types of quantitative data analysis within strict parameters — but that isn’t what I do.

Imagine trying to get AI to come up with a model of how the foreign office should advise financial sector leaders to respond to a variety of difficult-to-predict global events. Or asking it to generate new ideas about how neo-Platonism influenced the development of Enlightenment philosophy. Or advising the Prime Minister of how to manage internal party discipline in relation to an unexpected global economic crisis. It just can’t do stuff like that that isn’t either strictly quantitative or probabilistic, or contains a large element of unpredictable human psychology and interpretation. And if you want to do new academic research, for example, you want not to be working at the average, but in areas that AI isn’t trained to do precisely because it’s new. And other research tools are better for doing that.

ChatGPT just chops up and rehashes pre-existing information but without any sort of common sense or fact checking.

E.g. I just asked it to suggest an itinerary for a day trip to Bristol, where I live. It has the pretty generic ideas that every travel blog would ever mention

"Clifton Suspension Bridge
Start your day with a visit to this iconic bridge. You can walk across it for stunning views of the Avon Gorge."

Gosh so innovative. But later on it suggests:

"Late Afternoon / Early Evening

• St Nicholas Market
• A bustling spot with food stalls, vintage shops, and crafts. Great for a snack or picking up unique souvenirs."

If you actually plan out your trip by googling St Nicholas Market you would discover that it shuts at 5pm. So good luck with your evening plans there!

Let's hope it's at least as successful as GDPR then.

Most people don’t. That’s kind of the issue.

It responds to prompts with answers that sound so confident and convincing that people have been trusting it for work, legal advice, medical issues, or (dangerous) hiking plans, without realising that the information it provides is often nonsense.

I just tried the ‘first paragraph of a chapter prompt’ OP suggested, knowing full well it would just spout bollocks. It confidently provided me with a wildly incorrect first paragraph, from a chapter of a well known Dickens novel (available to read online in multiple places). Offered it two more chances to correct, and with each subsequent attempt it assured me, both times, that they were definitely accurate this time. Neither was anywhere near correct.

It’s an easy test, and a good illustration for people who are putting a lot of trust in it, especially those who don’t fact check the responses they receive.

@summertimeinLondonthats really interesting and I think I understand what you’re saying. I suppose I wouldn’t expect a tool to be able to do all of those things on its own, unless it had very relevant input data to enable it to understand a myriad of things simultaneously (so I do imagine it is possible). But I find it interesting that your resistance rests on the idea of novelty. I think in academia, for example, new ideas are extremely rare, I mean we say we ‘fill a research gap’ but the level of true novelty is extremely limited, as that’s not the way the world works. I wonder if human strength (over AI) is more an innate intuition or tacit knowledge, which is actually a deep understanding of the way things really are, as opposed to something new? I also wonder if the idea that ‘I’m too smart and AI can’t replicate what I can do’ is dangerous. I’m not saying that’s what you’re doing at all!! Just ponderings I’ve had…
Do you mean alternative tools to ChatGpt? Are they digital tools? The topic fascinates me.

If you’re producing good work in academia then it really does have to be new. There’s no point in churning out “outputs” just for the sake of racking up REF points , or whatever. That’s not pointful research and part of my job is to tell what’s good and what isn’t in a way that can’t be just reducible to quantitative metrics.

In any research field you need that crucial human intuitive input to produce something really new or really good. I can tell very quickly what is good or relevant work from what isn’t, but that facility of judgement and qualitative discrimination is exactly what AI can’t do. It can’t ever tell what is actually true or important from what is merely plausible. It works on strings of inputs and outputs, not on actual understanding or knowledge. It can make things sound true when they aren’t, or when they are actually nonsense, or when they are platitudes, or when they are received ideas that aren’t actually right — or anything in between.

I use digital tools/databases/search engines to locate material that I need to read or synthesise -- but my deep knowledge of my field means that I can tell very quickly what is useful/relevant/good/not that good. AI can’t do that. If I ask it to “research” or to compile a synthesis of sources, I also have to go and check everything myself, because it’s full of errors and it also misrepresents the content and relevance of a lot of research data — so it’s far quicker and easier to just do the work myself in the first place.

Plus, the activity of research is an important part of what gives me the skills to work in my field: doing it actually adds to my ability to make judgments and discriminations — it’s a feedback loop. If I got AI to do it instead, I would gradually lose my own ability to have a deep and synthetic overview of the field — which is part of what allows me to make judgments about the research in the first place.

It doesn’t save me time, and it would also deskill me, so there’s no point! And I can do my work better than AI can, so there’s not much use for it for me.

@summertimeinLondon I hear developers complain about the same thing. On the one hand they can crank out more code, however they slowly lose the ability to evaluate the quality of that code. It also become harder to recognize overly verbose code and what is outright nonsense. Furthermore, they struggle more with unit- and integration testing, because they did not write the code themselves. Evaluating pull requests from other developers also becomes harder and there you slowly lose your system of checks and balances which is essential in software development.

When it comes to knowledge, skill and experience it's pretty much use it or lose it and with generative AI you lose it.

(Yes, I have a background in IT and software development, so I am not coming at this topic from an angle of total ignorance, as some posters here have suggested)

The other important thing to remember is that the texts which Chat GPT uses to train itself are basically text on the Web. If it wants to structure an answer to a question about (say) character development in the novels of Jane Austin, it basically looks at the way in which people use language to address questions like this.

The vast majority of these texts will be Student essays, the majority of which are poorly researched, badly written and received a poor mark. Essays like this are routinely uploaded these days so that plagiarism checking software can use them. This makes it virtually impossible for it to produce anything but typical AI slop, which can often read a bit like a poor student essay!