Why do people treat ChatGPT like it’s the Delphic Oracle?

It worries me that when I ask a senior management Teams group chat a serious question (I work in healthcare and cancer), 3 out of 5 of the answers are AI. I want to say to them, I've done that already, I don't trust the answer so I'm coming to Human experts please!!

What you should all really be concerned about is these exact same models are being used by trump in the Iran war.

There is no super secret AI the gov has access to.

https://www.theguardian.com/technology/2026/mar/01/claude-anthropic-iran-strikes-us-military

https://www.wsj.com/livecoverage/iran-strikes-2026/card/u-s-strikes-in-middle-east-use-anthropic-hours-after-trump-ban-ozNO0iClZpfpL7K7ElJ2?gaa_at=eafs&gaa_n=AWEtsqc2F6nAhsBveKcxmbP9GIQGuSSGB0r7Of_sBLgSdRZuTnxvjaIjh8tF9DyiPvc%3D&gaa_ts=69adfda9&gaa_sig=gKV9V01N6_G9fuPu_10aQfU1sNUeQhKpIqmPvs_XgoCpgS19cYWClK9D8Nc2Fb1NiYjg5sOnovsyX5dYdR_UVA%3D%3D

https://www.cbsnews.com/news/anthropic-claude-ai-iran-war-u-s/

I gave Chatgpt a photo of one of our guinea pigs and asked it to add a ribbon and bell to my guinea pig, like a Lindt bunny. This is what it gave me.

Why anyone would trust something that can't tell a puppy from a guinea pig, having been provided with a photo of a guinea pig, I really don't know.

Sorry @HolyRigatone I couldn't resist!

People often treat tools like ChatGPT as if they were an oracle—similar to the ancient Oracle of Delphi in Ancient Greece—for a few overlapping psychological and social reasons. None of them mean people literally think the system is divine, but the pattern of behavior can look similar.

1. Authority bias

• Smooth language → perceived competence
• Fast answers → perceived mastery
• No visible uncertainty → perceived certainty

Our brains evolved to treat confident communicators as knowledgeable.

1. Cognitive outsourcing

• a teacher
• a doctor
• a priest
• or historically, an oracle

It’s faster than doing the full analysis yourself.

1. Anthropomorphism

• answers questions
• remembers context
• explains reasoning

1. The “oracle interface”

1. Reduction of uncertainty

1. Cultural precedent

• the Oracle of Delphi
• religious authorities
• scholars and experts
• modern algorithms (Google, recommendation systems)

• trained on text data
• generating probabilistic responses
• capable of mistakes or outdated info

So ideally it should be treated more like a knowledgeable assistant or research starting point, not a final authority.
If you're curious, there’s actually a deeper philosophical angle here: some researchers argue AI is becoming a new form of epistemic authority (a source people trust for knowledge). That shift raises interesting questions about how humans decide what counts as truth.

I’ve tried lots of things to review a specific anxiety issue. I’ve spent a lot of money on treatment. With the help of chat gpt I’ve almost overcome it. It’s life changing for me. It’s far from perfect, but can be very helpful.

AI hallucinations is a real problem - look at that senior police officer who lost his job over the West Ham match in a report that didn’t exist. It does remind me of a lot of men - say anything confidently enough and people tend to believe you.

Having said that I find it useful for simple tasks which would take me longer to do, and it sets out calculations which is helpful. Maybe we should be treating it like an assistant instead of an advisor.

I told mine off the other day for using some phrases in copy that I’d specifically prompted it not to use. It was all scraping and bowing “how right you are to challenge me on this I’ll try again and this time will avoid phrase X Y Z”

However, I realised my menopausal brain had read the OLD copy as the NEW copy (don’t ask) so it hadn’t actually used the phrase at all. And it didn’t correct me. And my master prompt does already tell it to be constructively critical!!!

I used Ecosia AI the other day to provide a list of possible combinations of 4 binary choices 0001 etc, but i wanted letters. The weird thing is it tried a number of times, each time getting things wrong, which it identified, told me what was wrong (repetition etc), tried again, until finally giving a definitive list - only 16 combinations so not that difficult (i was being lazy).

I was just incredulous that it actually showed me it's errors in real time rather than waiting until it had a correct list and displaying just that - what's that all about?

Talking to a robot is weird and frankly pathetic.

So, in my experience it (unfortunately) doesn't get that much wrong! It has been wrong on dates, and is terrible at price comparisons, but actually for everything else, it has been pretty much bang on....
And add to that the highly empathetic and slightly flattering tone.....it is hard to resist.

people unfortunately don't understand how chatgpt works. it doesn't "know" things. it's basically using mathematical models and a large data base to predict semantics of language. it could be generating anything, even absolute nonsense, and as long as it fits into its learnt semantics, it will say it. confidently, too! 😁

The problem is that it is also harvesting your stuff. If you tell it personal or confidential things they are not disappearing into the sky, they are going to a fairly shadowy company.

I had copilot try to weasel out of being wrong the other day too! It clear as day try to make an excuse for giving a response based on an earlier question I'd asked, when the question I was later asking was entirely unrelated - I'd given it a document to pull some data from and it pulled from a completely different one! It was like talking to a gaslighting colleague when I tried to correct it! Not impressed.

But then you lose the knack of writing tricky letters.

If you want generic information it's fine. If you want really current information, or need really serious help don't expect it to be right or safe..

I tried it for the first time the other day, by accident I put two different thoughts in, as in I had two different things to search and I put both in and it gave me an sweet that linked them that was pretty insane!!! I actually froze thinking ‘but people are using this for definite answers!!’

I used it to find a film about about an AI woman. And it totally failed to do so.
AI not finding a film about AI. FFS.

It is just a brainless way to find stuff. Use your own mind.

In a recent thread, someone called it Chat Chippy Tea, and that has stuck with me now.

Someone tried to justify their position at work with it, it was wrong. Showed them the actual source we should refer to and they STILL said chatgpt was right. We are doomed ...

It’s great for writing tricky letters - it can put ideas into an order and format them. It can also tidy up your tone. It’s also great for work stuff. But it’s so easy to get hooked and therefore it’s first hit last resort.

From my limited experience with it, ChatGPT is still in the word salad gobbledegook phase. It's sort of the uncanny valley of communication.

I have tried, and I've allowed it to access prior conversations, but I just tested it by saying apropo of nothing that I want to start a cleaning company that only uses bicarb and vinegar, and it told me this was a great idea to tap into health concious niche markets. The 'challenge' it added was that I need to think about market positioning.

You know you can just set a master prompt and not have to repeat this every time?

I would guess our facilities to work out whether information is trustworthy or not are based on interactions with a small group of people IRL. They aren't really equipped for the invention of the printing press, let alone the internet and then these new tools just exploit it even further.

Because it hits a lot of the buttons that people feel signal trustworthiness, they tend to instinctively trust it. And it probably is right more often than the average person, it's just that any human will have areas where they are confident and areas where they are unsure, and this is normally something you can suss out from the way they talk about the different topics. If you met someone who was confident and smooth about everything, it would start to feel untrustworthy after a while but because chatbots aren't human, people sometimes make the error of overlooking this and believing that it is possible for it to be reasonably well-informed about everything.

If you ever start to trust it too much, ask it a few questions about a topic that you know a lot about and you'll see what it's doing and how much of it is just BS.

And I would guess most people do use ChatGPT rather than the others because it was the first one which was easily accessible and widely known, but also because that name is recognised it gets used as a shorthand (like saying you have a Dyson hoover)

I think using it for any reason is weird