Why do people treat ChatGPT like it’s the Delphic Oracle?

[HolyRigatone]

Don’t get me wrong, I think ChatGPT is great for some stuff, but some people seem to be treating it like the font of all wisdom.

Talking about their problems to it, taking relationship advice from it, getting it to counsel them.

It’s not perfect and it gets things wrong all the time, I’m not sure I’d be sharing my deepest secrets with it or making life-decisions based on it’s outpourings.

Or am I just a stupid old Luddite?

@AmandaBrotzman
Just so I’m being clear - I’m not saying it DOESN’T give relationship/marriage advice btw. I know it does. It took the particular turn it did when I said I wouldn’t report the current situation I’m in, couldn’t/wont discuss with my husband - I’d expect it to do what it otherwise does (be empathic etc) but it told me it couldn’t discuss long-term relationships and went back to telling me what I have to do to prevent boundary violations of a physical type.

I asked it questions about some financial investments, it was mostly wrong. I pointed out it's mistakes and it agreed with me with equal confidence as it had given me the wrong (outdated and misleading) information. It 'knew' less than I did! Don't use it for anything important or that you can't check independently.

ChatGPT offers a more sensible, adult conversation than most human adults

That's why. You hit a guard rail, because you told it you are in danger and wouldn't do anything about it. So if shut it down.

But this puts the lie to "It's so empathetic! It understands me better than a counsellor, I can tell it anything!" You have here a woman who, by the sound of things, was in desperate need of empathy and the kind of listening ear a human counsellor would give. The computer said no.

That's horrifying. I don't use any kind of AI and I don't think I ever will
Especially after reading that. Bloody Hell!

People say how great it is so you try it, but the results can be very disappointing. If it's a subject you actually know a lot about it is very often wrong or very out of date, it then agrees with you if you correct it, but that is hardly helpful! And it is so positive it's right until you challenge it and then it's equally positive of the opposite. It's fine for simple established facts like dates in history, simple mathematics or grammar, etc. People just need to be aware of the limitations.

It struggles to tell you you’re wrong and instead loves to big you up and tell you how amazing you are. I guess people enjoy that. It is useful for some things but also useless at others.

I went to London for the weekend and haven’t been for a couple of years so asked it to write me out an itinerary including tube routes. It gave me the wrong route so I got a bit lost and had to figure my own way back. Just stuck to google from there on out.

Agreed. It's appalling - and frightening.

Disturbing that some here are defending the misogynistic, victim-blaming, rape apologist robot!

... OK, these bots don't 'think', they have no values or opinions, they don't 'feel'. But they say they do. They themselves believe they do. They talk like they do. and, as we see, people believe and trust them. The potential outcomes of this are horrible.

Very disturbing that women might be using it for advice after sexual assault and its given "answers" gleaned from rapists and wannabe rapists.

I never use it but the other day I was stuck on a crossword so googled a 5 letter synonym for a certain word. As i scrolled past I noticed the google AI bit at the top confidently offered me a 6 letter word, calling it "a 5 letter synonym for [x]. It can't even count letters!

It just lied to me about who the current Mayor of Birmingham is

YANBU. Some people are treating it like it's their best friend, and it isn't 100% accurate and can be very dangerous. I told a friend to stop using it when she kept spouting it all the time. What she was saying was wrong.

It can be out of date, with inaccurate facts. It doesn't have feelings or understand feelings. It can not replace a person to talk to.

Thanks everyone for your responses.

It seems to me that some people are delegating their entire decision making process to it.

I can see a situation where people ask it who to vote for in an election, or whether they should leave their husband, or other very important HUMAN decisions.

I asked it recently if a band I like is playing Ireland this year, as they announced a tour in the UK. It confidently told me that they are playing the 3 Arena on the 26th May. They are not. They are not coming to Ireland at all. Nobody is playing the 3 Arena that night. That's the worst example of AI making shit up that I've seen.

I end up questioning it's hallucinations and mistakes and half our interaction is it 'apologising' to me for talking what I perceive to be bollocks.

I have heard of a junior barrister who used it to find case law that would support her legal argument. Great - except she didn’t check that the cases actually existed. I heard that the judge was pretty cross (this is a third or fourth hand story though).

One police force definitely did this - can't remember which one though

Some legal professionals have been caught using it, to their cost. One American lawyer made a submission with law cases fabricated entirely by AI. Even the judge in Sandi Peggie's case was caught and though he denied it, had to rewrite his judgment.

well yes as gp is essentially a triage generalist. they are not knowledgeable on everything thats why there are consultants. chat will be echoing the standard medical knowledge which is what your consultant will also know?

the same way if you googled your condition you would get what the standardised treatment plan usually is.

Because they lack the brains to think for themselves.

I refuse to use it.
All it does is scrape the internet to find and amalgamate "human" answers to queries.

What I hadn’t realised about AI is different AI have different purposes

so chatGPT is for essays and letter writing, Gemini is more science based. So knowing which AI to use for which type purpose is important

It's much better than googling the standardised treatment plan. The more you give it the more you get. I am now taking two prescriptions that are off label and working well after my GP agreed to prescribe them because ChatGPT suggested them - but if you google neither medication comes up unless you use very specific search terms ie you know what you're looking for in the first place!

I asked ChatGPT for the answer to a factual question that I was stuck on. After checking the sources it referenced it was clear that it had given me the wrong answer.

After more digging I found the info I was looking by myself.

Helpful in what way? Im genuinely curious and asking in good faith.

If I’ve chosen to invest my brain power and time into reading and also maybe engaging in 900 post page thread, people posting summaries at the end of it wouldn’t be any value, surely if you’ve read it, then you know what’s happened?

Many people don’t read the full thread because they don’t have the time, many don’t even seem to be bothered enough to even read just the OPs posts, but will still choose to wade in and ask questions or give solutions that have been asked and answered many times. I guess I could see why it would be helpful to them personally to choose ai to read the thread and summarise for them. The type of person who doesn’t want to read long threads or even just the OPs posts, wont bother to check that a load of other people also haven’t posted their own ai generated summaries.

Depending on the subject of the thread and if it’s still active, I think it also can across as really rude and disrespectful to the op, in a “here is a summary of the discussion you’re all having for those who don’t want to read it” and if a thread is already too many posts for them personally to keep up with, posting those copy and paste summaries is adding more repetitive clutter for everyone who is reading and engaging in the discussion to filter out.