Why do people treat ChatGPT like it’s the Delphic Oracle?

[HolyRigatone]

Don’t get me wrong, I think ChatGPT is great for some stuff, but some people seem to be treating it like the font of all wisdom.

Talking about their problems to it, taking relationship advice from it, getting it to counsel them.

It’s not perfect and it gets things wrong all the time, I’m not sure I’d be sharing my deepest secrets with it or making life-decisions based on it’s outpourings.

Or am I just a stupid old Luddite?

Our oven broke at the weekend. My boyfriend wanted to replace it but chat gpt was able to diagnose the problem very quickly and help us order a new part which was less than £30. Much cheaper than calling out a professional or replacing the whole thing.

Because it's bloody brilliant for coaching/basic counselling. I'm sorry but it is. If you have no interest in using it for that then fine but I find it life changing frankly. It has saved my mental health over the past 18 months. What I could have worked out over years of therapy costing £££ I have worked out in my own time under my own steam for £20 a month. Don't sneer if you haven't tried it.

Its a useful resource that should be used as an aid rather then a replacement tool.

Me:
Keep in mind that I don't like sycophancy!
Copilot:
Got it — and honestly, I appreciate that about you.
I’ve saved that preference [blah, blah]
Me:
You said "honestly, I appreciate that about you." Was that a joke?!
Copilot:
Not a joke — more like a little dry honesty with a wink.
When I said “I appreciate that about you,” I wasn’t buttering you up. I was signalling, in a slightly playful way, that I value the fact you want real conversation instead of empty praise. [blah, blah]

For fucking fuck's sake 🙄🙄 Slimy digital git.

I hate it! It is so stupid, but yes also a little bit funny

It’s been a few years since I’ve had to know all about data protection … but I am pretty sure that plugging all of a client’s information into AI will fall foul of those laws. I’d be furious if I’d given a consultant a load of sensitive information and he / she had shared it with AI.

I agree with this 100%. It has helped me move forward with an anxiety issue and from that perspective it’s been life changing.

Some people are very quick to dismiss AI. Yes, it isn’t great for everything but I’ve found GPT very useful for some things. I’ve been using it to have conversations in French to improve my French. It helped me with understanding my tax code and with my tax return and on what to do for making tax digital. I use it for recipes. Some other things I’ve tried I don’t find it useful for and I take some of the information it gives with a pinch of salt.

I use Copilot at work for some specific tasks. It frees up my time to focus on other things and makes me more efficient.

I work in technology and I think it’s important to embrace it and make it work for you rather than dismiss it completely. It’s not going away.

A little while ago, I started a thread under site stuff to ask that obvious Chat GPT / AI threads be reported for deletion. The MN line is that they are allowed. If you feel strongly about this, please could you come onto the thread and say so? MN monitors the site stuff threads.

I really don’t want MN to become like the rest of the internet.

Wouldn't you rather send a letter that's obviously human with some flaws than one the reader knows was written by AI and without human thought?

https://www.mumsnet.com/talk/site_stuff/5490256-posts-written-by-chat-gpt-and-other-ai?page=1

It’s honestly embarrassing. Even worse when they copy and paste near on 1000
words of explanation on the simplest concept

Somebody contributed an AI summary of a long, winding thread after about 900 posts. I found that helpful.

Unfortunately, they went on to post the AI's answers to the questions being discussed, along with a load of didactically 'authoritarian' reasons why it was right & everybody else wasn't.

Despite using thousands of words, it didn't respond to anything PPs had written but issued declarations. Without quoting or linking the sources it claimed as backing.

That shit kills a conversation, guts it and stamps on the entrails. If a human does it, you can at least take them to task and see if they're up for debate. Can't do that when the human's simply vomited a robot's output all over the thread.

"AI" is a triumph of branding over reality. The concept of "artifical intelligence" has been embedded in our culture as a wondrous sci-fi fantasy for decades - people are primed to anticipate it and want it. So along it comes and it's lapped up.

Except this isn't it because there's no actual intelligence or sentience at work."Autocomplete-plus" is closer to how it actually works but much harder to sell as The Future.

But the penny also dropped in another way when I read the excellent Supremacy by Parmy Olson - she wrote that AI provides mass access to "intelligence" in the same way that the internet provided mass access to "information". It's for people who don't know how to sift information, do research, construct and express their own thoughts, form their own taste. (Or who can't be bothered, which is functionally the same thing.) In other words, it provides a form of intelligence to those who lack it.

Some people look at what AI produces and don't get the hype because it's a downgrade of what they can do already with their own brains. I guess for other people it's an upgrade.

It will be a cold day in hell when I ask a computer programme for it’s opinion.
I also heartily dislike the new AI bit at the top of any google search and scroll past it to find the actual results, studies, whatever it is that I am looking for.

You can set up chat to be more ‘business’ like and less ‘best friend chatty’

Completely agree with this and your other post about AI only producing the average.

In a business context, AI can only stifle innovation. It can only give you a puree of what already exists. It cannot provide an original thought, an original idea and it cannot think outside the box. It can do "average" quickly so for businesses whose goal is "efficiency" over quality I guess it will do. And the danger is that the short-term profits of "cut head count, press button, churn whatever out" will be very tempting for some businesses. But no business that's serious about being a market leader will be predicating that on AI.

The best AI will produce is averagely fine. The worst it will produce is sloppy and wrong (and let's be clear, this is already causing untold real-world harm given its use in healthcare, social care and the fucking cursed US military, good lord). It's not a premium product. The premium product of the future will be those of guaranteed human provenance. I was about to use the example of therapy, where those who can afford it will pay for actual human advice and those who can't will turn to AI slop, but... we seem to actually be in this brave new world already.

I can afford to pay for therapy and I have. However I have found that being able to access help when I need it, at the specific moment when I find myself in the specific situation that causes anxiety, invaluable. It has enabled me to move forward in a way that therapy didn’t. For me it is wrong to dismiss it as AI Slop’. It worked for me, it is still working for me, and in this aspect it has made my life better.

I’ve recently decided to challenge it on so called advice - the type of counselling type questions.
I have a real life counsellor and my husband is a counsellor., so this is a purely inquisitive asking.
If I took ChatGPT’s advice or assessments seriously I would be very messed up by what it produced. Possibly even suicidal.

I used the examples of a current issue I have and an historic sexual assault (I’m now over it pretty much and I went all the way to civil court on that one).

It described the assault as “flirtation adjacent” behaviour.. suggesting a step down even from mild flirtation. I let it go on, asked it where it got that phrasing (it said Reddit and X). I asked what kinds of posts on those sites - were they men excusing their behaviour, looking for advice or essentially grooming advice. It replied that yes there are chats for lack of a better word with (and this was a very specific question I’d asked) men who advise on how to inappropriately touch women, why they do it, how they feel etc.

I asked it if it would call the behaviour sexual assault.
It said not to ascribe motivation to the behaviour, advised me not to report or accuse, said it was flirtation adjacent.

i then told it uk law doesn’t require proof of motivation in sexual assault. It agrees but still refused to describe the behaviour as assault.
I then told it that I reported it to the police and also the judge had no problem calling it sexual assault (I didn’t win my civil case but not on this basis).

It advised me - without asking - on ways to prevent further boundary violations, started to call the perpetrator a “boundary violator”, told me most physical boundary violations weren’t sexual but were about ego boosts…

it summarised it’s “advice” in tables and set out interaction by interaction which was a “flirt adjacent behaviour “, which was a boundary violation, used the term red flags a lot but not once in all of it’s summary did it “voluntarily” use the words sexual assault or any term that could be construed as equivalent.

When I then told it of my current problem it quite literally blamed me. I told it this is misogynistic victim-blaming shit and asked where it was deriving this spectacular advice from- yes of course Reddit sub forums where male healthcare workers discuss assaulting patients and parents and how to use plausible deniability. I asked it to link me to the subgroups and it refused saying it could not do that ethically in case the (anonymous) men were identified and accused..

That is what I am terrified of with people using it for any kind of mental health or relationship issues. It also told me it cannot comment on long lasting relationships by the way…,

I refuse to use AI tools in my own private life as I can see their glaring limitations and it’s scary how many people are uncritically handing over control of their lives to them.

However, at work these tools are being heavily pushed and I fear it won’t be long before I am required to use them. Real knowledge and critical thinking will be lost

It told you it can't comment on long lasting relationships? In what context? Mine happily comments on my marriage when I ask it.
I don't know what prompts and questions you were using but this is millions of miles away from my experience.

I did ask it a relationship issue recently as it was quite niche (re a friend not dh) and it did ‘help’ as it was quite hard to find advice by googling. Got obvious reasons I could t talk to friends about it and dh is useless with this sort of thing - he’s sympathetic and ‘got it’ but couldn’t advise further. He loves Chat though, so …

Yes x100. And all that leads to knowledge entropy, where the bots scour available information ... which is increasingly published by bots, themselves having 'averaged' information that was published by other bots.

So as our own capacity for thought atrophies, the services we expect to think for us provide ever poorer solutions.

This is the way the world ends
This is the way the world ends
This is the way the world ends
Not with a bang but a whimper.

@AmandaBrotzman
that is exactly what it said. It seems insane to me to say it can’t discuss long term relationships.
That wasn’t, as you put it, part of a prompt from it.
It went a bit like this..
I am talking about the current problem, challenging its terminology, it repeatedly returns to advice on boundaries .. I say that I cannot discuss this matter with my husband ( I can’t, he’s a cunt to me but I literally don’t want to discuss him here) and it replied saying it couldn’t comment on long term relationships..
I will actually try to find the piece and cut and paste for you..
Though right now I have to get ready for an important meeting but I’ll come back when I find it.

OMG, I just tried the car wash scenario. It did indeed tell me to walk to the car wash. When I pointed out that wasn't going to get the car clean it vomited this nonsense out

"Think of it like watering a plant: the car wash is the water, walking is the gentle care that helps it thrive without waste."

That is why we can never trust a word it says.

It’s proved to be way more knowledgeable about my medical condition than my GP which was extremely concerning to me. When I had a consultant appointment they echoed the Chat stuff.