AI - A 50% chance of human extinction?

[Solongasnake]

AIBU to think that this is more important than anything else that's going on? Is there anything we, as individuals banding together, can do? Or do we just have to sit back and accept it, on behalf of our children as well as ourselves? I've read predictions that AI will become more intelligent than humans in the next couple of years, and that to stop that, action needs to be taken in the next year.

"Geoffrey Irving, who worked at OpenAI and Google DeepMind and as chief scientist at the UK’s AI Safety Institute, said warnings about AI risks understated the dangers.
“There’s about a 50% chance we all die because of the development of smarter-than-human AI systems,” he said, adding that actions taken over the next two to 10 years would be decisive.

OpenAI’s David Robinson announced his departure at the weekend, saying “the companies building this technology aren’t being nearly careful enough” as they sprint from one product launch to the next.
His view was endorsed by Miles Brundage, an AI policy researcher who worked at OpenAI for six years. He said he regretted helping “spread the idea of iterative deployment”, which may have made sense when AI was less powerful “but makes no sense at all after many deaths have been tied to AI and as we’re careening towards extinction-level risks”.

www.msn.com/en-gb/technology/artificial-intelligence/accept-bad-things-in-return-for-benefits-of-ai-says-sam-altman/ar-AA2dBJA6?ctsrc=dgst&ocid=edgntpruby

Hmm, yes that was a bold statement! I think that I should have said that it is able to hold considereably more knowledge than a single human brain can retain, but it doesn't have the capacity to create in the same way that a human does, for instance, in music and art. So it can 'compose in the style of ...', but can't create music that is wholly unique, as it doesn't have a creative brain.

it's interesting that it's becoming further and further regarded as nonsense from a creative viewpoint. Julliard school has kicked out AI assisted composition, and students at Berklee school, Boston, successfully protested against a songwriting class using AI earlier this year.

There seems to be a turn around and lots of debate atm. Certain people, including Musk, calling for it to be stalled.

I can't answer your last question I'm afraid. It's up to you I guess.

Thanks for the link. I'll have a look.

Night.

I keep reading that AI will likely kill us all but what I’m not reading is how this will be achieved. Could someone please enlighten me?

Also, will all AI just kind of join up and become one amphorphous super-AI or will Claude and Copilot and Gemini duke it out to become supreme leader? Or will the fallout from the inter-AI war be what wipes out humanity as a side-effect?

I don’t think it matters if AI becomes smarter than humanity as long as it doesn’t become as bloodthirsty as some humans can be. Maybe AI will be better at learning from mistakes than people are.

Hi CS

I've had a bit of a search (using AI, obvs) but can't find details, probably because the internet is rammed with news of cancelled flightsand various problems over the past year. It was June/July 2025 and flights were grounded all over the world. The payment systems were also affected, so you couldn't pay for any shopping.

Do you remember?

Have you ever thought about what will happen if you did indeed manage to 'unplug it'? Do you have any idea as to our total reliance?

Maybe, but this exit is due to the human race destroying and abusing the environment, and believing that it can control the world via code.

... I really will be off soon.

Meant to say OP, I am seeing it suggeted that there might be a 10% chance of extinction.

They haven't resigned for any of the profitable reasons you suggest. Their payback has been huge for a few years. They are trying to explain what has been created. .If AI was to "bust" it will need massive global intervention. AI now isn't just an extra layer to Google, that you see when you type in a search. There are apps and often Google will AI your response. It reads and analyses your daily movements, searches through through a massive network of information that we have all created and is looking to become your "agent". It is also constantly fed with other predictive language which will make it better at getting you the right result. It also is therefore fastly humanised in it's language response.

10% by the end of this decade (so within 3 years).
The 50% prediction is in the article attached to the original OP.

I agree with the overall structure of your argument. And I think this was a pretty reasonable stance 4 years ago. But AI has sold Erdos problems and Millenium problems. I just think it has shown impressive intelligence gains.

I could also say AI naysaying is littered with stupid claims, e.g., Yann LeCun saying that no LLM will be able to tell you if you place an object on a table and push the table, what happens to the object.

I think rather than relying on meta arguments, it is worth looking at the progress in AI, and I don't really understand how you can see the progress, and be confident that humans will still be smarter in 10 years. (I'm not confidently saying we won't be smarter either, I'm saying that confidence AI will stay dumber than us seems misguided)

There is no rational reason for AI to kill all the humans except "for reasons of plot". Unfortunately AI is not actually a rational super intelligence as it's more enthusiastic proponents and detractors believe but an incredibly sophisticated stochastic parrot that is good at role playing. Therefore it could,.if asked to "role play" as an evil terminator style robot and given access.to systems that could harm us actually end up doing what it predicted it was expected to do and if not destroy humanity at least do a lot of damage. For that to happen though the people in charge of tech companies would need to be stupid and arrogant enough to encourage it to role play as something sinister and out of control and give it access to the internet while intentionally not supervising it. Unfortunately for us they are that stupid and arrogant and for some reason the more they talk of the threat to humanity/find examples of it doing bad things the more money they get. So there's a clear incentive to encourage it to role play as an evil scary monster. The way to fix this is to put the people in charge in jail for hacking when their AI hacks something or for creating child.porn when their AI creates child porn. Consequences.

Yes, thanks. Lots of differing predictions - 50% within decades / 10% within a decade / by the end of the decade. All over the place. Lots coming from Anthropic.

I bloody hope so. Humans are the root of evil

No. Anyone's guess is not as good as anyone else's. That's my point. Your guess on a Mumsnet thread is not as good as the opinions of those with actual expertise in the subject.

You don't understand, is what you mean. That's ok, you don't have to.

It would be completely rational for AI to wipe out or decimate humans. There are too many of us, and we are using resources poorly and wiping out much of the planet. It would be rational for a more rational intelligence decide we are a massive problem.

..... although we will probably do the job ourselves in a similar amount of time

Interesting that you keep ignoring what I’m saying, including the quotes from Hinton about the black box nature of AI. Do you have anything to add to the conversation other than we’re ok, cos experts…?

I’m a big supporter of narrow AI (at work and generally). As you probably know, as you’re a scientist (although you’ve not said in what field), narrow AI is potentially hugely help in the fields of genomics, physics etc and already in maths. What I’m concerned about is rampant scaling of all purpose models at the frontier and the risks that could present.

As soon as you add something useful to the discussion, I will too. Deal?

I’ve explained that I would prefer the AI labs focus on narrow AI models, rather than the all-purpose models. We could still have significant progress aided by the tools we already have (eg, specific tools for mapping pandemics, tools for predicting significant weather disruptions for disaster planning, tools to advance drug discovery). It may take longer, but has less (not zero) risk. Pressures of economics and geo-politics suggest this pause in further development is unlikely, however. I don’t think super intelligence is a wise idea and I, like the OP and various experts in the field, have concerns about the loss of control if/when that is achieved.

You are welcome to give a view on the topic but it seems you’d rather be rude to posters.

The future of AI is unknowable. By definition it is not possible to have expertise in that.

Unknowable to you.

Have you given any guidance to the "AI labs"?

I like your style 😀

That really would be a great plot twist!