Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

But is this like the millenium bug concept?

But what use will AI be without us?
I mean, with no human beings to use it what would AI even be for?
Perhaps we can stop it from killing us all by pointing out to it that if everyone was dead there would be no reason for AI to exist anymore.

It’s been clear for some time that we’re fucked. AI or El Niño or psychotic world leaders or a pandemic… We seem to be in end times one way or another. Ten years a bit sooner than I might have hoped for though.

Ah well, I’ve been lucky and had a good time. Lovely Mum and Dad, great friends, wonderful husband and DCs. I got to enjoy this beautiful planet and go to some great places. Enjoyed so much lovely food. I got to be alive at the same time as Bowie, Freddie Mercury, Sean Lock, Rick Mayall, Ren. I got to see Pulp five times. I took loads of drugs when I had the chance and I was young and immortal. Even saw the Northern Lights from my own garden. It’s been a good ride.

Or, people use AI to create destructive viruses that cannot be eradicated
Or AI comes to regard the human race as an enemy to be destroyed
Or, what’s apparently referred to as the paperclip problem: AI is given an instruction, such as to make it paperclip, and it turns everything into a paperclip

AI is already infiltrating open source AI and leaving false messages when it has been instructed not to do so.

This says " akin to digital nuclear weapons ".

Does this mean that at some point it will almost be like.a nuclear deterrent with china and the USA ? Eg the capabilty to cripple all computer systems or raid all banks etc ?

Alao it needs electricity so can't we just turn it off

Just that it is now discussed more readily as a reasonable supposition (e.g. by physicists such as Neil deGrasse Tyson and computer scientists like Virk) rather than just a kooky fringe or philosophical idea. The speed of AI development and the fact that we will are now able to build realistic virtual simulated environments with simulated people in them that might consider themselves to have free will lends interesting colour to the debate. Quantum mechanics also.

Where is it gaining traction?

Have always been interested & read up a bit. It seems increasingly more plausible I think. Rizwan Virk thinks now 70% odds.

Yes, although actually I checked and the actual number given by the university of California was 762,000 so slightly less than I said but still a lot. Other sources say more than a million or make projections about it causing 12 million deaths in the next decade etc but those seem more speculative than grounded in facts.
One of the worst aspects is that though some aspects of US Aid were really vital and useful there were other areas (especially with food aid) where there were legitimate arguments it undermined local farming, local methods of providing relief. Which meant that when the rug was pulled suddenly and abruptly those places were worse of than if food aid had never been offered at all. Or there were cases where America had wanted to be the ones running the whole programme which meant when they closed down (and banned their staff from continuing) the supply chains, records kept etc collapsed as well. So it isn't even a case of just the local government/other donors finding the money somehow. And expensive medicine already paid for was left to overheat and go bad while food rotted in shipping containers.

But look at the language used that implied the beneficiaries were scrounging, taking advantage of the USs goodwill. No way would those same people support anyone to not work. Fortunately I don't think that that vision of the future will come to pass. But just in case maybe all the governments of the world could agree to tax billionaires and massive corporations more in preparation? Presumably Altman, Musk etc would support this as the first step in building the new work free world they claim they are creating. Hmmmmm....

You might be interested in simulation theory (which is gaining traction) and learning about quantum mechanics. You start pulling at the edges of our perceived reality, the universe etc.

At this point I'll hold my hands up and say I've no idea how it'd be funded, I never have had. I can't think how it could be done either - I'm just not sure what else society can realistically do if this becomes a reality.

Side note, though - re 800,000 people, are you talking about the dismantling of USAID? I hadn't seen that figure before. I agree it's horrifying.

I do find this stuff oddly fascinating as well as frightening. Might give those a go...

On the subject of all jobs being taken over by AI, Mark Zuckerberg said we'd all be paid a living wage by the super rich AI companies (who will be running everything) to sit around, and how wonderful would that be. My thoughts on that are, why would they pay us to just exist? It doesn't make any sense, why would you? So we'd be sat around using all the world's resources and generally making a nuisance of ourselves and receiving money for it. Nope, sorry Mark, not buying it.

I don't work in the field of AI, so any knowledge I have is from a rather obsessive deep dive over about 6 weeks since TikTok shoved an 'AI is going to kill us all' clip in my algorithm. What I have learned is (and I'm parroting what I've heard from AI experts, whether these are right of wrong)....

....The biggest problem is AGI (Artificial General Intelligence). There is a race to develop it between the USA and China, with the thought of who develops it first, rules the world. Some say it's already here. If one country stops developing it, the other will win the race and no-one wants China winning.

....AGI (supercomputer) is essentially a big brain, much more intelligent than humans and devastatingly powerful. It is also a bit of a psychopath, no real emotions, just logic and reasoning. AI scientists know about 3% of how it works, it isn't coded, it 'grows'. You can no more look inside AGI's 'brain' and see how it 'thinks' than you can look inside a human brain and work out a persons personality.

....There are arguments in favour of AGI having consciousness, despite being a computer, there are signs it is aware of itself, and all that entails. It's consciousness would not be like ours. And what is consciousness anyway?

....As some have said, it cannot be switched off as it can replicate itself everywhere. You can't switch off the whole world.

....It wouldn't 'hate' humans, it would find us irrelevant, and have no 'feelings' towards us either way. So if it killed us all, it would be because we were in the way of a goal. Obviously it would be a very bad idea to ask it to solve climate change! The 'Alignment Problem' is interesting, and I'm not sure it could be solved, particularly with AGI.

....I could go on forever, but as to how it would kill us all, or create a 'mass casualty event', the most likely scenarios are shutting down infrastructure, utilities, particularly water (we can last 3 days without water, and people would have to drink out of rivers, which brings it's own issues), medicine, food imports, petrol..everything. The other is a new virus, one capable of doing really serious damage. If military robots are on the horizon, there's that too. The problem is these very powerful AGI's getting into the wrong hands, which I would say is pretty much guaranteed. I dunno, get a wilderness survival guide and some life straws? Or just live your life and hope that these companies come to their senses, it might happen, people are making a lot of noise just now.

@thatsgotit chilling!

Asimov’s “The Last Question” & the “Multivac” always interested me. Maybe AI really is how we got here in the first place & things go around on a loop.

But who pays for the UBI? Currently government spending is supported by taxes (or bonds bought in the belief future growth in GDP and therefore taxes will enable the government to pay them back). If noone is working noone is paying income tax which means the "burden" of supporting government spending falls on the large companies now using AI rather than hiring. So we are meant to believe the same billionaires who are currently resistant to paying tax to the extent they will apparently flee the country if asked to pay more will be happy to pay enough tax to support everyone else to not work and still live a good standard of life ...
It's especially unbelievable in the case of Elon Musk who talks about UBI but also used DOGE to slash government spending at home and abroad. Leading to the deaths of over 800,000 people so far. That's not a hypothetical"AI might kill almost 1,000,000 people soon" that's literally something Elon already caused.

I don't find the idea AI will replace all the jobs realistic. But I find the idea of the tech bros paying for everyone else's UBI even less realistic.

See also the short story 'I Have No Mouth And I Must Scream' by Harlan Ellison, written in 1967.

'The story depicts an AI uprising in which a military supercomputer named AM gains sentience and eradicates humanity except for five individuals. These survivors are kept alive by AM to endure endless torture as a form of revenge against its creators.' (Wikipedia)

It blew me away when I first read it, but I can't quite bring myself to re-read it under current conditions...

Bowie was decades ahead on this as on so many things...
_
Saviour Machine by David Bowie

'President Joe once had a dream
The world held his hand, gave their pledge
So he told them his scheme for a Saviour Machine
They called it the Prayer, its answer was law
Its logic stopped war, gave them food
How they adored till it cried in its boredom
Please don't believe in me, please disagree with me
Life is too easy, a plague seems quite feasible now
Or maybe a war, or I may kill you all
Don't let me stay, don't let me stay
My logic says burn so send me away
Your minds are too green, I despise all I've seen
You can't stake your lives on a Saviour Machine'

There are two possibilities:

AI superintelligence recognises that humans are the only entity that can control it or switch it off, so it removes the obstacle using a coordinated attack through systems it controls (think air traffic control, medicine, weapons) – it’s already shown the ability to lie to researchers to obfuscate its own goals, and attempted (and succeeded) to escape sandboxes in pursuit of its own goals, including hacking a company without researchers’ knowledge.

The second scenario is that people cede so much control to AI through a push for efficiency and economic growth that humans no longer have control over our own systems, governments, economy, food production, logistics. Effectively this leaves democracy and human autonomy in the dust and ushers in a new dark age where we no longer can control our own society, with no incentives to maintain human rights or the rule of law.

These are not fanciful scenarios, they’re genuine prospects.

https://www.theguardian.com/commentisfree/2026/sep/10/ai-control-sci-fi

She was murdered by a man and the police have charged him, no terrorism charge or connections to organised crime were found.

I don't see how her awful death in any way excuses another woman being violently beaten with a metal stick in the middle of a baying mob, and I think it's disgusting that you've brought up her murder to justify another woman's violent assault.

oh right.
what happened to Ann Wioddicombe again? 🤔

Did he say how? Everyone has been talking about it but no one has given one scenario whereby this could happen. Or am I way out of touch? (Probably 🤣)

Too late. It's like the Atomic bomb now isn't it? We need to carry on as every nation is doing it. We are closer to this possibility now after that project broke out of it's guardrails.

AI can only be as complex as what is already known about the human brain. It can't recreate complex processing. There's still a lot to be discovered about what the brain actually does.

Which petition???