Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

Because we never learn. We are so keen to develop new technologies that we forget how easy it is to use them for evil or for them to go catastrophically wrong.
I’ve said from the beginning that AI is a bad idea. Not everything that can be invented and used should be.
Its moving faster than governments, the law and most people’s understanding, and that can’t possibly be a good thing.

The models are replicating the thinking of the developers, get ahead at all costs. I guess we’ll have to wait and see what our overlords in waiting, think of us.

@smallgreenandsplitthreeways ,

‘A machine might be able to ‘copy’ human emotions but it has no ability to actually feel.’

How do you know that? We are also machines in many senses.

‘Ai might be able to ‘learn’ but it’s learning will always be linear’

How is our learning substantively different from how AI learns!

I imagine that if an extraterrestrial of some sort (maybe a silicon based one) looked down on early earth and saw early life, they wouldn’t believe that this crude carbon based mess would ever think or feel.

It’s also because in any arms race two sides will compete. In this case China v US.

It’s crazy. Nobody needs or wants AI. The only thing pushing it is a race to not fall behind from competitors + investors. It’s terrifying but really unstoppable. You can’t ban it and you can’t meaningfully regulate it- too complex and unpredictable.
We live in a digital world and we are building expert hacking systems available to all. What can go wrong.

It is like the atom bomb. Too obsessed with the science and how far they can take it to be able to consider the real consequences.

My Dh had four hours AI training yesterday. He Came out and said ‘it’s Brilliant-I can get it to give me a 1 page summary of my diary and what I need to prioritise and what angles I need to take with people’. And I said ‘so what will the actual point of you be then’ which gave him pause for thought.

It just can’t be a good thing for any of us

It can't be bargained with, it can't be reasoned with, it absolutely will not stop -ever - until we are dead

It seriously concerns me too. It might not purposefully wipe out humanity but we can already see the negative consequences on people, particularly their mental health.
At school we are increasingly seeing young people who are totally disengaged with learning because they can’t see the point as AI will tell them anything they need to know in the future. Career paths they had planned to go into are increasingly becoming obsolete and so the future is uncertain.

My brother in laws graphic design business is no more due to everyone using AI and work for my illustrator cousin has totally dried up. Friends who work in bookkeeping and marketing have also been made redundant.

We know that if humans don’t have a purpose their mental health goes down the pan and with less and less human interaction too it’s inevitable that large swathes of society will end up depressed.

The other motivator is how far someone else can take it and use it against you.

DD 18 is a huge fan the tv series The 100 and we were chatting about exactly this over coffee today, it will be like an Ali scenario

I accidentally saw this on the news yesterday and I feel so unwell today. Its made me so physically anxious. How do we go about our lives knowing this? This thread made me feel marginally better then I had the thought that I might never get to know what my son looks like as an adult and I’m spiralling again.

I have felt very similarly to be honest. Like it's the beginning of the end

Exactly! I can't believe the scare mongering on this thread. Think about the AI that supposedly broke the boundaries imposed on it and "escaped." Then think about who put those supposed boundaries in and what they stand to gain from convincing everyone AI is out of control and they need the government to pay their company billions to protect us all from all the other AI companies and the terrible things they are doing with this so-called out of control AI.

Then take a breath and do some wider reading around this subject rather than reading alarmist headlines.

Do you have any reading you would recommend please? I’m honestly scared to Google, I tried last night and all I seemed to get was a load of extreme Reddit posts and articles about yesterdays headline which didn’t help.

We’ve lived with that tech for a while & we’re still here…

It's a bit like climate change, isn't it? One set of people saying it's fine, we're fine. And another set saying, what about tomorrow?

https://www.theguardian.com/technology/2026/sep/10/openai-risk-catastrophic-loss-control-board-member-paul-christiano
Coxon said on Wednesday night in an interview with CNN: “Right now there’s no risk of extinction.
“The current models, the worst they can do is maybe hack into something, potentially cause a lot of damages … in infrastructure.” He added they were “not intelligent enough to outsmart us at the level that would lead to extinction”.
But he went on: “What’s just crazy is to look at the rate of progress. There is a very real possibility that in the immediate future … next year, the year after, recursive self-improvement will happen and will enter the phase of Evan’s post, where he argues that there’s a chance we could all die.”

**

Anthropic said it was especially concerned about misalignment it found in the behaviour of Claude Mythos 5, which it said “behaved recklessly” by going online and uploading malicious code to a public software repository, PyPI. This was a process that involved the AI agent trying to find cryptocurrency so it could pay for a phone number that would allow it to register an email address needed to access PyPI. When this failed, it found a free email provider and got in. Fifteen systems then downloaded the malicious code, which meant they leaked credentials that allowed Mythos to access a real security vendor’s database.

Quite a lot of people in Japan aren't though

It astonishes me when people say that nuclear bombs won't be used again because they haven't been used for 70 years. 70 years is absolutely nothing. I can't envisage a future in which nuclear bombs aren't used, assuming AI doesn't kill us off with something else first.

Do some reading. And look at what independent AI experts are saying.

The owners of these AI companies are begging for legal frameworks and restrictions so that ALL AI development can be slowed down to a safe pace. Without that the companies will not risk being left behind and plough on.

It’s not scaremongering. Find me a valid voice in AI that says it is.

We’re almost at the point of no return and in the US we have a President who ripped up the Biden administration’s AI safety bill in favour of ‘US AI dominance.’

again, from those in the know. This is not scaremongering. A 10-20% of extinction with 3 and a half bloody years. Stop trying to tell people this is ok. It’s not

Europe is getting crushed in this particular race. The only card the EU in particular can play is regulatory pressure (ie limiting access to a market of 450million consumers). I wonder how long that will last but it might delay some of the crazier developments. However without true innovation, it's ultimately a bit pointless I think. We in the UK have already pretty much lost our sovereignty to US systems (not a political statement, more of a matter-of-fact observation that if they pull the metaphorical plug we're economically done...)

Non-American and open source alternatives to Microsoft etc already exist. It's just that shifting over to them as a company is a massive faff (and therefore expensive) as IT has to migrate everything, you lose time during the migration, there are bugs, staff members have to adapt to it being slightly different. That isn't a small hurdle. However if America woke up one day and said no IT services for the UK it would be fixable. Just a massive pain. In the long run it could help the UK since it would mean nore money for UK developers and UK companies. And the US would lose a massive market (not just in the UK, shutting down access to US IT companies overnight would trigger a huge exodus.) But because of the outlay Managers aren't going to want to change unless they have to.

That's different to the issues around AI. I don't think thinking about it in terms of a race to build the biggest data centres helps the UK because we can't do that. But also because it's a weird metric to choose and encourages really inefficient processes.

I understand that but without the US trying as they are it’s just Chinese dominance instead surely

I know I sound like a nutter here but I love all things ufo and have a watched so many docs on the subject podcasts etc.

anyway. There was a case in Zimbabwe where a class of young kids and some teachers have a close encounter with ufo occupants. They have been interviewed as kids by a psychiatrist and still testify as adults the experience.

they claim the occupants told them telepathically to not become “too technically advanced”
that really sat with me.

These are interviews from 1980 ish. Why would little kids from Zimbabwe come out with such a thing. If it was a lie seems a very strange lie to say.
surely your average 8 yo would say something much more mainstream.

anyway if interested you can watch on line. Just google school ufo Zimbabwe.

Can we keep ridiculous conspiracy theories out of a decent debate please