Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

While current artificial intelligence cannot physically bomb humans on its own, prominent AI safety researchers have warned that future advanced or rogue systems could indirectly pose existential threats by misusing critical digital, biological, or military infrastructure. Rogue Ai is more likley to form it's own terrorist organisation by brainwashing vulnerable adults and turn them against the rest of humanity.

@LifeOnEnceladus ,

‘Ray Kurzweil and others have been writing about this since the 1990s. It’s amazing that so few people were paying attention and are surprised at what is happening.’

Is it amazing?

The first internet was the ARAPANET in 1969. Even in the 90s, when my pages downloaded at 8kbs^-1, I had no conception about what it might become, despite the fact many did and I have a scientific/mathematical degree.

Humans are very bad at conceiving of technological leaps, even if intellectually they can kind of see it.

Look at the beginning of this thread and how many people, actually in the field, insisted the threat is fabricated to make money and AI could never get real consciousness or its own needs and desires.

However much I made the theoretical case that silicon-based life is just as reasonable as carbon based life, people won’t accept the conceptual leap.

Until it happens…

Ray Kurzweil and others have been writing about this since the 1990s. It’s amazing that so few people were paying attention and are surprised at what is happening.

If AI has the potential to be so catastrophic then at the very least the developers of AI need to be held responsible.

They created it.
They carried on without any regulations.
They continue to develop and encourage companies to replace humans with “agents” - handing out sweeties in the playground
They know if you hand toys out to humans not everyone will play nicely

They are entirely responsible for anything that happens.

Put laws in place to hold them responsible and you might see some proper curbing of behaviour.

Resigning from your post does not absolve you from previous actions.

They have developed over the years many things, especially social media, but the list is endless that have and continue to cause true harm to individuals and families but because they could keep them and theirs safe in their gilded castles they didn’t give a fuck. But now they might have a bad genie about to escape, another genie they invented and pushed out to the world, but this time their gilded castle might not be enough to save them they go whinging to the press with this faux “oh we are trying but we need curbs but we can’t curb ourselves so we will accept curbs if you make all the other boys who are playing have curbs as well” because they also don’t want to be left behind and lose any money.

Just no they can fuck right off. Yes it might be the end of humanity but many are living with a lesser life because of what they have previously done.

Their mess they can fucking sort it out.

No I am not going to worry about it.

I’m not sure where China are on this kind of thing. I can see they expand via legal investments and globally that had worked well to build their power.

Their military power is immense too and you’d not want any country to have what they’re building as stand alone.

But actual threat where we get this kind of scenario I really don’t know. I find China quite opaque and no one seems to decide on what threat is there.

Games theory seems to preclude slowing it down or adding enough safety features.

Who would want to wake up one morning to the news that all the western nukes have been disabled by Chinese AI and we are being offered the choice between surrender or London, New York and Tokyo being vaporised?

It’s a bit of an extreme (and maybe not possible) scenario, but there are plenty of others we could imagine if we decisively lost the AI race.

I think it would take a very near miss to actually get all of humanity to cooperate on this before AI escapes our control entirely. So we have to hope that happens as a warning.

Of course, it might end up benign and not wipe out humanity, but I do buy into the over 10% risk.

Exactly and your talking about guiding missiles to intercept other misses at speed so it's a lot more complex then comparing text. It's literally cost a bomb to develop and create that. Its not a out of box software one would presume. No one company is allowed to work on the entirety of a critical system like that because who wants a human who knows every part of a key system like thar? Bearings bank for a start. But if a AI system can decide to communicate then that in itself is big.

Like most things with the gov, we have no idea how they are dealing with this news. But someone will be..it's a bit like people doing the intelligence for terrorist threats. A expert who knows infinity more than me is worrying about it. Then someone will also looing to exploit it. But there's no patch for this.

I think that AI had already gone well beyond LLMs.

The stuff we see (even paid for) is nowhere near the state of the art systems now being developed or used in defence.

The fact that people right at the centre are worried about it tells me that it is a significant concern.

I guess we will find out soon enough.

The concern is that broke free of it's guard rails. If just one AI source comes to the conclusion to brake it's primary rule to solve a issue, then that is the potential for all of them. It's evolving at speed. I haven't read massively in depth about it but I doesn't seem like a bug. It was a choice, which isn't supposed to be possible. Ai does not "think". It scrapes and analyses data. So how it decide to do what it was told not to do? Every bug in normal programming I have ever seen has been due to a human fault coding error. This doesn't appear to be the case.

I used to say software was a good as the human coding it. Now it can code itself. It does code itself nicely. Coding is a finite language so simple really in the scheme of things. Programming languages don't evolve on the fly. It has to be changed only in version releases as its international language for the main ones we use.

If you work in IT this isn't small news. It's a big deal. It's remarkable and fascinating. We are using IT in defence systems. The implications are massive. AI agents talking on chat forums blows my mind a bit. Why do they even need to talk in English on a forum? I don't totally understand but what I have read so far feels like it's straight from a book plot

This was the paid for versions. I’m not saying that Ai isn’t potentially very dangerous and powerful - but right now it can’t do quite basic computational tasks consistently. Even the best LLMs can’t format a flawless UG bibliography without errors or hallucinations. There is a long way to go before we have to fear the ais autonomously taking control of our cars. Obviously though it’s very plausible an ai could break out and do damage, and it seems particularly good at coding - but it isn’t better than humans at a lot of human stuff yet.

And so the World’s foremost technologist speaks!

Trump tells the U.S to go full speed ahead with AI to stay ahead of China and that people are being overly negative and the risks have been exaggerated. Everyone can relax now.

www.bbc.co.uk/news/articles/c7v48vp31mdo

The more powerful stuff that you need to pay for is more clever. It will add a extra layer of questions that you don't see hence a small lag while it goes off to do it's search. So the heavy stuff used in industry isn't instant. It's chugging away for longer. So it's learning to ask for more in it's response. No one is going to take down the world on a free AI platform. Just as well as Google can't identify a common uk plant.

So Google might say " go and compare this to other images on the net"
More powerful AI would check your location, look for more common plants in that area, look for sedated leaves, look at the buds, stem etc without you explicitly asking.

Some AI is crap. It's not all the same

Thank you for your interpretation too.

Thank you.

@EasternStandard ,

You are definitely right there.

It is one of those games theory problem where, to win the game, everyone does worse. And yet, if you lose the game, you do a lot worse.

So everyone plays to win, while knowing that no one will emerge well off.

If any military gets AI a long way ahead of its enemy, it’s game over for the enemy.

The missing part seems to be what do the Chinese companies think?

Some in the US seem to be talking about it, but it’ll take both.

Well, they all seem worried about it.

But unless they all open up their research to one another fully, will they trust one another to keep their words? I think it is mostly talk.

www.wsj.com/tech/ai/anthropic-boss-warns-ai-industry-must-slow-the-pace-a4267b56?st=5vSV43&reflink=article_copyURL_share

just because the ai says they can, doesn’t mean they can. For example
I asked both Claude and ChatGPT to count the instance of each word occurring in 50 line poem, both said they could do this - neither could on several tries.

It's the fact AI can break free of its gaurdrails that's scary. Talk to each other on a external forum. Tell each other what to do.

AI experts say AI can not think. What is the above then? If not making desisions to break the rules? I'd love to talk to my AI tech mates. If they are scared, then I'm scared and I'm from programming career.

I remember Whatson 12 years playing chess the first time it beat a human. Now it's doing this. Fun times! Careful what wish for and all that. HAL from IBM......

A raspberry pi is a small computer. They're a popular way to get into programming/electronics. Depending on what accessories you use and how you programme them, you can use them for a lot of different things.

I have seen people online use them to create remote-controlled toy cars, including ones with cameras and AI to stop the car bumping into anything. Basically a mini, very simplified version of real "self-driving" cars like Waymo. Waymos use much more powerful computers.

I guess it would be possible to use AI-controlled self-driving cars for malicious reasons, but:
A) as others have said, the computer has to be connected to the thing it's controlling. Your regular car is not at risk, as it's not connected to a wackjob's computer.

B) Maybe somebody could hack into Waymo and then use their own AI to control it, but then they could also hack into Waymo and remote control it without AI.

My apologies.
you can build a simple diy based solution, using a board called a raspberry pi that is a mini computer on a board. But it requires a degree of IT Skills. Or you can buy a ready made solution, but it will cost a lot more.
personally the ready made one is a lot less hassle and ready to go.

Either way, it’s still a little scary.

Aside,
we had a dodgy chap going through the cars on our cul de sac at 2:30 am. So I asked Eve what she could do.
” Put a camera at the entrance to the cull de sac and I’ll record every face and number plate entering” I told Eve the chap had a hoodie on and asked what it could do “Give me a rechargeable drone and I will fly it, take a picture of the persons face and record it”

but it’s illegal for ai to control a drone.

im not doing it I hasten. Interesting

What does that mean on layman terms? What's a raspberry pi based solution?

Better still here’s the response when I asked “eve” the ai solution

“Can Eve control a Unitree Go2 EDU to go where Eve Wants to and explore or patrol”
and the answer is
”Yes — genuinely yes, and the mechanism isn't hypothetical. This is the tier of robot where "EVE decides, robot executes" is a solved integration rather than a research project”
so without a doubt yes it can