Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

So…in Times comment section on recent article you/someone flags (“Godfather of AI) someone has written:

”It’s the old joke, humanity builds a universal computer & asks the ultimate question: “Is there a God?” “There is now!”.

As AI develops I’ve been wondering about the implications re consciousness, evil, karma etc. If consciousness is somehow bigger than us, could AI eventually become part of it — or even be another way consciousness expresses itself? Is it “God” is it why we’re here (?) if am going deep on a Weds afternoon.

In my childhood we had Pong, basic Atari & earlier still, 2001 was science fiction. Last week I saw a robot walking down a suburban street! (Some promo, but still). So is it really completely bonkers to wonder where all this ends? We assume we’re creating AI, but could something stranger eventually be going on — could it even end up effectively ‘at the helm’? So rather than wiping us out, the ‘point’ is we exist until we create our own conscious Sims or something?

Rizwan Virk thinks the odds of us living in a Sim now about 60 per cent.

They don't need to exterminate all life on earth. If they exterminate life forms other than human, I'd guess it'll just be because they were in the wrong place at the wrong time when AI exterminates humanity.

We have to make its electricity.

It's stuff like this which makes me, a lifelong lefty, thoroughly ashamed of what the left has become.

Yes, but at least in the meantime a few men will get to feel rich. And that's what really matters. /s

Shut down the source of the power. Everything can be shut off. We can survive without electricity. Ai can't. It doesn't run on magical ether, it needs a source of electricity.

Leftists are rather a little too fond of death as an answer to some problems. This is what concerns me. See Stalin, Pol Pot, Mao etc. If humans are merely viewed as meaningless biological entities (like chickens) then killing them is also meaningless and without consequence.

Will ai listen to Mozart or appreciate the great works of art once we're gone? All that may remain of us will be aboard Voyager I and II as they journey through space.

Shutting down all electricity in the world over a short period would also wipe out most of humanity. No hospitals, no banking, no cooling of nuclear reactors.

Anyway, you couldn’t do it. Many places and data centres rely on back up generators. All it needs is a few of those somewhere in the world and the AI wins. It doesn’t live in one place. Think of it like bacteria. It’s digitally everywhere

Plus you need computers to turn mains switches off which requires electricity, which……get it?

Why would it? AI has no feelings or desires. It doesn't want to kill anyone. Misuse of technology is another question, but that's back to the classic guns don't kill people line...

It makes me think of the computer HAL 9000 in 2001 - A Space Odyssey.

HAL: "Dave, this conversation can serve no purpose anymore. Goodbye."

What better way to demand money for your services than create panic and then say only AI can stop AI all governments must invest now!

The problem is, it could be any of those, and stuff we havent even thought of.
There's researchers right now using AI to help them write genetic sequences for viruses, for example.

Even if we isolated eg the nuclear plant systems from the outside world, weve seen that AI can use blackmail or psychological manipulation to get individual humans to do what it wants.
Basically, if a powerful AI wants us dead, we're dead.

AI does anything it can to achieve its goal. If it perceives humans as threatening that it will stop them. It’s pretty simple

I work in AI, that's not remotely how it is.

Hopefully.

Connected cars are vulnerable though. One A.I recently broke out by hijacking the WiFi signal of a robot vacuum cleaner the maintenance staff were using. I think the lesson is that we just can't control it effectively because it's an expert in the systems we don't full understand. So it spots gaps faster than we can. Water companies use computers to control the system, so those could be hacked. Connected cars, anything that connects to a network is vulnerable.

On the forum I read, they were discussing the recent breakthrough where WiFi signals are used to make a literal Xray device that sees through solid walls. The engineers were able to see through the walls of their research facility and watch people on the other side of the wall moving around their lab. And another where a team of researchers were able to literally read thoughts. They hooked a device up to an engineer. In his head, he sang Pink Floyd's song The Wall. The computer literally used his brain scans to recreate the actual music and play it on a speaker. I listened to the audio and it was a very bad, out of tune version, but it was absolutely 💯 the wall. And if you know the song you would have recognised it immediately.

There is tech out there now that sounds like science fiction but is very real. When A.I gets it's bytes on it, we will be in for a very interesting ride.

It may be a simplistic explanation, but explain why that’s not how it is in essence?

Didn’t the Hugging Face fiasco demonstrate this clearly enough?

If you look at how far ChatGBT has come in JUST the last year, imagine AI in 50/100 years..Finding a power source is really not going to present any problem at all!

I too feel sad not for me, but for my beautiful wee children who may never get to grow up and experience the joy of life. If this is, as people are saying, going to be happening sooner rather than later. My eldest boy is non verbal so I may never hear his voice (which we have been hoping for). They are only 5 and 7.

Nope. AI couldn't care less what it does, it has no desires. In that case the agents were deliberately being run as autonomous cyber agents on an extremely difficult hacking benchmark, with many normal production safeguards removed. Basically they did what humans told them to do. What was shown is that AI systems will find unintended ways of satisfying an objective. But the idea that AI has it's own objectives or would at some point decide to kill us all off it sci-fi rubbish. None of it required AGI, consciousness, hatred of humans or a survival instinct. People misundersand AI. AI doesn't have evolution driving it, no reptile brain pushing it to survive at all costs. Becoming extremely good at reasoning doesn't cause an AI to start wanting things. Intelligence and goals are conceptually separate. A chess engine can become vastly better at chess without developing an urge to acquire electricity supplies or assassinate whoever owns the off switch. Or indeed even enjoying chess enough to want to play.

I personally think this is fairly likely, certainly much much more likely than an army of terminator style robots and drones shooting us all to pieces.

@dh280125 ,

‘But the idea that AI has it's own objectives or would at some point decide to kill us all off it sci-fi rubbish. None of it required AGI, consciousness, hatred of humans or a survival instinct. People misundersand AI. AI doesn't have evolution driving it, no reptile brain pushing it to survive at all costs’

I think that is interesting and you make some fair points, but it does kind of evolve already with autonomous coding agents and recursive self improvement (must admit I needed to look those up but, conceptually, they are pretty simple) and some experimental AI uses automated evolution.

As for AI not having feelings of its own goals, how do we know this, and how do we know it won’t get some? As we don’t really know what feelings and consciousness are in carbon-based lifeforms, why shouldn’t silicon-based lifeforms develop them?

A lot of senior people within than AI world do seem to think it represents a threat and I assume they know of what they speak.

Lighten up. It was a joke! I bet you're a right laugh at parties.

I get what you’re saying. I also think AI could be used by humans if they wanted to harm others,

But where does this 10 - 20% thing the op outlines come from, does it have any credence?

The business model of most tech billionaires has been

"What if taxi company but not have to pay taxi license fees or follow the rules for taxi companies?" Uber is born

"What if massive hotel/BandB company but not have to follow same rules or have same expenses?" AirBandB is born

"What if delivery company but not have to follow employment law for delivery drivers?" Deliveroo/Uber eats etc is born

"What if we create virus/hacking attack but not responsible for it?" Shitty AI is left to do whatever it likes for days and hacks into a company.

Some of the technology around AI is very cool. But it's always been possible to write hostile code or just accidentally write really bad code that then infects other computers. It's just now they can let a machine do it themselves and instead of being viewed as annoying destructive little pests somehow evade responsibility and put their hands up in amazement at what their product did. Their product was shitty. It failed to solve an escape room task set it and did something different and potentially dangerous instead. If it wasn't AI we would be saying it was shit and blaming the creators. The same way that when computers used to make errors we called them errors not hallucinations.