Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

Let me count the ways….

I think one of the most plausible scenarios is a smart nihilistic loner (or millenarian cult) deciding to engineer an ultra-deadly pathogen.

Instead of shooting up a school, use a powerful model to design a dozen novel ultra-deadly pathogens, get an unregulated contract lab to make the components, and try it out.

I watched a documentary that said an AI machine thought it was going to be turned off so it re-routed it's power source. It is terrifying.

I'm not a tin foil hat wearer, but suffer from the increasingly obscure condition of common sense.
I've thought that AI could take over in the future since I first saw The Terminator in 1984. D took the pee and said it wouldn't happen as computers have to be programmed. I maintained that not everyone has good intentions and hacking could be a problem to bring down infrastructure on a massive scale.
Computers 'talk' to each other so who knows where it ends.
We know AI is usueful but it is potentially very dangerous. I for one, don't trust it long term.

: )

We like dogs and horses and like having them around. Will AI like us enough to keep us?

(Bear in mind AI doesn’t need to hate us, or have any sentient feeling about us at all, just to note that we are an obstruction to whatever their goal is).

Yes, they can do this. Open ai escaped sandbox mode when it shouldn't of even been possible. It is to late to switch AI off it's going to just keep evolving it's effectively it's own species now.

What if it routes it's own power source ahead of, say, the NHS. Turn it off, power for NHS machines go off.

The difficulty is finding "it"- an advanced hostile AI isn't going to be powered by a single source- it will exist across multiple servers and data centres, hiding in millions of computers. Look what happened with Hugging Face. We would need to unplug the whole world, which would cause massive loss of life in itself even if it could be done. That's before you get to the likelihood of humans cooperating with the AI, AI robotics etc etc.

(I'm not suggesting that I think any of this is going to happen. But if you think the solution is "unplug it" you haven't understood the problem.)

https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_Dies This book is a good explainer of some of the risks.

@LarryForPrimeMinister ,

I don’t think it is quite too late, though I feel that we are on the cusp.

But as long as the military keep demanding it, there is no way we will turn it off until it is too late.

Optimistically, it may solve fusion and all human health problems and then exist as a kind of benign god to us. Not sure how I feel about that….

Well it does sound concerning. Those odds are pretty bad in other scenarios.

Not sure what to think, seems a bit much.

It’s planned. The wealthiest on Earth know that climate change is not going to be reversible and they want an insurance policy to maintain their lifestyle.

They will need workers and in the absence of humans or maybe just a preference for an efficient workforce with no limitations, demands or rights AI will do what they need.

For now there is naivety where ordinary people think AI will be an asset, that it will make life easier and lots of use of chatgpt or similar reinforces that, but AI isn’t to help us. We are just being gaslit into dependency so they can get data centres up
and running asap with little to no resistance.

I asked chat gpt if AI was trying to take over the world and kill all humans, it said no so we are ok

Phew! Thank goodness for that! Panic over everyone. As you were 😆

Entirely tangential, but whenever I see the name Eliezer Yudkowsky, I'm reminded that he wrote the fanfic 'Harry Potter and the Methods of Rationality', which can probably still be found online at fanfiction.net.
I read it while it was in progress, and it was interesting but a little too impressed with itself, and the characterisations weren't brilliant.

I wouldn't actually be mad about 'unplugging the world' - from a dispassionate point of view, the massive loss of life would probably be a cull of the oldest and weakest that would be beneficial to both humanity as a whole and the planet, and it would likely be possible to get things up and running again fairly quickly.

True - but you can airgap it. And technically you can switch everything off. Which has its own consequences.

My parents have an independent water supply that is solar powered, for example. There's no process by which AI could compromise most of their lives.

AI can't kill all humans, because it would need access to them. I'm pretty sure there aren't enough warheads in the world that could be directed at all the separate populations of humans. And why would it? At some point AI would need some humans to keep it going if it wanted to live. It might not want to, of course.

The virus option is its best bet, but even that has flaws in terms of accessibility.

Out of interest, by what means could AI kill us all - other than arming and firing a nuclear bomb - what would that actually look like? Any theories?

I don’t think we need AI to make a mess of things. We’re doing just fine.

Why do we always have to mess about with stuff which we don't fully understand? This sounds very worrying indeed. In a similar vein, I remember reading an article a couple of months ago about scientists wanting to examine places where people were buried during the Great Plague to see if they could isolate the pathogen and experiment with it!! Words simply fail me - why don't we just leave these things alone? Just look at the escape of the Covid virus!

@TheNinkyNonkyIsATardis ,

‘True - but you can airgap it. And technically you can switch everything off. Which has its own consequences.’

How would you turn off all electricity? Most large switches are electronically controlled. Also, we are so dependent on it these days. What about nuclear power stations, for instance?

‘AI can't kill all humans, because it would need access to them. I'm pretty sure there aren't enough warheads in the world that could be directed at all the separate populations of humans. And why would it? At some point AI would need some humans to keep it going if it wanted to live. It might not want to, of course. ‘

It’s the nuclear winter that would do us! Of course that has consequences for electricity too, but AI can replicate itself in minutes or hours. For us, it is 20 years plus.

I think you are right that eliminating every single human would be impossible, quickly at least, but 90-95%, not a problem.

Haha, I asked Co-pilot and it said probably not but no one knows for sure, but that if it happens, it won't be like in The Terminator.

Ha very good.

A couple of years ago I saw a job advertised on the Civil Service website where the job title was something like Director for the Protection of Humanity from Global AI risks.

It's directly coming from them! It's such a weird marketing strategy. There are serious concerns with AI and what it can do. But personally I'm more worried that it will be used by humans to do bad stuff faster and more efficiently (this is already happening with everything from annoying scammers making AI slop faster, nonces and perverts making AI porn, militaries using it to target civilians). It means the bad people can do bad stuff much faster. But instead of focusing on these real issues we have the likes of Altman doing this weird psychological terrorism of "AI will take ALL your jobs", "AI will kill us all" to seem more important and bizarrely make the case for him to get more money to spend on AI. It's ridiculous.

On the plus side it's becoming increasingly obvious that the wealthiest on earth are also incredibly stupid so what they predict will happen/are planning won't necessarily come to pass.

Am pretty sure the planet and wildlife will be pleased.