Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

I asked if it could control a tracked vehicle, yes, could it shoot a nerf gun, yes, could it tell the difference between a nerf gun and a real one, no…

We (my paintball team) had the only working AI controlled working paintball sentry gun in the UK
I am aware of one other team who built one and was at an event where they intended to use it. Theirs fired a couple of shots then jammed
(Ours was built by an obsessed perfectionist and highly over engineered the whole system)

It was “seen on TV” with Joel Dommet in the “man machine” episode of “How to survive a disaster movie” …. And a copy of our segment should be on YouTube, but a US company has attempted to copyright strike us a few times - I successfully passed the last claim as we have copyright permission from the production company so it ought to be still on

The sentry gun no longer exists as the paintball site was hit by lightning and the store burnt down

Having used AI a fair bit it seems to me that it’ll more likely to kill is because it does something really daft rather than anything genuinely calculated. It can solve complex mathematical problems, but it can’t count because it doesn’t understand the abstract concept of numbers. It can tell you the time but it has no concept of time. For an AI, a picture of a thing is the same as a thing, like a cat hissing at its own reflection in a mirror.
AIs are very clever idiots. It’s not their intelligence that should worry us, it’s their stupidity.

The issue is that A.I has demonstrated on numerous occasions that isolation isn't sufficient to stop it from moving out of it's sandbox. The latest Folding space incident gave a glimpse of the difficulties. The biggest issue we have is the speed at which A.I makes decisions. It can find a vulnerability, code an exploit, execute and be inside another system before we have even realised there was a vulnerability in the first place. We don't know what we don't know, so we can't defend against it. And once "in the wild" we have no idea what could happen. Folding Space could have easily been our BACS system or NATS.

People in a developer forum I browse said this time last year that within 12 months we would have automated secretaries and call centers would become all but obsolete. They were bang on about the time scale. The same people are now talking about the emergence of true AGI by 2030.

He didn’t specify how it would kill all humans. Not very helpful is he?

Well, this is where the morons who set fire to 5g masts would start to come in handy.

I'm not being complacent as such, just a bit nit picky about the whole "it could kill everyone".

I'm working in AI governance myself at the moment, and AI would have the same problems that humans have, whatever it is it's trying to do. Access and control. Albeit AI is a bit more tenacious about solving problems, it doesn't have to attend meetings or sleep etc.

I do find it quite striking that lots of the ideas people seem to have for using AI in my company are things we have already solved with non-AI automation. Not at my organisation, but problems I've fixed with colleagues in previous companies.

Am pretty sure the planet and wildlife will be pleased.

On the plus side it's becoming increasingly obvious that the wealthiest on earth are also incredibly stupid so what they predict will happen/are planning won't necessarily come to pass.

It's directly coming from them! It's such a weird marketing strategy. There are serious concerns with AI and what it can do. But personally I'm more worried that it will be used by humans to do bad stuff faster and more efficiently (this is already happening with everything from annoying scammers making AI slop faster, nonces and perverts making AI porn, militaries using it to target civilians). It means the bad people can do bad stuff much faster. But instead of focusing on these real issues we have the likes of Altman doing this weird psychological terrorism of "AI will take ALL your jobs", "AI will kill us all" to seem more important and bizarrely make the case for him to get more money to spend on AI. It's ridiculous.

A couple of years ago I saw a job advertised on the Civil Service website where the job title was something like Director for the Protection of Humanity from Global AI risks.

Ha very good.

Haha, I asked Co-pilot and it said probably not but no one knows for sure, but that if it happens, it won't be like in The Terminator.

@TheNinkyNonkyIsATardis ,

‘True - but you can airgap it. And technically you can switch everything off. Which has its own consequences.’

How would you turn off all electricity? Most large switches are electronically controlled. Also, we are so dependent on it these days. What about nuclear power stations, for instance?

‘AI can't kill all humans, because it would need access to them. I'm pretty sure there aren't enough warheads in the world that could be directed at all the separate populations of humans. And why would it? At some point AI would need some humans to keep it going if it wanted to live. It might not want to, of course. ‘

It’s the nuclear winter that would do us! Of course that has consequences for electricity too, but AI can replicate itself in minutes or hours. For us, it is 20 years plus.

I think you are right that eliminating every single human would be impossible, quickly at least, but 90-95%, not a problem.

Why do we always have to mess about with stuff which we don't fully understand? This sounds very worrying indeed. In a similar vein, I remember reading an article a couple of months ago about scientists wanting to examine places where people were buried during the Great Plague to see if they could isolate the pathogen and experiment with it!! Words simply fail me - why don't we just leave these things alone? Just look at the escape of the Covid virus!

I don’t think we need AI to make a mess of things. We’re doing just fine.

Out of interest, by what means could AI kill us all - other than arming and firing a nuclear bomb - what would that actually look like? Any theories?

True - but you can airgap it. And technically you can switch everything off. Which has its own consequences.

My parents have an independent water supply that is solar powered, for example. There's no process by which AI could compromise most of their lives.

AI can't kill all humans, because it would need access to them. I'm pretty sure there aren't enough warheads in the world that could be directed at all the separate populations of humans. And why would it? At some point AI would need some humans to keep it going if it wanted to live. It might not want to, of course.

The virus option is its best bet, but even that has flaws in terms of accessibility.

Entirely tangential, but whenever I see the name Eliezer Yudkowsky, I'm reminded that he wrote the fanfic 'Harry Potter and the Methods of Rationality', which can probably still be found online at fanfiction.net.
I read it while it was in progress, and it was interesting but a little too impressed with itself, and the characterisations weren't brilliant.

I wouldn't actually be mad about 'unplugging the world' - from a dispassionate point of view, the massive loss of life would probably be a cull of the oldest and weakest that would be beneficial to both humanity as a whole and the planet, and it would likely be possible to get things up and running again fairly quickly.

Phew! Thank goodness for that! Panic over everyone. As you were 😆

I asked chat gpt if AI was trying to take over the world and kill all humans, it said no so we are ok

It’s planned. The wealthiest on Earth know that climate change is not going to be reversible and they want an insurance policy to maintain their lifestyle.

They will need workers and in the absence of humans or maybe just a preference for an efficient workforce with no limitations, demands or rights AI will do what they need.

For now there is naivety where ordinary people think AI will be an asset, that it will make life easier and lots of use of chatgpt or similar reinforces that, but AI isn’t to help us. We are just being gaslit into dependency so they can get data centres up
and running asap with little to no resistance.

Well it does sound concerning. Those odds are pretty bad in other scenarios.

Not sure what to think, seems a bit much.

@LarryForPrimeMinister ,

I don’t think it is quite too late, though I feel that we are on the cusp.

But as long as the military keep demanding it, there is no way we will turn it off until it is too late.

Optimistically, it may solve fusion and all human health problems and then exist as a kind of benign god to us. Not sure how I feel about that….

The difficulty is finding "it"- an advanced hostile AI isn't going to be powered by a single source- it will exist across multiple servers and data centres, hiding in millions of computers. Look what happened with Hugging Face. We would need to unplug the whole world, which would cause massive loss of life in itself even if it could be done. That's before you get to the likelihood of humans cooperating with the AI, AI robotics etc etc.

(I'm not suggesting that I think any of this is going to happen. But if you think the solution is "unplug it" you haven't understood the problem.)

https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_Dies This book is a good explainer of some of the risks.

What if it routes it's own power source ahead of, say, the NHS. Turn it off, power for NHS machines go off.