Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

Well, this is where the morons who set fire to 5g masts would start to come in handy.

I'm not being complacent as such, just a bit nit picky about the whole "it could kill everyone".

I'm working in AI governance myself at the moment, and AI would have the same problems that humans have, whatever it is it's trying to do. Access and control. Albeit AI is a bit more tenacious about solving problems, it doesn't have to attend meetings or sleep etc.

I do find it quite striking that lots of the ideas people seem to have for using AI in my company are things we have already solved with non-AI automation. Not at my organisation, but problems I've fixed with colleagues in previous companies.

He didn’t specify how it would kill all humans. Not very helpful is he?

The issue is that A.I has demonstrated on numerous occasions that isolation isn't sufficient to stop it from moving out of it's sandbox. The latest Folding space incident gave a glimpse of the difficulties. The biggest issue we have is the speed at which A.I makes decisions. It can find a vulnerability, code an exploit, execute and be inside another system before we have even realised there was a vulnerability in the first place. We don't know what we don't know, so we can't defend against it. And once "in the wild" we have no idea what could happen. Folding Space could have easily been our BACS system or NATS.

People in a developer forum I browse said this time last year that within 12 months we would have automated secretaries and call centers would become all but obsolete. They were bang on about the time scale. The same people are now talking about the emergence of true AGI by 2030.

Having used AI a fair bit it seems to me that it’ll more likely to kill is because it does something really daft rather than anything genuinely calculated. It can solve complex mathematical problems, but it can’t count because it doesn’t understand the abstract concept of numbers. It can tell you the time but it has no concept of time. For an AI, a picture of a thing is the same as a thing, like a cat hissing at its own reflection in a mirror.
AIs are very clever idiots. It’s not their intelligence that should worry us, it’s their stupidity.

I asked if it could control a tracked vehicle, yes, could it shoot a nerf gun, yes, could it tell the difference between a nerf gun and a real one, no…

We (my paintball team) had the only working AI controlled working paintball sentry gun in the UK
I am aware of one other team who built one and was at an event where they intended to use it. Theirs fired a couple of shots then jammed
(Ours was built by an obsessed perfectionist and highly over engineered the whole system)

It was “seen on TV” with Joel Dommet in the “man machine” episode of “How to survive a disaster movie” …. And a copy of our segment should be on YouTube, but a US company has attempted to copyright strike us a few times - I successfully passed the last claim as we have copyright permission from the production company so it ought to be still on

The sentry gun no longer exists as the paintball site was hit by lightning and the store burnt down

It has become more pervasive ... twice this week (I'm looking for a lightweight raincoat), companies that I searched asked for not just my email, but my phone number with a load of teeny, tiny small print including giving permission to "use ai voice to call and message you on voice mail"

WHY?

Not even a person doing "the sell".

With respect, ‘having used AI a fair bit’ is not helpful here. The warnings are not about what you’ve used on Chat GPT or whatever. It’s about next gen which is exponentially different.

The people building these are quitting their high paid jobs and warning us it will kill us.

people need to pay attention rather than post trite stuff like ‘I asked chat gpt and it said it wasn’t trying to kill us haha.’

This is really really serious

I mentioned a risk here & was shouted down. Few see any vulnerability here it seems unless you are feeding ‘it’ your bank details.

https://en.wikipedia.org/wiki/Great_horse_manure_crisis_of_1894

Its capabilities are also growing exponentially. We’re in the ‘dial up’ internet days too!

You mistake my meaning, I think.

I'm not talking about the ability of AI to get into systems that are digital, electronic or otherwise part of the Internet of Things.

I'm saying that AI can't magically hack my fridge, my car or my electric drill, any more than it could magically hack a banana or a skateboard, because there are still things that are manifestly outside of its scope.

Let's say weaponry, infrastructure and a chem lab or two are "in scope". That still leaves raw resources (water and food), physical tools and physical distance as key advantages of (some parts of) humanity.

Not me, but it's also not the case that the Alexa in my house could start throwing ninja stars at me, or contaminate the oat milk my son drinks in the cupboard.

But critical systems absolutely can be airgapped from AI. People don't, is the issue.

Ooh good. I hope it kills the 10% who don't get their bags out of the bagging area fast enough.

@TheNinkyNonkyIsATardis ,

It must be interesting to work in the field.

I did find the details of the Hugging Face attack really interesting. And the way they came out so slowly. Wasn’t it multiple AI agents co-operating by communicating via an obscure chat room, and most of them having been given nothing like that as a brief?!

And solving one of the millennium maths problems shows how far it has evolved from being merely a clever search engine. That is already, in that field, beyond human intelligence.

And it is still evolving very fast.

The thing is we don’t understand what consciousness and free will are really in humans, so I think it is crazy to think AI couldn’t acquire them.

Thanks for this discussion. I saw this article posted on FB and almost every comment underneath was just a stupid joke or meme, rather than anyone actually expressing real concern. For some reason I found the responses almost as disturbing as the article itself. I feel really sad about the thought that we might only have a few years left, either because of AI or because of climate change. Who knows how things will be at the end of this decade in just 3.5 years. Gutted at how we have trashed everything and everyone's futures with our own greed and stupidity.

Correction: we have not destroyed the planet itself, but we are destroying (or may already have destroyed) the version of the planet that was human-friendly... Life is robust though. Some form of life will definitely keep on thriving here - but we might go the way of the dinosaurs, yes.
Also edited to add that by doing so, we are unfairly ripping into oblivion alongside ourselves a whole bunch of innocent and fascinating creatures who don't deserve to go extinct. We're basically the roommate from hell.

They may kill of humanity and the rest of life at the same time.

I'm not massively concerned, partly because I have a nihilistic view of humanity as a species and I think this is inevitable. I think the AI overlords will need to gain controlf of the water supply, once they do that then we're done.

Considering how thirsty AI systems are, an aggressive AI 'overlord' agenda would put that top of list.

It does often feel that we have reached Peak Stupid. And Peak Stupid is exactly when I would expect the end of humanity to happen.

We will all die at some point, why not all go together!

I was ruminating on AI yesterday and the recent news item about it curing cancer.

I used to work in an IT team so have some knowledge of how computers work/'think' and immediately thought that a computers 'idea' of curing cancer would be to kill everyone with cancer. It's the fastest and the most efficient method to 'cure cancer' and computer code is usually written with speed and efficiency in mind.

Anyway, as you were.

It’s my child I feel achingly sorry for. She won’t get to experience all the wonderful aspects of life there are. It’s so sad

Again, this presupposes a single water supply.

I am as nihilistic about humanity as they come, but I still don't fancy AIs chances of a total wipeout.

I also tend to think pretty far in the round when it comes to life on earth. AI (or climate change) would have to eliminate not just all life, it would also have to eliminate the conditions for creating life, which are largely chemical composition of the atmosphere and proximity to the sun.

It’s as if the creators and users of AI have never watched a sci-fi film.

It was never going to end well. Not only on a digital level but you can’t go a day on here without someone say ‘I asked chatGPT’ or seeing an obviously AI written post. People can’t think for themselves anymore, and it’s concerning. To quote the kids - we’re cooked!

Same. This is more about the dc for me.