Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

@PistachioTiramisu ,

Plague is bacterial, not viral. It responds to antibiotics.

https://youtube.com/shorts/1x21YLQru5g?is=lDiuIWvzbE_fxXy7

I heard someone say it’s like chimps trying to imagine all the possible ways humans could be a threat to them.

BBC News - Anthropic blocks 'malicious use' of AI that could develop biological weapons
https://www.bbc.co.uk/news/articles/cx2zrrpkx20o

It seems it's humans using AI to develop weapons rather than AI out to get us (at the moment...).

Like a PP so much of this seems to be due to the personality of the tech bros leading the AI companies.

Yes, we are our own destruction. Humans are really stupid. They can create this kind of monster but they're not able to find cancer cure. How strange.

Persephonia1966

I really like (if that can be the context for such a worrying subject) your posts on this - I finally understand the paperclip scenario - even though I saw that small video once on YouTube about the AI that was tasked to make greetings cards and managed to get onto the internet and made everything - all planetary resources including humans - into materials for greetings cards - same thing - duh. Your reasoning is very clear and easy to read - thank you.

I myself see things in a more emotional way - I read now that AI is being used to create viruses ..

'While this technology has not been used to generate human pathogens, the study confirms AI can design viable, non–natural viruses.
'This underlying capability means the technology could potentially be used to create dangerous pathogens with potential to be a bioweapon, though safeguards within the AI model, which are in place in this study, can help mitigate the risk.'

OK – so imagine that AI is tasked with creating viruses with a view to attacking viruses …
Could humanity be seen as a virus? Here are some reasons I think it absolutely could be ….

• We infest the earth making it sick with pollution. We are killing our host.
• We kill other organisms and even ourselves to survive.
• We ‘mutate’ through learning and use of the earth’s resources whilst weakening our host.
• Without mankind the earth would recover – during lockdown the skies were clearer and fish were seen in the rivers of Venice for the first time in many years.

I know some people are saying to deprive it of electricity or just 'turn if off' - erm ... then we would need to live without electricity???

I am very worried - I was very worried about a year ago and read a lot of books on the subject - but this week it seems to be ramping up.

Unfortunately - we are in the hands of superpowers in a race for dominance over this potential asset/weapon - in my mind now a probable race to the bottom.

On the regulation issue I don’t see that the U.K. and EU imposing more of this when US and China steam ahead. And if the US were to hold back then it’s China out there with increasing world power.

I mean we’d have nothing in response if China alone were left to just run.

Oh yes thank you. That’s exactly what I am talking about.

It’s just doing its completely unrelated mission and boom. Everything fucked. It doesn’t even care.

Also malicious actors in other sectors, not beholden to major governments, getting their hands on the relevant AI. It is probably only a matter of time.

No. You can't turn it off. Think of it like a plant. As long as there is a seed in the ground it can grow back. If you look at the Anthropic break out, you can see how quickly these can spread. Anthropic could cut everyrhing off, but if there is any code left, anywhere, as soon as there is power the seed will germinate. It would be like trying to stop grass from growing. Unless we literally shut down every computer, everywhere. A.I can't be turned off.

As for mutual destruction, it doesn't quite work that way. The biggest players wont be in control. That's the point, that's why people are speaking out. The A.I is faster than we are. The danger is that it will decide on a course of action, independently, it will then execute those protocols before we have even realised what it had planned. Or worse, a state like NK decides to release an attack on the west..

Thanks

It's a hard concept to get ones head around isn't it

Very very very worrying.

Is it AI acting alone that’s more of a threat or a person / state using it against others?

Both.

Have you not seen all the Sci Fi films? Only Will Smith can save us.

On the BBC news today it said that people are saying the Prime Minister should have a Kill Switch to turn off AI in the UK. That sounds like fantasy, doesn't it?

Given what we've seen of his anger management strategies in recent times... in that case, we're doomed!

It all sounds like fantasy to me but unfortunately it seems like this is for real. I hope some clever team of people is working out how to create a kill switch - somehow!

Can we just give it a leaflet and tell it not to kill humans?

Most of you are misunderstanding the US/ China issue. US is widely seen to be 3-6months ahead of China and it’s believed that if the US slows development that would slow Chinese development too - because they are mostly following the leader.

Also, there are bilateral China / US talks in September to discuss working together and not safety.

so it’s not ‘the US needs to create it so China doesn’t.’ Not at all.

I thought China claimed to be ahead of the US ? I mean specifically for low cost chip LLM training.

The idea being that China still managed to do it's first LLM model, deep seek, while a US ban on exporting NVIDEA chips were in place. And now China say they can almost match US tech, but at much less cost. Or something like that.

There was a lot of stuff said at the time about smuggled chips.

And of course, China now claim they can make the chips.

good old UK government eh

https://www.bbc.co.uk/news/articles/c3eq7kl5l00o

How do you figure that? Humans have not killed all humans yet, despite having been around for millennia. Smarter-than-human AI is not fully here yet, so has much less of a track record of not killing everyone.

It certainly is a good job that AI could never secretly replicate itself onto other machines, and humans will always know exactly which computer AI is running on. It is a good job that AI could never blackmail, threaten, or bribe people who might turn it off.

I think it is fine to believe that AI risk is over hyped. But smart people sincerely believe in it. They have not missed an argument quite as simple as "computers use electricity".

Bee populations are now increasing. We have overcome colony collapse syndrome. There is no meaningful way that we only have 50 harvests left.

It is really bad to spout stupid stories about doom, because it means when someone has a sensible concern, it sounds like one of the stupid ones.

That's like having a "kill switch" for computer viruses. It's not how they work. What I would like someone to do is introduce actual penalties for companies that break the law, I don't think it should be possible to hide behind "AI". Eg if you deliberately leave an AI agent completely unsupervised for days with instructions that basically tell it to keep doing stuff and it hacks another company (Hugging Face) you don't get to pull a surprised Pikachu face and say it's sooo scary and unpredictable it did that. Even old school viruses if released into the wild could spread further than their creators expected. That doesn't mean they dont have responsibility for creating the virus.
It turns out Elon Musks AI and possibly other AI models had the worst categories of CSE in the data setz they trained it on. That's not the AI randomly generating porn. That's real images of real children being abused that were deliberately included. It's illegal to have those images on your computer (unless you are the actual police and even then it's controlled). Saying it was there for "research" isnt defense. There isn't any justification for AI companies to hold those images in the first place. It's still distributing images of CSE. Less seriously but still not OK you shouldn't get to break copywriter.law "because AI".

The real problem isn't AI becoming "sentient". That's neither here nor there in terms of the harm it can do and it feels like a deliberate distraction from the actual dodgy stuff the AI companies know they are doing.