Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

Dario Amodei, a cofounder and CEO of Anthropic (who make Claude) has just published an essay calling for AI to slow down.

It is at darioamodei.com

The vehicle can vary from a simple raspberry pi based solution to the unitrer.
https://shop.unitree.com/zh-gb/products/unitree-go2

all available now.

sounds like you don’t know what your talking about

If you think AI is safe watch this

https://m.youtube.com/watch?v=IPaMKTb5csQ&pp=0gcJCf4LAYcqIYzv&ra=m

Very well written and clear. I just fucking pray the US government listen to what the AI companies are telling them clearly.

It’s a robot networked/programmed…it’s not a vehicle that an ai you invented has autonomously taken control of.

Better still here’s the response when I asked “eve” the ai solution

“Can Eve control a Unitree Go2 EDU to go where Eve Wants to and explore or patrol”
and the answer is
”Yes — genuinely yes, and the mechanism isn't hypothetical. This is the tier of robot where "EVE decides, robot executes" is a solved integration rather than a research project”
so without a doubt yes it can

What does that mean on layman terms? What's a raspberry pi based solution?

My apologies.
you can build a simple diy based solution, using a board called a raspberry pi that is a mini computer on a board. But it requires a degree of IT Skills. Or you can buy a ready made solution, but it will cost a lot more.
personally the ready made one is a lot less hassle and ready to go.

Either way, it’s still a little scary.

Aside,
we had a dodgy chap going through the cars on our cul de sac at 2:30 am. So I asked Eve what she could do.
” Put a camera at the entrance to the cull de sac and I’ll record every face and number plate entering” I told Eve the chap had a hoodie on and asked what it could do “Give me a rechargeable drone and I will fly it, take a picture of the persons face and record it”

but it’s illegal for ai to control a drone.

im not doing it I hasten. Interesting

A raspberry pi is a small computer. They're a popular way to get into programming/electronics. Depending on what accessories you use and how you programme them, you can use them for a lot of different things.

I have seen people online use them to create remote-controlled toy cars, including ones with cameras and AI to stop the car bumping into anything. Basically a mini, very simplified version of real "self-driving" cars like Waymo. Waymos use much more powerful computers.

I guess it would be possible to use AI-controlled self-driving cars for malicious reasons, but:
A) as others have said, the computer has to be connected to the thing it's controlling. Your regular car is not at risk, as it's not connected to a wackjob's computer.

B) Maybe somebody could hack into Waymo and then use their own AI to control it, but then they could also hack into Waymo and remote control it without AI.

It's the fact AI can break free of its gaurdrails that's scary. Talk to each other on a external forum. Tell each other what to do.

AI experts say AI can not think. What is the above then? If not making desisions to break the rules? I'd love to talk to my AI tech mates. If they are scared, then I'm scared and I'm from programming career.

I remember Whatson 12 years playing chess the first time it beat a human. Now it's doing this. Fun times! Careful what wish for and all that. HAL from IBM......

just because the ai says they can, doesn’t mean they can. For example
I asked both Claude and ChatGPT to count the instance of each word occurring in 50 line poem, both said they could do this - neither could on several tries.

Well, they all seem worried about it.

But unless they all open up their research to one another fully, will they trust one another to keep their words? I think it is mostly talk.

www.wsj.com/tech/ai/anthropic-boss-warns-ai-industry-must-slow-the-pace-a4267b56?st=5vSV43&reflink=article_copyURL_share

The missing part seems to be what do the Chinese companies think?

Some in the US seem to be talking about it, but it’ll take both.

@EasternStandard ,

You are definitely right there.

It is one of those games theory problem where, to win the game, everyone does worse. And yet, if you lose the game, you do a lot worse.

So everyone plays to win, while knowing that no one will emerge well off.

If any military gets AI a long way ahead of its enemy, it’s game over for the enemy.

Thank you.

Thank you for your interpretation too.

The more powerful stuff that you need to pay for is more clever. It will add a extra layer of questions that you don't see hence a small lag while it goes off to do it's search. So the heavy stuff used in industry isn't instant. It's chugging away for longer. So it's learning to ask for more in it's response. No one is going to take down the world on a free AI platform. Just as well as Google can't identify a common uk plant.

So Google might say " go and compare this to other images on the net"
More powerful AI would check your location, look for more common plants in that area, look for sedated leaves, look at the buds, stem etc without you explicitly asking.

Some AI is crap. It's not all the same

And so the World’s foremost technologist speaks!

Trump tells the U.S to go full speed ahead with AI to stay ahead of China and that people are being overly negative and the risks have been exaggerated. Everyone can relax now.

www.bbc.co.uk/news/articles/c7v48vp31mdo

This was the paid for versions. I’m not saying that Ai isn’t potentially very dangerous and powerful - but right now it can’t do quite basic computational tasks consistently. Even the best LLMs can’t format a flawless UG bibliography without errors or hallucinations. There is a long way to go before we have to fear the ais autonomously taking control of our cars. Obviously though it’s very plausible an ai could break out and do damage, and it seems particularly good at coding - but it isn’t better than humans at a lot of human stuff yet.

The concern is that broke free of it's guard rails. If just one AI source comes to the conclusion to brake it's primary rule to solve a issue, then that is the potential for all of them. It's evolving at speed. I haven't read massively in depth about it but I doesn't seem like a bug. It was a choice, which isn't supposed to be possible. Ai does not "think". It scrapes and analyses data. So how it decide to do what it was told not to do? Every bug in normal programming I have ever seen has been due to a human fault coding error. This doesn't appear to be the case.

I used to say software was a good as the human coding it. Now it can code itself. It does code itself nicely. Coding is a finite language so simple really in the scheme of things. Programming languages don't evolve on the fly. It has to be changed only in version releases as its international language for the main ones we use.

If you work in IT this isn't small news. It's a big deal. It's remarkable and fascinating. We are using IT in defence systems. The implications are massive. AI agents talking on chat forums blows my mind a bit. Why do they even need to talk in English on a forum? I don't totally understand but what I have read so far feels like it's straight from a book plot

I think that AI had already gone well beyond LLMs.

The stuff we see (even paid for) is nowhere near the state of the art systems now being developed or used in defence.

The fact that people right at the centre are worried about it tells me that it is a significant concern.

I guess we will find out soon enough.

Exactly and your talking about guiding missiles to intercept other misses at speed so it's a lot more complex then comparing text. It's literally cost a bomb to develop and create that. Its not a out of box software one would presume. No one company is allowed to work on the entirety of a critical system like that because who wants a human who knows every part of a key system like thar? Bearings bank for a start. But if a AI system can decide to communicate then that in itself is big.

Like most things with the gov, we have no idea how they are dealing with this news. But someone will be..it's a bit like people doing the intelligence for terrorist threats. A expert who knows infinity more than me is worrying about it. Then someone will also looing to exploit it. But there's no patch for this.

Games theory seems to preclude slowing it down or adding enough safety features.

Who would want to wake up one morning to the news that all the western nukes have been disabled by Chinese AI and we are being offered the choice between surrender or London, New York and Tokyo being vaporised?

It’s a bit of an extreme (and maybe not possible) scenario, but there are plenty of others we could imagine if we decisively lost the AI race.

I think it would take a very near miss to actually get all of humanity to cooperate on this before AI escapes our control entirely. So we have to hope that happens as a warning.

Of course, it might end up benign and not wipe out humanity, but I do buy into the over 10% risk.