Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

https://apple.news/ACzBLMZs9QaOE2CigCdE5ww

Then they absolutely need to be dealt with, though I doubt they are storing it so I expect they get around the law. Honestly, using it to train their LLMs should be cause enough for the companies and CEOs to be dealt with.

My main fears around legislation is that some countries, such as the US, have ruled it legal to have AI CSE material. Luckily I think it is rightfully illegal in the UK!

I agree that it's not right though disagree with your point about legislation. It doesn't need to catch up. It is already illegal to have CSE on your own computer. It is not legal for companies to store CSE on their computers. It is not legal to hold massive amounts of CSE on hard drives. There is no, as far as I am aware "training my LLM" loophole for this. If there is it can be closed. But if they are/have been storing CSE then they are already breaking the law as it stands basically. Of course it's worse the product produced by processing the CSE can do more harm and maybe we need more laws to deal with that side. But just the processing massive amounts of CSE is already very illegal.

Absolutely, and there is the fear it might become more available. Like the start of the internet where teenagers sent round shock gore website things before it was more regulated. I remember a 'friend' sending me a video that I turned off as soon as I realised what it was, but it still sticks in my head 30 odd years later. I really worry about my children and what they will be exposed to.

Legislation related to it will catch up, but it's far too slow. You can't just download children's suffering to train your software and take zero accountability for it, it's just not right.

There needs to be way more attention drawn to this aspect.

The thing around AI is that it can't create something from nothing. So if you ask it to make an image of Will Smith eating spaghetti it will use it's vast database of images that are most likely to fit the words Will Smith and images that are most likely to fit the words eating spaghetti and produce the most probable combination of the two. So we get an image of Will Smith eating spaghetti. Likewise if it's asked to make an image of a celebrity naked it can combine images of the celebrity with pictures of naked adult men/women. And if it is asked to make a realistic image of a small child....

The only way an AI is able to make CSE type material is if it has been extensively trained on CSE basically. So even though the "images" are claimed not to be real they are based on the real suffering of a real child. Actually many many children. So there are two sets of victims - children who have had sexually explicit images generated of them from their normal photographs, and abuse victims whose images were used to generate those images.

AI companies throwing up their hands and talking about how users are "evading the safeguards they built in" to do this are way more culpable than they claim. And there's a place for people like that. It's called Prison.

@Ladyoftheapple ,

That, in a microcosm, is the UK’s problem.

We expect (rather than working for) a leadership role on the world stage whilst not bothering to build alliances or develop at home.

I actually think Burnham has some good ideas if he can resist the lure of being a world statesman. We need to be at the forefront of technology at home first, rejoin the EU and work to form meaningful alliances which could demand a global voice.

https://www.dailymail.com/news/article-16153463/Andy-Burnham-speech-chamber-New-York-world-unite-AI-risks.html#

Everything around CSE with AI disgusts me and terrifies me. Am I right in thinking it was recently ruled in the US that AI CSE is legal? Could be wrong on that, I don't want to Google it because it makes me want to physically vomit.

AI can be regulated. We did it with Nukes, and on a country level we have prisons and fines. These are headed by companies. Make the CEOs accountable for the crimes their product facilitates.

The issue is legislation takes time to catch up and this is moving too quickly.

@Persephonia1966 ,

I think that internal regulation is totally possible. And everything you suggested sounds good.

I am more sceptical of international regulation. How do we know what is going on in a secret lab in China? How could we make sure that they abided by whatever was agreed to (and vice versa).

Honestly I’m a bit suspicious of all these “look how powerful and scary AI is” stories, I wonder how much of it is planted, bigging up the AI companies to make them sound more advanced and sophisticated than they are to build on their brand and appeal. Not saying we don’t need regulation, of course we do, but I’m taking in these stories in a measured way.

I’m more concerned about the environmental and social factors tbh, and these stories conveniently detract from those.

A start would be that when someone uses AI to hack into another company etc that person is held legally responsible. Rather than being allowed to throw up their hands in suprise and talk about how the AI agent "shocked" them all through it's actions.
I know that then some people will ask how the owners of those AI agents will "control" them but if you read in detail the events surrounding the "AI breakouts" it was very easy to avoid and the companies were extremely negligant in allowing them to happen.
Another law that already exists that could actually be used would be the laws against creating and sharing images of child sexual exploitation. It is now being reported that at least some AI companies (Musk cough) used massive amounts of images of CSE including the worst types in the data sets they used to train their AI. Not only was there no need for that but it is also very much illegal to have that material (you can't just argue it was for "research" not gratification it's still illegal). But for some reason it isn't being followed up on.

Of course, it is extremely unlikely that the Trump government will pursue the American based owners for this. But there are very obvious measures thate could be taken if people wanted to. The whole "AI is so powerful and independent" thing seems partly calculated to discourage regulation/accountability for a group of very rich human men.

I think AI regulation is a bit of a non-starter. I guess they have to try and talk about it, because the alternative isn’t worth thinking about.

How will the regulation be enforced? Is there going to be a UN body which can enter everyone’s AI labs and see what is happening.

And how will they recruit people to do this given what AI professionals are paid in the private sector?

I'm hoping it's bluster before his talks with China on Thursday. It is in both of their interests to control AI. It's only a matter of time before someone uses it to disrupt either country in a big way. They've given every dissenter a team of specialists, essentially, outside of all the other risks to the human race it is a massive national security risk for governments.

Problem is he has done significant amount of insider trading with his sons in AI so greed is fueling him. But their money means nothing if people have the hacking power to breach banks. People have already hacked state institutions with it now, it's just they stopped short of doing harm.

On top of that 22 UN nations have called for regulation so there is pressure there. I just worry it will take a catastrophe for the world to do something, like with Nukes. We have a chance to step in before it happens here.

We can only hope the midterms are heavily against him. Sadly he now has his own state propaganda TV so the MAGA crowd won't source news elsewhere. Fortunately lots of his voter base are near data centres so their water is being disrupted, their energy bills are sky high, and now fuel is astronomical. Hopefully that will help them see sense.

It's scary, only thing we can do is hope the world and the AI companies don't let a greedy corrupt president destroy everything.

Not looking promising 😳

https://apple.news/Art1x0PtpTzCOAA2O-VVFKA

I'm not an expert in AI! I don't want to present myself as such. I do know about computers though and probability maths which is the backbone of the LLMs really. And knowing about AI is important to my job but my job is not developing AI if that makes sense. So I don't want to pretend that I know everything about AI but I think I know enough to know what it isn't if that makes sense?

'a model of a thing is not a thing'
exactly @Persephonia1966 👏

I think you might be the expert we've been waiting for (crossed posts earlier)

@Persephonia1966 ,

Thank you for taking the time to reply to me at length. I do appreciate your obviously expert perspective.

However, as you might expect, although it gives me more pause for thought, I still disagree with some of it.

‘By AI do you mean LLMs?’

I mean any and all AI. It only takes one to evolve consciousness for AI to have evolved consciousness. I am far less expert on the detail than you here, but I am not sure that affects the generality.

‘I also think the definition of "human level intelligence" is being written to fit what is measurable and what computers can do. It's the wrong way round.’

In a sense, until we know what consciousness is (as you alluded to), it’s all we have got. The fact that the Turing test seems not to have worked is maybe just a problem with that test, and not the methodology. I guess we could ask the question the other way around. How do we know we’re not some kind of chemical AI? We couldn’t prove it beyond doubt.

i also think that is where having some kind of physical body comes into play. We have developed a whole endocrine system to avoid destructive behaviours. So, when we ‘feel’ frightened of dying, it is our way of avoiding a behaviour which stops us propagating our genes. In the same way, some AI has shown to be averse to its own destruction. I know there are more prosaic explanations of why, but you could see it as very early form of this.

‘That isn't really the main problem. See the points above. Humanoid robots are cool and all and show great engineering skill. But a model of a thing is not a thing.’

I get your point and I know a human-seeming robot isn’t a human being! I was more thinking in terms of evolution (recursive learning). Our evolution was very much based on our prehensile thumb and what we could do with it. And I am thinking that movement and sense is very useful to acquire thinking, and certainly feeling. However, maybe AI doesn’t need that stage.

’if you are flexible.inyour interpretation of MRSGREN the East India Company was also alive’

I think that is a bit fatuous! I guess it fails on growth, but I can see the others without too many mental gymnastics.

I guess I am ultimately in the camp of finding the idea of AGI more interesting than some terrorist getting hold of AI and making a bio weapon or an atomic bomb, although of course those are possible.

post fail, sorry!

There might be something crucial going on in what evolution has produced in us (and other animals) over millions of years that can't be replicated without re-running the whole thing under similar conditions. (I wish more AI bods were biologists.) 

That's why I liken current AI to chemical reactions - no matter how complex and whizzy they get, we wouldn't expect a mind to emerge from those. But I'd be intrigued to hear a convincing counter argument. I love this stuff! (And, somewhat conversely, I haven't ruled out panpsychism somewhere in the universe - it's a big place  😉 )

I'm not an expert - I'm a biologist (and author), and did a 3rd of my degree on philosophy of the life sciences, which gave me an interest in this topic. Hoping a real expert comes along to chat about this soon. It's so fascinating! Before I did that degree I was of your mind, @Newbutoldfather , especially as someone mad on Asimov in my teens. But John Searle's Chinese Room argument really convinced me - until such time as the science of consciousness and AI theory advances, anyway. I know it's been refuted by some, but I don't buy the refutations. (He also had a tin-can thought experiment which I'll try to dig out.)

There might be something crucial going on in what evolution has produced in us (and other animals) over millions of years that can't be replicated without re-running the whole thing under similar conditions. (I wish more AI bods were biologists.) 

That's why I liken current AI to chemical reactions - no matter how complex and whizzy they get, we wouldn't expect a mind to emerge from those. But I'd be intrigued to hear a convincing counter argument. I love this stuff! (And, somewhat conversely, I haven't ruled out panpsychism existing somewhere in the universe - it's a big place  😉)

Agree

The annoying thing is that the people owning the companies have become completely obsessed with "winning" the AGI race. And they are very convinced that AGI is potentially dangerous, destructive, like the terminator. Basically it's based on science fiction. LLMs have uses but part of why they are so favoured is, I think, because they are good at the Turing test. I don't think the Turing test means anything when you are deliberately designing something to pass it and Turing himself suggested this. So now we have very sophisticated LLMs that are good at "roleplaying" what the creators want. Every single "super scary" jailbreak/hacking event has been caused by AIs trying to do what they "think" they are supposed to do based on prediction.

The danger here is that if the tech bro types are so convinced that AGIs will destroy the world and the AI is "trying" to meet those expectations then it could end up saying or even doing destructive things because that's what it's prediction software is telling it the "character" whose role it is playing would do.

So we could end up with a computer acting aggressively "for reasons of plot". Which would be super annoying. The way you fix that is by making CEOs liable for the stuff their machines do in the same way a farmer is responsible for his out of control combine harvester. Rather than letting them effectively train their AIs to be nuisances because it makes them feel important.

Oh BTW by weird poorly thought through philosophical musings I didn't mean people on here. It's more the tech bros themselves that are the worst for it. But I do think it's a distraction

Since you complained noone had refuted your points:

1.But AI is not a toaster! Its main purpose is problem solving in an abstract sense
By AI do you mean LLMs? Because AI is a very general term and now covers plenty of things that have been around a while as well as new ideas/development. LLMs are a subset of that and their main purpose is more like roleplay and word play through an incredibly sophisticated probability matrix applied either to words or images.

In that sense, it is much like us debatable. We don't fully agree on what defines consciousness and what makes us human. This is also what makes the "will AI develop sentience" argument so annoying because it distracts everyone with weird poorly thought through philosophical musings. Love philosopjical misings. But it's irrelevant to the "will AI kill us all". Language/communication are a very big part of that, but I would argue it's not the only thing that matters. I also think the definition of "human level intelligence" is being written to fit what is measurable and what computers can do. It's the wrong way round.

It has been taught to play games, design proteins, solve abstract maths problems etc. Yes it is good at solving "language" type puzzles because that is what it is built for. In the same way a toaster is built to make toast. Abstract mathematical problems are effectively just language puzzles at that level. Likewise coding is a language so it can write code. It can also translate between human speech and code which is why it's used for vibe coding. This is genuinely impressive. Its not solving these puzzles in the way humans do and doesn't see language in the way we do but it is so sophisticated someone who doesn't know about Markov chains might think it is. Hence why it doesn't do well at "how many rs in strawberry" questions. They corrected that mistake but then similar mistakes keep getting identified because it can't "learn" what it did wrong and generalise in the way it can with other mistakes (the whole point of machine learning).

Of course, to become like us, it would probably need to sense like us That isn't really the main problem. See the points above. Humanoid robots are cool and all and show great engineering skill. But a model of a thing is not a thing.

Apparently it is impossible to separate the ‘thinking’ part of the human brain from the ‘’moving’ part as they have co-evolved. there we go!

To me, it doesn’t take a huge leap of the imagination to see AI embedded jn robots (and the military will really want this) evolving into conscious thinking beings. Yet but it does require a leap of imagination of some size! If your imagination is highly athletic (a good thing) it is likely not a big leap for it. I do think AI embedded in war machines is concerning. But not because of that.

If you look at good old MrsGren, recursive learning AI embedded in a state of the art robot would qualify as living if you are flexible in your interpretation if you are flexible.in your interpretation of MRSGREN the East India Company was also alive. As was the Roman Empire. Momentum isn't the same thing as life
Plus, If we are as flexible in our interpretation of consciousness then the universe is also sentient. And of course colonies of ants or trees could be both living and sentient. But colonies of ants and trees don't recognise our consciousness in a way that satisfies our ego which in my opinion is a big part of the interest in "AGI" and the various definitions of what it would look like.

There are a lot if people in the business who keep saying AGI is unlikely.

I have posted several times saying that I disagree and explaining why.

No one seems to want to refute the specific points that I have made. I clearly don’t know and am not an expert (as I have said), but I think that my general arguments are sound. I would love a genuine expert to explain to me why I am wrong.

(as well as several high up within AI don’t seem to think it unlikely at all, although people have reasonably questioned their motivations).

I think agi is less likely than:

• llm agents escape their boundaries and fuck up something important (I use a lot at work and they are forever ignoring instructions and then saying ‘oops’ when you point it out)
• Companies laying off 1000s of knowledge workers to appease the shareholders and this then destroys their businesses (no one works so no one can afford your product) and the public sector (no higher rate tax payers paying in)
• vastly over valued ai company stock tanks and the trillions of dollars spent go poof overnight which hastens
• general societal collapse (hungry, poor people, no trust in anything anyone says because of ai generated fake news everywhere, no future for the vast majority)

An autonomous ai wiping us out as it is the best thing for the planet is actually a better outcome than us slowly killing ourselves off through our own actions tbh.