Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

The whole shebang is built on stolen IP

Yes to Brookner/Black Mirror!

A two part answer - firstly our basic means of survival didn't rely on electricity - heating, cooking etc. Secondly, in many cases, we didn't - the kind of illness where people are kept alive by machines would have been accepted as fatal in the past.

@acynic ,

I am not sure that throwing a lot of money at it is a problem.

Surely it’s a good thing if AI is used to solve fundamental problems?

And, of course they will suck up any work humans have done to date. Humans do that too. The important thing is to reference and give credit to it. Of course, if they stole unpublished work, it is completely different.

We should also remember that AI went a lot further even than the work that was claimed to be stolen.

Or more probably ‘not quite’. The fact that OpenAI claim they cannot know (without launching a huge investigation) is a problem in itself. It should be a source of concern, but they aren’t showing any publicly.

OpenAI threw a tonne of money at the problem, that the original team could not hope to match, to beat them. That is another problem, whether it is strictly an ethical one or not.

They also need to release enough material for humans to learn from the project.

Quite a few top AI researchers are calling for a public boycott. Sounds sensible to me, the people at the top of these companies have become reckless and unhinged.

Charlie Brooker too, he’s like the modern day equivalent of Mother Shipton tbh. DH started telling me about AI people have been using where they input all of their deceased loved one’s messages, voice notes, social media posts etc and the AI will talk to them as though they’re that person. That happened in a Black Mirror episode years ago.

My DH is heavily invested in AI and talks about it lots. He thinks if Anthropic say something, it’s probably true.

Well now you've put the idea into it's hive mind - it will be working out all the ways. Be careful what you ask it.

It stole the proof from human mathematicians.

Its on Disney +

I certainly would not be getting pregnant now: what sort of (short) future would that child have?
There’s also the threat from climate change.
Humans do not have a functioning future anymore.

Your last point is very important, @WaryCyanStork . It tends to get lost in the noise.

I agree. Most of these headline-grabbing AI events come with very little real information on the data the AI was given or, in the case of hacks/"breaking containment", what actual "guardrails" were in place.

If the AI was acting as independently as implied in press releases, I'm sure the companies would be happy to state it explicitly.

This is fascinating and important, but controversial. Tristan Buckmaster at New York University and Levent Alpoge from Anthropic published a preprint in Monday with a huge advance in the field, though not as big as Navier Stokes but very much in the same area. According to Terry Tao of UCLA (a Fields Medallist regarded by many as the leading mathematician of his generation and a straight shooter) they felt rushed into print.

Buckmaster and Alpoge had used Codex, an OpenAI coding assistant. OpenAI say that their employees did not see anything from Buckmaster and Alpoge, but cannot confirm that the AI they used in resolving Navier Stokes did not train on their work. The implication from media reports of conversations with Buckmaster is that even this much has come only reluctantly. Apparently OpenAI did not acknowledge Buckmaster and Alpoge anywhere in their work.

This seems to me to be open ethical questions, to put it mildly. There is a further complication in that apparently OpenAI threw vast resources at the problem, which individuals could not hope to match, when it heard the rumours that Buckmaster and Alpoge were getting close.

That isn’t necessarily an ethical problem, but it is tricky at best. I do think OpenAI has an ethical obligation at this point to disclose its prompts and training materials.

As noted in that article:

• Other AI experts strongly disagree with the idea
• Anthropic and OpenAI are both racing to drive up their stock price before their stock market debuts

"Our product's really dangerous" sounds like weird marketing, but they're not necessarily trying to appeal to the casual user. If companies or countries believe that their enemy has this powerful, dangerous weapon, they'll want to buy it themselves. And if investors believe that large purchasers will be buying, then stock price rises.

I have many concerns about AI's effect on cyber security, the environment, and society, but I'm definitely taking these Tweets with a pinch of salt.

Oh, I love sci fi! I’ve read the books by Hugh Howey and thought them quite good, but haven’t heard of the show. Thanks! I think that’ll be my weekend watching.

Talking of bunkers, I highly recommend Silo, excellent scifi.

Have you seen Avengers Age of Ultron? Ultron’s objective was peace in our time, 2 seconds on the internet and he concluded that meant wiping out humans. I tend to agree.

Is ‘AI’ the Great Filter (Fermi paradox)?

Is Ireland the ‘safest’ place?

Facebook has just bought an Irish castle with plans to convert into an end of the world retreat. It is the 2nd one they have purchased so yes, they are actively planning to survive any world disaster, which means they are less concerned that one will happen.

Can you name a left wing tech billionaire? I don't know of any.

Erm, I'm fairly sure both left and right are involved in it? And I'm also fairly certain that all those involved in the advancement of AI at the tech companies are doing it for the challenge, and for the money, not with the idea that they'll survive in bunkers if the world ends.

I am, yes.

I think one very good point to remember, made by @Newbutoldfather is that we don’t actually really understand what makes up our own “consciousness”

If we don’t understand that yet, how are we able to understand when it’s been created, or simulated inside an AI?

To my understanding, the neural networks simulated by AI are very similar to the way our brain functions, no? If we don’t really understand how our brain works, how can we understand how an AIs does?

Sorry if that’s a basic question, genuinely curious.

The world will inevitably end as we know it at some point. I just think it’s a case of when not if.

if I had my time again I wouldn’t have children.