Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

The NT Times had a recent article quoting their intelligence chief. He essentially said that good AI, and of course rogue AI, would make it harder or even impossible for the CP to keep its lock on power.

I agree with you about humour in dark times, BTW.

@acynic ,

‘The NT Times had a recent article quoting their intelligence chief. He essentially said that good AI, and of course rogue AI, would make it harder or even impossible for the CP to keep its lock on power.’

That is both interesting and ironic. The ruling party that has built the first techno-authoritarian state now has to worry about rebellions by both carbon and silicon-based life forms.

I am sure Orwell would have written brilliantly about it.

Agreed, @Newbutoldfather

Edit: NY Times in my post above

This has freaked the living shit out of me! Until the last week I was apparently absolutely clueless going about my life thinking about mortgages and savings and retirement and even moving house and now I’m thinking what is the actual point! I’m a ball of nerves 😪

@Ladyoftheapple ,

I never get this, sorry! I think that if you are terrified of a small chance of AI destroying us all, you need therapy.

I grew up in the Cold War years. Any day, I might have not woken up because nuclear Armageddon might have happened overnight.

My late mother told me about the 13 day crisis which came very close to nuclear war. Two out of three Russian submarine officers voted to launch a nuclear missile. Very luckily for all of us, the third voted against.

Apparently, there was a lot of partying, drinking and …shagging! We all might die any time so, whilst we should try and avoid it, it shouldn’t stop us enjoying our lives. It is very much a generational thing, being constantly surrounded by news (and fake news).

@Ladyoftheapple I suppose you have to decide whether you will cope better if you just carry on as normal, hoping for the best. But I'd definitely not put yourself through major hard work and stress on, for instance, moving house when you don't need to. I'm carrying on as normal, but at the back of my mind is always the knowledge that I'm now living in a real life science fiction book, and that at some point we will find out what the "overlords" we have created have decided to do with us. I wonder whether we will see society start to break down due to this, and how that will happen.
Our downfall is because we have managed to be both very clever and very stupid at the same time. Remember the theory of why we haven't encountered extra-terrestrials? It's because no form of life in the universe has reached the stage of being able to travel large distances through space. Due to every technologically advanced species destroying itself first.

@Scoffingbiscuits ,

‘Due to every technologically advanced species destroying itself first.’

That is just one theory, albeit a neat one! The other one is that we are unique in the universe as a species with intelligence and consciousness. The probabilities that people use to calculate it are random really and that js perfectly possible.

Also that maybe interstellar travel is impossible. The distances are vast and the speeds required close to light speed. (what’s always puzzled me is, assuming we ever got close, how would we stop hitting things?! Space is pretty empty, but there are a fair few small objects and, at that speed, even hitting a pebble would be pretty catastrophic).

The likelihood of human beings being unique in the universe as a species with intelligence and consciousness must be extremely low. We know that there are vast numbers of planets out there which are capable of supporting life. Ironically, humans have become bored of waiting to be invaded by a superior life form, and have chosen to create one here on Earth.

@Scoffingbiscuits ,

‘The likelihood of human beings being unique in the universe as a species with intelligence and consciousness must be extremely low.’

Why? We don’t really know what consciousness is, so I think it is impossible to put a probability on. I think the probability of life is high, but maybe the way it has evolved on Earth is unique. It is hard to multiply an extremely large unknown number and an extremely small unknown number and get a meaningful result.

‘Ironically, humans have become bored of waiting to be invaded by a superior life form, and have chosen to create one here on Earth.’

I like that! And I tend to agree.

It's horrible isn't it. I'm really affected by it, but most of the people I know aren't. I just don't know how they are not.

I've been struggling with it since the news and have gone on a bit of a doom spiral, reading about all of the horrible things that are happening because of AI development. The only thing that has allowed me to try and have some hope is having control over the only thing I have control over - preparing for potential upheaval in the form of internet going down or blackouts. It's measley but I'm creating a stockpile of food, medicine and water to last a few weeks. Akin to what the government is recommending for el nino.

Maybe just focussing on what you can control can help you? If there is ever a vote, or a chance to be heard, take that chance. Other than that, just buy tinned peaches and think about ways to turn your wheelie bin into a waterbutt... Hopefully the people in power are working away at things to fix this because I cannot believe they don't care about their children.

On the plus side this warning was picked up by every single news broadcaster I've seen so it's hit the mainstream. This means it's on peoples' radars and every bit of new worrying information will add more pressure for the world leaders to take action.

From a quick google: "current hypotheses suggest there could be billions of potentially habitable, Earth-sized planets in the Milky Way galaxy". That's in just one galaxy!

"Recent estimates tell us that there could be as many as two trillion galaxies in the observable Universe." That's just in the observable universe!

"Theoretically, there could be an infinite number of galaxies, as the size of the Universe itself could be infinite."

So I really don't think it's very likely that we humans are unique in the universe as a species with intelligence and consciousness.

I find looking at those numbers comforting in a way. I hope others do too.

@Randomusename Have you asked people why they're not concerned? Are they in denial? It's a massive thing to take on board. And people are actually very adaptable. One moment they're having sleepless nights because their child didn't get into their preferred primary school, and the next they're telling us that everyone being wiped out isn't so bad really - after all, it was fun while it lasted.

@Scoffingbiscuits

It's a mixture, really. Some friends write it off as another doom news story for headlines. Others believe that it's PR. My partner is concerned but says that there is nothing he can do so he may as well not worry.

I'm of the mindset that if you write it off and not try to do something, then for whatever happens you will be less prepared. That mindset does come with a lot of stress though.

What I don’t understand is if AI is going to wipe us out in the next couple of years then why aren’t we all up in arms?! Why are we all going to work and acting like nothing is happening!! This is about the whole human population, our children and our children’s children! Surely rather than writing policies to “slow it down” we should all be working together to try and stop it altogether and taking down these data centres?!!

I felt like that about a week ago @Ladyoftheapple and I was in a pretty heightened state of panic over several days with high heart rate, palpitations, excessive sweating, lack of sleep and thoughts of doom. I couldn’t understand why everyone seemed so normal just going about their day. But the body and mind can’t sustain that level of panic indefinitely. And I realised I can’t do anything at all about any of it apart from refrain from using AI (which I’ve always done anyway), signing some petitions and writing to my MP, all of which I’ve now done; and now I’ve just got to get on with my life really, whether I’ve got 4 months, 4 years or 4 decades left to live. I’m still worried and sad and most of all furious with the tech bros of the world, but those feelings are futile, so I distract myself and carry on and hope that the world’s billionaires start to care more about humanity than $ which seems unlikely, but there’s always hope.

Everyone is normal until it directly affects their lives and it will, but then it’s probably too late. AI is capitalisms wet dream.

It’s exactly the same as we’ve seen with the climate crisis isn’t it @Monicacaca ? It was all fine when the floods and droughts were happening to those people over there in poor countries, but now the shit is beginning to hit the fan here, a few more people are starting to notice/believe in it (still plenty ignoring it though).

I'm hoping that everyone saying it's PR are right, but when the UN are talking about the existential threat it's hard to believe that.

There are a few major world talks on it in the weeks to come. I'm hoping sense prevails. If nuclear weapons has proven anything so far, even the most volatile states have survival instincts. China is the big 'bogeyman' the US wheel out regarding AI, but they currently have the stricted AI regulations and they aren't likely to want to loose control of anything, given who they are. The US public are also turning against Trump, with even some of his big right wing supporters speaking out against AI in the alongside the left like Bernie Sanders.

Sadly I'm very worried it will take a catastrophe for people to really take notice. No idea what that will be, or who it will be against. I can see cyber attacks taking down food and power supplies being likely, or banking systems getting wiped.

For now we will just keep hearing of how these companies lost control again and their AI went on a hacking spree with absolutely no recourse.

Regardless of the existential threat - it's going to cause huge amounts of upheaval regarding job losses. People will take notice then, hopefully.

Geoffrey Hinton has been warning about this for years. They call him the Godfather of AI, because he was involved in creating it originally. And he has a Nobel prize. He knows what he's talking about. He's now saying that there may be one year to do something about it. I have to say, I'm not confident.

Some of the people I actually hate most are the American Democratic party. They had to achieve one thing in their term of office (between the 2 Trump terms) - make sure that Trump wasn't re-elected. But they relaxed and put their feet up and enjoyed themselves, doing all kinds of things which worked against that. We've now all been dragged into the worst case scenario.

If it was humanity being wiped out by a virus or whatever, leaving the rest of life on earth to live in peace, I could accept that relatively readily. But God knows what AI will turn into in the long term.

AI ‘Godfather’ Geoffrey Hinton warns Congress has little time to rein in AI - [İLKHA] Ilke News Agency

The vibe on Facebook/meta/whatever it's called now... seems to be that if tech billionaires are warning about the dangers of AI, it could be because AI "believes" the main problems in the world will be solved by the redistribution of wealth.
Imagine...

We were raised fascinated by the possibility of alien life. Films and books always described those fictional creatures as much more advanced than us. We still haven't found them in space but I guess we ( well, people cleverer than most of us here) created them. Right in our fold. They'll take the measure of us and go "boom, get rid". The end. I hope not but it is starting to trickle in most people's minds that if climate change doesn't get to us first, AI will certainly have a shot.

Ah...Actually, imagine that...

The fear that specifically the owners of the companies have is a very very old fear. It's basically

1. AGI could happen and our LLMs gain true sentience. We want to be the first company that happens to because for some reason it's a race now. They would stop being only machines and become super competent servants we own.
2. What if once they gain sentience our servants don't do what we tell them or talk about us behind our backs? That would be terrible.

I am sceptical about the likelihood of LLMs evolving into "AGI". But it is amusing that there is a while tech bro doomsday cult and it's basically expressing the same fears that Victorian era rich gentlemen used to have at root.

That's an interesting scenario. A request to AI to show how to run the world so there is no hunger or inequality, no homelessness, no violence, no drugs, cures for health conditions. Who would AI kill off first? Our RF could.be in trouble!

Why are you sceptical?