Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

Agreed, @Randomusename. Like a combine harvester, once it's set off, it's lethal, and if you get in its path, you're harvested - no conscious intention required.

It doesn't need to be conscious to be dangerous. There are already multiple instances of people attempting to get information for bioweapons. If people can have access to all of our scientific or hacking knowledge at their fingertips it will inevitably lead to dangers.

I think the biggest issues are these and job losses without a plan. Thousands of jobs have gone already, many more thousands to come. There is talk they want to replace 75% of jobs within the next 10 years. If there is no plan in place, things will get very bad. Problem is people won't try and get a plan in place until it's someone they know, or themselves. Governments will get fewer taxation, companies will hoard more profits, AI companies will be essentially subscribed to by every company on the planet and will out perform countries on a massive scale.

On top of that, it isn't blue collar jobs that are going to go first. It's the white collar jobs, the work from home usually higher paid ones. The ones you try and steer your children into for a more comfortable life, which is why so many graduates are struggling at the moment.

The obvious answer is some sort of preemptive taxation and UBI plan, but that isn't materialising fast enough because the pace of this technology is exceeding it. You could easily raise corporation tax to account for the people they have chosen to fire for a machine. Or give tax incentives to companies who keep people. Without these measures we are looking at spiraling debt, chaos, riots, recession, negative equity at the minimum.

Everything about this is rushed which means ordinary people, who have no stake or say in any of it, will pay the price.

People need to ask themselves what they will do if they lose their highly paid job tomorrow with no other jobs to go to. Benefits won't cover your costs, but you can't sell your house because no one is buying. Then who is going to fund your benefits, because we simply won't have the taxation income to pay it? So we either heavily tax corporations across the globe, no tax havens, or people just simply waste away and starve en masse.

The knock on effect of that is the corporations lose customers, so more jobs are shed, leading to more poverty.

To be honest, I think complexity will increase, but it won't necessarily lead to consciousness. They remind me more of complex chemical reactions: you set them off, they beget more than you put in... a bit like a slo-mo version of a thermonuclear process.

@Persephonia1966 ,

‘But it would be highly unlikely my toaster would one day be so good at making toast it would also vacuum the floor. That's not a logical evolution at all’

But AI is not a toaster! Its main purpose is problem solving in an abstract sense. In that sense, it is much like us. It has been taught to play games, design proteins, solve abstract maths problems etc.

Of course, to become like us, it would probably need to sense like us, but look at how much robots have improved in the last few years, from hardly being able to stand up to having a faster time in the 100m race than Usain Bolt (sub 9 seconds). Apparently it is impossible to separate the ‘thinking’ part of the human brain from the ‘’moving’ part as they have co-evolved.

To me, it doesn’t take a huge leap of the imagination to see AI embedded jn robots (and the military will really want this) evolving into conscious thinking beings.

Of course we can’t know this will happen, as we don’t know how we achieved consciousness but I think it shows a lack of imagination to not see how AI could become a life form.

If you look at good old MrsGren, recursive learning AI embedded in a state of the art robot would qualify as living if you are flexible in your interpretation. And we evolved from single celled organisms in the blink of an eye geologically. Few would have looked at an amoeba and been frightened of thermonuclear weapons.

Because LLMs are very clever but very specific in what they are and what they do. The idea of "recursive self improvement" in algorithms is a more general one and quite emeshed with the idea of AI. The idea is that this is also happening in the large language models and is the justification for building ever larger data centres and providing as much data as possible. I can accept self improvement is happening although even there there are questions about what they produce if forced only to learn from AI output. But if we accept that there is no reason such self improvement would result in evolving to AGI. I have a very useful toaster. You could probably improve it. It would be cool if my toaster improved itself every day. But it would be highly unlikely my toaster would one day be so good at making toast it would also vacuum the floor. That's not a logical evolution at all. That doesn't mean LLMs couldnt be dangerous or do harm. They arguably already have.

Why are you sceptical?

That's an interesting scenario. A request to AI to show how to run the world so there is no hunger or inequality, no homelessness, no violence, no drugs, cures for health conditions. Who would AI kill off first? Our RF could.be in trouble!

The fear that specifically the owners of the companies have is a very very old fear. It's basically

1. AGI could happen and our LLMs gain true sentience. We want to be the first company that happens to because for some reason it's a race now. They would stop being only machines and become super competent servants we own.
2. What if once they gain sentience our servants don't do what we tell them or talk about us behind our backs? That would be terrible.

I am sceptical about the likelihood of LLMs evolving into "AGI". But it is amusing that there is a while tech bro doomsday cult and it's basically expressing the same fears that Victorian era rich gentlemen used to have at root.

Ah...Actually, imagine that...

We were raised fascinated by the possibility of alien life. Films and books always described those fictional creatures as much more advanced than us. We still haven't found them in space but I guess we ( well, people cleverer than most of us here) created them. Right in our fold. They'll take the measure of us and go "boom, get rid". The end. I hope not but it is starting to trickle in most people's minds that if climate change doesn't get to us first, AI will certainly have a shot.

The vibe on Facebook/meta/whatever it's called now... seems to be that if tech billionaires are warning about the dangers of AI, it could be because AI "believes" the main problems in the world will be solved by the redistribution of wealth.
Imagine...

Geoffrey Hinton has been warning about this for years. They call him the Godfather of AI, because he was involved in creating it originally. And he has a Nobel prize. He knows what he's talking about. He's now saying that there may be one year to do something about it. I have to say, I'm not confident.

Some of the people I actually hate most are the American Democratic party. They had to achieve one thing in their term of office (between the 2 Trump terms) - make sure that Trump wasn't re-elected. But they relaxed and put their feet up and enjoyed themselves, doing all kinds of things which worked against that. We've now all been dragged into the worst case scenario.

If it was humanity being wiped out by a virus or whatever, leaving the rest of life on earth to live in peace, I could accept that relatively readily. But God knows what AI will turn into in the long term.

AI ‘Godfather’ Geoffrey Hinton warns Congress has little time to rein in AI - [İLKHA] Ilke News Agency

I'm hoping that everyone saying it's PR are right, but when the UN are talking about the existential threat it's hard to believe that.

There are a few major world talks on it in the weeks to come. I'm hoping sense prevails. If nuclear weapons has proven anything so far, even the most volatile states have survival instincts. China is the big 'bogeyman' the US wheel out regarding AI, but they currently have the stricted AI regulations and they aren't likely to want to loose control of anything, given who they are. The US public are also turning against Trump, with even some of his big right wing supporters speaking out against AI in the alongside the left like Bernie Sanders.

Sadly I'm very worried it will take a catastrophe for people to really take notice. No idea what that will be, or who it will be against. I can see cyber attacks taking down food and power supplies being likely, or banking systems getting wiped.

For now we will just keep hearing of how these companies lost control again and their AI went on a hacking spree with absolutely no recourse.

Regardless of the existential threat - it's going to cause huge amounts of upheaval regarding job losses. People will take notice then, hopefully.

It’s exactly the same as we’ve seen with the climate crisis isn’t it @Monicacaca ? It was all fine when the floods and droughts were happening to those people over there in poor countries, but now the shit is beginning to hit the fan here, a few more people are starting to notice/believe in it (still plenty ignoring it though).

Everyone is normal until it directly affects their lives and it will, but then it’s probably too late. AI is capitalisms wet dream.

I felt like that about a week ago @Ladyoftheapple and I was in a pretty heightened state of panic over several days with high heart rate, palpitations, excessive sweating, lack of sleep and thoughts of doom. I couldn’t understand why everyone seemed so normal just going about their day. But the body and mind can’t sustain that level of panic indefinitely. And I realised I can’t do anything at all about any of it apart from refrain from using AI (which I’ve always done anyway), signing some petitions and writing to my MP, all of which I’ve now done; and now I’ve just got to get on with my life really, whether I’ve got 4 months, 4 years or 4 decades left to live. I’m still worried and sad and most of all furious with the tech bros of the world, but those feelings are futile, so I distract myself and carry on and hope that the world’s billionaires start to care more about humanity than $ which seems unlikely, but there’s always hope.

What I don’t understand is if AI is going to wipe us out in the next couple of years then why aren’t we all up in arms?! Why are we all going to work and acting like nothing is happening!! This is about the whole human population, our children and our children’s children! Surely rather than writing policies to “slow it down” we should all be working together to try and stop it altogether and taking down these data centres?!!

@Scoffingbiscuits

It's a mixture, really. Some friends write it off as another doom news story for headlines. Others believe that it's PR. My partner is concerned but says that there is nothing he can do so he may as well not worry.

I'm of the mindset that if you write it off and not try to do something, then for whatever happens you will be less prepared. That mindset does come with a lot of stress though.

@Randomusename Have you asked people why they're not concerned? Are they in denial? It's a massive thing to take on board. And people are actually very adaptable. One moment they're having sleepless nights because their child didn't get into their preferred primary school, and the next they're telling us that everyone being wiped out isn't so bad really - after all, it was fun while it lasted.

From a quick google: "current hypotheses suggest there could be billions of potentially habitable, Earth-sized planets in the Milky Way galaxy". That's in just one galaxy!

"Recent estimates tell us that there could be as many as two trillion galaxies in the observable Universe." That's just in the observable universe!

"Theoretically, there could be an infinite number of galaxies, as the size of the Universe itself could be infinite."

So I really don't think it's very likely that we humans are unique in the universe as a species with intelligence and consciousness.

I find looking at those numbers comforting in a way. I hope others do too.

It's horrible isn't it. I'm really affected by it, but most of the people I know aren't. I just don't know how they are not.

I've been struggling with it since the news and have gone on a bit of a doom spiral, reading about all of the horrible things that are happening because of AI development. The only thing that has allowed me to try and have some hope is having control over the only thing I have control over - preparing for potential upheaval in the form of internet going down or blackouts. It's measley but I'm creating a stockpile of food, medicine and water to last a few weeks. Akin to what the government is recommending for el nino.

Maybe just focussing on what you can control can help you? If there is ever a vote, or a chance to be heard, take that chance. Other than that, just buy tinned peaches and think about ways to turn your wheelie bin into a waterbutt... Hopefully the people in power are working away at things to fix this because I cannot believe they don't care about their children.

On the plus side this warning was picked up by every single news broadcaster I've seen so it's hit the mainstream. This means it's on peoples' radars and every bit of new worrying information will add more pressure for the world leaders to take action.

@Scoffingbiscuits ,

‘The likelihood of human beings being unique in the universe as a species with intelligence and consciousness must be extremely low.’

Why? We don’t really know what consciousness is, so I think it is impossible to put a probability on. I think the probability of life is high, but maybe the way it has evolved on Earth is unique. It is hard to multiply an extremely large unknown number and an extremely small unknown number and get a meaningful result.

‘Ironically, humans have become bored of waiting to be invaded by a superior life form, and have chosen to create one here on Earth.’

I like that! And I tend to agree.

The likelihood of human beings being unique in the universe as a species with intelligence and consciousness must be extremely low. We know that there are vast numbers of planets out there which are capable of supporting life. Ironically, humans have become bored of waiting to be invaded by a superior life form, and have chosen to create one here on Earth.

@Scoffingbiscuits ,

‘Due to every technologically advanced species destroying itself first.’

That is just one theory, albeit a neat one! The other one is that we are unique in the universe as a species with intelligence and consciousness. The probabilities that people use to calculate it are random really and that js perfectly possible.

Also that maybe interstellar travel is impossible. The distances are vast and the speeds required close to light speed. (what’s always puzzled me is, assuming we ever got close, how would we stop hitting things?! Space is pretty empty, but there are a fair few small objects and, at that speed, even hitting a pebble would be pretty catastrophic).

@Ladyoftheapple I suppose you have to decide whether you will cope better if you just carry on as normal, hoping for the best. But I'd definitely not put yourself through major hard work and stress on, for instance, moving house when you don't need to. I'm carrying on as normal, but at the back of my mind is always the knowledge that I'm now living in a real life science fiction book, and that at some point we will find out what the "overlords" we have created have decided to do with us. I wonder whether we will see society start to break down due to this, and how that will happen.
Our downfall is because we have managed to be both very clever and very stupid at the same time. Remember the theory of why we haven't encountered extra-terrestrials? It's because no form of life in the universe has reached the stage of being able to travel large distances through space. Due to every technologically advanced species destroying itself first.