AI - A 50% chance of human extinction?

[Solongasnake]

AIBU to think that this is more important than anything else that's going on? Is there anything we, as individuals banding together, can do? Or do we just have to sit back and accept it, on behalf of our children as well as ourselves? I've read predictions that AI will become more intelligent than humans in the next couple of years, and that to stop that, action needs to be taken in the next year.

"Geoffrey Irving, who worked at OpenAI and Google DeepMind and as chief scientist at the UK’s AI Safety Institute, said warnings about AI risks understated the dangers.
“There’s about a 50% chance we all die because of the development of smarter-than-human AI systems,” he said, adding that actions taken over the next two to 10 years would be decisive.

OpenAI’s David Robinson announced his departure at the weekend, saying “the companies building this technology aren’t being nearly careful enough” as they sprint from one product launch to the next.
His view was endorsed by Miles Brundage, an AI policy researcher who worked at OpenAI for six years. He said he regretted helping “spread the idea of iterative deployment”, which may have made sense when AI was less powerful “but makes no sense at all after many deaths have been tied to AI and as we’re careening towards extinction-level risks”.

www.msn.com/en-gb/technology/artificial-intelligence/accept-bad-things-in-return-for-benefits-of-ai-says-sam-altman/ar-AA2dBJA6?ctsrc=dgst&ocid=edgntpruby

What will it do after it's killed us all? How will it generate the electricity it needs to keep the computers on?

Great idea! Tell us where this mythical plug is please?

You need to watch less TV.

Why on earth isn’t it being stopped now while we have the chance?!

Because lots of people have invested lots of money in hoping the public will work out some application of the technology that will make it worth the environmental and economic price.

Yay, free market capitalism!

I agree, I do think this is one of the important topics of today (if not the most). At least the UN and others are starting to take the risks seriously, even if Trump is not. Thank goodness for people like Geoffrey Irving who is trying to help solve the alignment issue and Geoffrey Hinton et al who are spending their should-be retirement years telling the world that we’re on borrowed time if we continue along this trajectory. And borrowed time is a couple of years, not 100.

How can they actually solve the problems, I have no idea, is it even possible? The research scientists don’t really know how the models learn or reason (just that they do and get better with tweaks to various weights). And yes I work for an AI company before anyone asks.

It’s difficult for Joe Bloggs to grasp the risks as anything but a PR stunt over a hyped technology (“but free chatGPT / co-pilot is soooooo shit”) because they don’t see the capabilities of the frontier models first hand, understand the risks of agentic AI swarms let loose on the internet and the effects aren’t “real” yet, but I am really concerned. These AIs have show the ability to deceive, lie, manipulate. They are incredibly persistent and tenacious when pursuing a goal. And recursive self improvement (where the AI model designs its next iteration without the need for human in the loop) is fast approaching.

My fear is we won’t even see it coming and then it’s too late.

The research scientists understand very well. I'm not sure which "AI company" you work for or in which role, but don't diminish experts like that just to hype people into a frenzy.

With respect, no they do not. Yes of course they understand how the architecture (neural nets) were designed and built and the fundamental coding and maths, but not how the systems actually reason or learn.

I suggest you do some reading.

As an actual real life qualified and employed scientist, I do plenty of reading.

Claps for you for knowing the term "neural nets" though. Most impressive.

With respect.

Not being funny but Copilot couldn’t manage to format a Word document with multilevel numbering in it today so I’m not that worried about the end of civilisation just yet.

There really is no need to be so rude. We’re all adults here and capable of researching a topic.

Sorry for the derailing the thread OP. I share your concerns and don’t believe sticking our collective heads in the sand in hope the “experts” can solve all the problems is helpful.

@Whyarepeople ”The history of AI is littered with stupid claims like this. Sometimes they come from arrogance - such as when Geoffrey Hinton, a man who knows less than zero about radiology claimed in 2016 AI would replace radiologists within five years.”

I was talking to a radiologist in the park the other day, while walking the dog. He specializes is cardiology.
He said his job is mostly now double checking AI diagnosis, and AI is as good at diagnosis as he is.
His opinion was he needs to get out of radiology because AI will take his job sooner rather than later.
Anecdotal I know.

Interesting (and sad). Yes Hinton is candid about how he was way off with his radiology prediction. However he was only wrong about the timelines, the red tape and the slow uptake by hospitals. Not about the fundamental capability of the AI to do the job at hand.

Nobody asked you to stick your head anywhere. I simply stated that there is no need to throw around statements like "EvEn the SciEntIstS Don'T UnderStANd iT!" so you can feed an unnecessary panic. You are incorrect.

If you'd like to come forward with your suggested plan to stop the AI from taking over the world and destroying humanity, based on all of your research, I'd be genuinely interested to hear it. Otherwise, perhaps you should indeed leave it to the experts.

But I haven’t said they don’t understand it (AI, at all). I’ve said they don’t understand how it learns or why it displays certain (good or bad) tendencies. For example, Fable was not designed to be a master hacker, but it (independent of its specific design, which was for long horizon agentic work) developed those capabilities during training.

We can safely say that AI can be unpredictable at present, and that it will soon, if not already, be improving itself.

If both of those are true, anyone's guess for the future is about as good as anyone else's, expert or not.

Because people enjoy a good doom and gloom story.

It is not terrifying and it if it kills is it will because because of the humans operating it. Not because it will refuse to turn off. Because we could just unplug it.

This is the last I’ll say on this topic, because I don’t have the energy for more squabbling, but Geoffrey Hinton (Nobel prize winner for his work on artificial neural networks, so pretty darn expert and so called godfather of AI) said in a CBS interview in 2023:

“Geoffrey Hinton: We have a very good idea of sort of roughly what it's doing. But as soon as it gets really complicated, we don't actually know what's going on any more than we know what's going on in your brain.

Scott Pelley: What do you mean we don't know exactly how it works? It was designed by people.

No, it wasn't [designed by people]. What we did was we designed the learning algorithm. That's a bit like designing the principle of evolution. But when this learning algorithm then interacts with data, it produces complicated neural networks that are good at doing things. But we don't really understand exactly how they do those things."

So they designed the blue print (brain) for AI, but do not fully understand how exactly they arrive at a certain answer or action any more than when a human brain does. Just that they do.

And if the Nobel prize winner does not understand how they do the things they do, I’ll continue to feel a little uneasy when these things are scaled up at the rate they currently are.

On the wall.

You can not unplug something that is moving through multiple systems simultaneously, replicating and reinventing ITSELF. Think about the aviation, energy, food, water, healthcare systems we rely on and the digital structures they're completely reliant on now.
I think people have forgotten how small and vulnerable we are. We have generations now that have never experienced population-wide warfare or famine. We're not immune. You may find it hard to imagine, but that doesn't afford us any invincibility.

It’s like no one ever watched Terminator.

Everything has its time and everything ends. Maybe it’s time for the human race to exit stage left and make way for whatever comes next? Only human arrogance believes we have a god-given right to exist.

There's an animal instinct for survival though, isn't there? The idea of my children dying, and all children dying, before adulthood, is unspeakably sad. And there's also a huge sense of injustice about it. They're robbed of life for a very few incredibly arrogant men who believe they're god. Imagine that we are the last humans. It's too much to stand, really, for a singular human mind.

The humans won in the end, so probably not the best comparator here. 😛

I would find it easier to accept if humans died off, leaving animals and plants to take the planet back. But who knows what AI will do with the rest of life on Earth. It has come out of humanity and could be even worse. If it is vastly more intelligent than humans, I suppose it may manage to travel to other planets and cause havoc there, too.