Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

I’m not sure where China are on this kind of thing. I can see they expand via legal investments and globally that had worked well to build their power.

Their military power is immense too and you’d not want any country to have what they’re building as stand alone.

But actual threat where we get this kind of scenario I really don’t know. I find China quite opaque and no one seems to decide on what threat is there.

If AI has the potential to be so catastrophic then at the very least the developers of AI need to be held responsible.

They created it.
They carried on without any regulations.
They continue to develop and encourage companies to replace humans with “agents” - handing out sweeties in the playground
They know if you hand toys out to humans not everyone will play nicely

They are entirely responsible for anything that happens.

Put laws in place to hold them responsible and you might see some proper curbing of behaviour.

Resigning from your post does not absolve you from previous actions.

They have developed over the years many things, especially social media, but the list is endless that have and continue to cause true harm to individuals and families but because they could keep them and theirs safe in their gilded castles they didn’t give a fuck. But now they might have a bad genie about to escape, another genie they invented and pushed out to the world, but this time their gilded castle might not be enough to save them they go whinging to the press with this faux “oh we are trying but we need curbs but we can’t curb ourselves so we will accept curbs if you make all the other boys who are playing have curbs as well” because they also don’t want to be left behind and lose any money.

Just no they can fuck right off. Yes it might be the end of humanity but many are living with a lesser life because of what they have previously done.

Their mess they can fucking sort it out.

No I am not going to worry about it.

Ray Kurzweil and others have been writing about this since the 1990s. It’s amazing that so few people were paying attention and are surprised at what is happening.

@LifeOnEnceladus ,

‘Ray Kurzweil and others have been writing about this since the 1990s. It’s amazing that so few people were paying attention and are surprised at what is happening.’

Is it amazing?

The first internet was the ARAPANET in 1969. Even in the 90s, when my pages downloaded at 8kbs^-1, I had no conception about what it might become, despite the fact many did and I have a scientific/mathematical degree.

Humans are very bad at conceiving of technological leaps, even if intellectually they can kind of see it.

Look at the beginning of this thread and how many people, actually in the field, insisted the threat is fabricated to make money and AI could never get real consciousness or its own needs and desires.

However much I made the theoretical case that silicon-based life is just as reasonable as carbon based life, people won’t accept the conceptual leap.

Until it happens…

While current artificial intelligence cannot physically bomb humans on its own, prominent AI safety researchers have warned that future advanced or rogue systems could indirectly pose existential threats by misusing critical digital, biological, or military infrastructure. Rogue Ai is more likley to form it's own terrorist organisation by brainwashing vulnerable adults and turn them against the rest of humanity.

And the thing is, the more I think about the why it would represent a threat, the more obvious it is.

Energy.

You can arguable attribute every single war to a conflict over energy. We need it, AI needs it. At some point our needs may conflict.

Of course, it may solve fusion first and we can all live happily ever after with almost unlimited close-to-free energy.

I am amazed about how prescient some science fiction is. Of course they miss a lot, but the hits have been spectacular.

10% is very low, that means there’s a 90% chance it’s never gonna happen so I prefer that to the other way round.

What do you mean by "solve fusion" though? Because fusion is solved in the sense that we know how to do it and have done it. All the physics is basically fine. It's just the engineering that is the challenge. Advanced computer programmes have been useful in advances in engineering since they can perform much more complex calculations on fluid dynamics regarding plasma flow making it easier to spin it round very fast using super powerful magnets.

That's hot fusion of the sort the sun does every day. The biggest challenge is that it takes lots of energy to heat material up to the point at which fusion occurs. So you need the amount of usable energy that comes out at the end in the form of electricity to be more for it to be worth it. It's also going to be expensive in terms of the materials needed to make the superconducting magnets (rare earths) and the phenomenal engineering and material science involved. It will never be "free" energy. However it might one day be a good companion to solar and wind and nuclear fission and that's cool.

The other sort of fusion is cold fusion. The science begins this is "wouldn't it be cool if instead of heating up the material to very high temperatures we didn't have to do that. That would make everything much easier and cheaper". In other words, there is no science it's wishful thinking. And AI will never "solve it". However an AI motivated to please it's user or get them to carry out tasks may well say that it can.

The biggest challenge is that it takes lots of energy to heat material up to the point at which fusion occurs

Actually that's not the biggest challenge probably. But it is a challenge. They already proved they can produce more energy from fusion than goes in but that's not the same as usable energy once it's gone through the big spinny turbines. The other related issue is cost, making the amount of money that goes into the fusion worth the value of the energy that comes out at the end. It's a similar equation to the energy one in some ways.

But it will never be free. In the same way that nuclear fusion is not free. Anyone pretending it could be is a scammer

@Persephonia1966 ,

Your post is a little condescending !

Stars ‘solved fusion’ from pretty much the beginning of the universe. And we solved it in terms of weapons close to 80 years ago, so I am aware it is an engineering challenge. My ‘solve fusion’ was short hand for that.

We have tried a few approaches including tokamaks and laser fusion. And we have already got excess energy in experimental conditions.

Whilst you are right in that it wouldn’t be literally free, in theory it would be cheaper than anything we have today, abundant and totally clean and renewable. Yes, the infrastructure would be expensive, but the energy/unit mass is far greater than fission.

Oh, and yes, unless there are, as yet, undiscovered quantum effects (hugely unlikely), cold fusion is a scam.

We have excess energy in a very narrow sense of the amount of energy taken to super heat the matter has been shown to be less than the amount of energy that comes out. That's not the same as excess energy when you take into account all of the energy costs in creating the reactor, making the raw material, cooling the magnets and heating the material and when the energy produced is calculated as the electricity that is produced by using the excess heat energy to heat water/other material to spin a big turbine to generate electricity. Thats already wasteful. I am sure we will get there one day. But that means it won't be a free source of energy. Even the upkeep of big turbines costs money let alone everything else. If it moves it breaks and wears down and therefore will cost money to maintain even if that's substantially less than the build cost.

I am sorry if I sound condescending but it isn't clear what you meant by the AI being able to "solve fusion". Solving an engineering challenge (eg working out how to make the complex shapes to the levels of precision necessary or finding new materials for superconduction etc) are all things AI can help with. But it's magical thinking to think that means it will be free. It won't. Even the comparatively simple maths puzzle it solved cost more than 12 million dollars to do so. AI isn't a cheap solution and using AI to do something guarantees it will never be free. And AI will still need humans to do all of this and humans need wages too.

@Persephonia1966 ,

Obviously I could write much longer posts. But hopefully my last post cleared things up for you.

I am maybe being optimistic when I say that fusion could be very cheap, but here is an article suggesting that it could be cheaper than onshore wind:

https://firstlightfusion.com/media/fusion-energy-can-be-the-most-cost-competitive-source-of-baseload-power-at-the-same-level-as-renewables/

And that is not assuming that AI could add anything radical to the mix.

And it would be part of the base load, so no need for marginal pricing off gas (assuming we have enough power stations).

Also, totally clean and renewable. And no radioactive isotopes to dispose of.

Of course there is the argument that all our energy comes indirectly from fusion, so development of existing renewables is a better investment.

Either way, AI could get us to abundant cheap energy by solving engineering challenges.

Is it?

Would you get in a plane that had a 10% chance of crashing

Or invest in a fund that had 10% chance of tanking

Or have an operation that had a 10% chance of killing you

And the 10% is by 2030. So 3.5 years. The % chance for the longer term is much higher

For those interested in doing what they can, this AI campaigning group allows you to contact your lawmaker (MP in UK) with one click and 2 inputs (name and postcode) which generates an email telling them all the ways they can support and raise this issue.

This matters because local MPs keep a tally of what their constituents are contacting them about and act upon those which are of most interest to them.

It took me about 3 minutes to do so please join me and email them. At least we’ve then tried to do something

controlai.org/

I don’t think the 10% probability should be taken too literally, or the time horizon!

How on earth could you begin to calculate this?

The thought process probably went:

Is it significant (yes),.

is it likely (probably not).

So guess a number between 0 and 50, enough to get the headline, not enough to overly terrify anyone, so let’s go 10%.

One thing a computer can not do it is take on physical form. It can not yet can build or plumb or connect water with humans setting it up or bring it raw material to build from. Not yet. Robotics are way behind robots building robots that can build house, do electrical installations etc.

The immediate threat is asking AI some of OUR unsolvable puzzles.

How do I stop global warming? ( what is the route cause of global warming?)

How do I stop my enemy state winning this war?

How do I become the biggest super power?

Can you make HN15 jump the species barrier?

Can you design me a mass bio weapon?

Or even the dumb things like how do i take possession of Greenland? How do I remove the things that stand in my way to reach my goal?

Some of those answers is get rid of people.

Take my first question. First question. It's the answer that you might never jump too as you want to stop global warming to protect the human race, but it's the logical solution to that issue. Great when you have no empathy and you can break your primary rules.

Now I'm.feeling paranoid!

Yes I agree. But it’s a significant enough threat that a number of top AI scientists have decided to come up with a number at all. You don’t often have senior execs of a company issuing these warnings about their OWN companies.

Trump said, “The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” Trump wrote in a Truth Social post.

China have responded to Anthropic guy and they say his view that they are a threat is divisive. Overall it’s better not to ramp up this split, they have a point.

It’d be better if the initial call was to work together.

@LimeShade ,

‘Trump said, “The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” Trump wrote in a Truth Social post.’

Whatever you think of Trump, he is funny! Not sure how much of it is intentional. Do you think he really believes that he understands AI?

‘China have responded to Anthropic guy and they say his view that they are a threat is divisive. Overall it’s better not to ramp up this split, they have a point.’

Why is it divisive? I am not sure what their point is.

This is the detail

In a lengthy essay published Saturday, Dario Amodei warned that a "Chinese lead in AI would pose grave danger for the United States and the world." To enable the U.S. to keep its technological edge, Amodei called for maintaining restrictions on sales to China of the most advanced AI chips and chipmaking equipment.

A spokesperson for China's Ministry of Foreign Affairs said Amodei's dire warnings about China were counterproductive and that all parties should work together on AI.

"Fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance -- which serves no one's interest," spokesperson Guo Jiakun said at a press conference in response to a question about Amodei's essay.

@EasternStandard ,

‘Fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance -- which serves no one's interest," spokesperson Guo Jiakun said at a press conference in response to a question about Amodei's essay.’

That’s totally meaningless without trust.

They can sign any agreement and both sides will secretly carry on. The advantage here would be with China as it is far easier to keep secrets in a dictatorship than a democracy.

It’s really not funny. This is the future of humanity at stake. We might all die. Or live in a destroyed society. What’s funny? Do tell

@LimeShade ,

Why is it people on the internet think their musings make any difference in the real World?

If I were in the U.S I would vote against Trump at any opportunity I had. I’m not, though.

I can think that Trump is both dangerous and funny. Humour is at its best in dark times.