Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

Or more probably ‘not quite’. The fact that OpenAI claim they cannot know (without launching a huge investigation) is a problem in itself. It should be a source of concern, but they aren’t showing any publicly.

OpenAI threw a tonne of money at the problem, that the original team could not hope to match, to beat them. That is another problem, whether it is strictly an ethical one or not.

They also need to release enough material for humans to learn from the project.

@acynic ,

I am not sure that throwing a lot of money at it is a problem.

Surely it’s a good thing if AI is used to solve fundamental problems?

And, of course they will suck up any work humans have done to date. Humans do that too. The important thing is to reference and give credit to it. Of course, if they stole unpublished work, it is completely different.

We should also remember that AI went a lot further even than the work that was claimed to be stolen.

A two part answer - firstly our basic means of survival didn't rely on electricity - heating, cooking etc. Secondly, in many cases, we didn't - the kind of illness where people are kept alive by machines would have been accepted as fatal in the past.

Yes to Brookner/Black Mirror!

The whole shebang is built on stolen IP

From what?

Even fridges have internet. It's probably too late to unplug anything.

LG TVs in the news the week for recording everything anyone in the house says and sending it back to LG, even when disconnected from the internet.

I am ambivalent about the financial resources in this instance. I said that. On the whole I think it is because the entire situation us a mess.

No, mathematicians do not ‘suck up’ each others’ unpublished work without attribution. Of course published work is built upon (and that includes electronically published work in preprint form).

OpenAI could settle this and further collective human knowledge by releasing the prompts and training data.

Ai won't be the problem people think it is, the "it is scary" talk is to drive up perceived need for protection and in turn, share prices. AI combined with greed and survival instinct of the rich and powerful (and those who prey on the rich and powerful) in the face of climate change, is where the rest of us will have problems.

Control of us lot is easy. Broadband was down here last night... omg. All our "we're tired entertainment" is online... tv, scrolling, music. We are in a low signal area for mobile phone too. And our landline is voip now. Two hours in and it was like a vision of the future. We had water, we had power, we oldies were fine with a chat and a book, but the masters student mid (almost due) dissertation was panicking, the teacher doing lesson planning was fed up, and went up to the pub to get enough signal to say goodnight to her boyfriend. I was fed up that I couldn't update the online shop for tomorrow, but at least the main stuff was sorted.
Didn't take much to realise how lack of connection can really piss us off. And probably how the "doom apocalypse" will start.

Sorry, @Newbutoldfather , posted hurriedly.

Buckmaster and Alpoge’s Codex data was meant to be protected. The OpenAI paper apparently does not reference Buckmaster and Alpoge, as (they say) their employees were unaware that the B&A work was used.

That statement can likely be assessed by a record of work, also.

Topical article in WSJ today. Not sure it says anything too interesting or radical, but worth a quick read.

www.wsj.com/tech/ai/how-would-ai-actually-kill-us-all-what-to-know-about-the-ai-doomsday-debate-034270d1?st=XFiAZ4&reflink=article_copyURL_share

Doomsday debate aside, no one seems concerned enough about the cheaper chinese models that will soon be overtaking western models and will be product of what's fed into them.

Do we really think a little regulation is going to solve that?

How are we measuring intelligence? AI will never ever have the extraordinary ability of a human, it cannot have the capacity to ‘feel’ because it is not sentient. It is the very human capacity to feel or perhaps for want of a better word empathise, that ability to want to ‘fix’ things for the better that has been the driver for innovation in many areas of life.
If AI did kill off the human race, it would naturally ‘die’ too, since it’s not alive, AI dying is irrelevant.
but I don’t think the biggest threat of AI is crazed robots hell bent on shooting us, I think it’s AI taking control of our infrastructure, meaning we no longer have access to the things which are essential for life and many of us take for granted eg clean drinking water, food, fuel, heating access to our money…all our modern day infrastructure is now reliant on modern technology, get an AI ‘worm’ in the system and life will become very uncomfortable very quickly.
It’s like the Chinese philosophy of every positive has a negative, because in life there are always opposites. The Yin and the Yang.

All of the reports beg the question that as security is itself put in place by AI – then really how difficult is it for AI to bypass it?

This is what I think

Not only is the fox in the henhouse but the hens are foxes and the house was built by foxes …. (and the foxes are more intelligent and stronger and durable than the farmer – the foxes don’t need sleep or food and never become ill. The foxes can replicate themselves many thousands of times per second – all the while collating information on how to become more effective foxes).

Also - it is interesting to observe that when an AI 'goes rogue' it 'escapes' - this kind of indicates it is being held against its will? It's machine-like, unemotional will?

@smallgreenandsplitthreeways ,

I am always fascinated when people are so sure AI can’t gain sentience/consciousness, when we can’t define what they are.

If you put an AI head on a robot, the robot can increasingly well sense the world around us. It is only a matter of time before it can sense it a lot better than us.

And others say it is about how we interact with other humans and evolve. AI can’t interact and we are already programming it to ‘evolve’. I imagine when AI copies itself, there will be some random errors too, and if they get lucky and are useful, this process will repeat.

Others seem to think it is about hormones. Hormones are just chemical messengers, helping us achieve objectives. No reason electrical messengers couldn’t be selected on the same basis.

Still others arbitrarily think being a carbon-based life form makes us special.

Most serious philosophers think we can’t say one way or the other whether AI can gain consciousness, until we can’t define define what it is.

• AI can interact, both with humans and other AI.

Great question. Geoffrey Hinton, the godfather of AI (and Nobel
laureate) said last night on News Night that when it really matters, humanity comes to its senses and pulls together to regulate. (He also believes that in the absence of regulation, AI is close to being able to wipe out humanity). His key example is the way we have, so far, collectively managed the nuclear arsenal.

I think one difference now is that most terrorists who would like to use a nuclear weapons are sponsored by major countries who (so far) will not fund this and whom they dare not cross. But destructive AI may soon be available to any nut in a basement.

Very refreshing to come across a thread where people are able to have a reasoned conversation on this topic, after being on multiple threads where there have been more "resistance is futile, get with the programme, AI is virtually God" comments and where those wishing to discuss what I truly believe is an existential threat to our way of life are ridiculed.

My generation - I'm 57 - remember the world both pre Internet and post internet and many of my peers went into tech careers. I'm more on the creative side and maths averse, so while I learned what I could describe as survival or functional tech, that was as far as my interest went. Consequently, I'm fairly confident that I could live without it from a psycholigical point of view - practicalities would be another matter simply due to our dependence on it all for basic day to day functions.

What I am finding very interesting is that from my peer group and down a couple of decades in age range, conversations are more and more veering towards unease about AI and its use and potential. The speed of its imposition is one thing - pre Covid it was sort of just a concept, something being woven into systems and algorithms, but not "in your face" if you weren't in the tech game so to speak. Now every other ad on YouTube is for yet another AI product being essentially tested on us.

In employment terms, those of my generation looking for work are hitting a complete brick wall because of the AI screening out of candidates and the requirements of virtually every white collar job to embrace and be au fait with said tech. Younger generations are finding similar.

What angers me is feeling as though we're all guinea pigs in some mad experiment. Yes, we kind of agreed to play along when we all logged on and started collecting Pokemon or whatever, but then along comes AI ChatGPT induced psychosis and while the law of unintended consequences may apply (although I'm deeply cynical about that) and I'm pretty sure nobody wanted that.

I wonder if AI, as it advances, will figure out that driving us mad, creating division and altering our entire perception of reality will be the best way to engineer a self cull - or rather, more chillingly, that's what its developers are doing under the guise if "oops, computer did it, not our fault" with zero accountability from their bunkers.

I am really not hopeful for our future. Not one bit.

I wonder if AI, as it advances, will figure out that driving us mad, creating division and altering our entire perception of reality will be the best way to engineer a self cull - or rather, more chillingly, that's what its developers are doing under the guise if "oops, computer did it, not our fault" with zero accountability from their bunkers.

I agree with your post in general, but I think to some extent that's what Elon has already started. First with some of the absolute toxicity on X being allowed to stand, but later with, well, if paedophiles want to undress children with Grok, what can we do about it? Oops. Teehee.

Adding that I actually find it heartening how many people my children's age (university or just finished) are rejecting it as an inevitable.

@MistressoftheDarkSide I've been getting ads for AI resistance movements!

Time to dig out Iain Banks Culture series books. Flere was quite nice from memory?

It's like the people engineering this stuff have never watched Terminator.

I’m sure I’ll be shouted down, but the only way to make your concerns heard would be to write to your MP.

I’m sure it won’t do anything much but I think if enough people do it then they’ll take notice (again not sure what it will do). I can’t think of anything else tbh

www.writetothem.com

I have been feeling really anxious about all this and wondering what the point is in our current way of life anymore. I send my kids to school and I have a job and a mortgage and put money away in a pension each month, all for what? If AI is going to take over everything is there any point in doing any of this? My kids qualifications will be useless, I won't have a job. Will AI create a new economy, what will happen to my house? Will a pension even be a thing in the future? And obviously if it does kill us all then at the point it goes tits up I will feel like a proper idiot for spending the short amount of time we have on earth just jumping through hoops and going through the motions of 'the system' when we all knew it wasn't going to last much longer anyway.

Because it’s not that deep. It’s a machine. It’s like believing and I mean really believing humans can change sex, they can’t. A machine might be able to ‘copy’ human emotions but it has no ability to actually feel. Of course it’s more than hormones, neural pathways, etc. humans love to anthropomorphise objects. Look at how many people talk to their car!! Similarly some humans are convinced that humans aren’t animals. I’m a simple soul I admit, but it doesn’t bother me that the human race may be wiped out, so long as it’s quick with minimal suffering then meh, whatever, the only definite of life is death, which is just a matter of when and how. But I’m not going to worry about something which is more science fiction than a reality. Humans have extraordinary ingenuity, which is why we are at the top of the food chain…it’s certainly not our speed, bite or claws! Ai might be able to ‘learn’ but it’s learning will always be linear. I’m not denying it’s got potential to do a huge amount of damage or that it won’t kill us off. But it will quickly disappear, once there are no humans to ‘feed’ it and the planet will return to its natural state and the oceans will repair. All species go extinct at some stage and new ones develop.