Amodai's article is interesting, though I find it very carefully phased to avoid pointing out that much of this comes from AI companies, not the AI itself.
When discussing the Open AI-Hugging Face incident, he's leaving out that the Open-AI experiment intentionally gave the AI agents impossible instructions within a framework where they couldn't quit to see what they would do. They found a way to communicate and work together which ultimately lead to what he describes. Calling it 'misaligned' and going on about the AI agents doing things that were not in their remit is to take the responsibility away from the fact that their remit was intentionally impossible and the whole point was looking for what they would do beyond the impossible instructions.
Yes, what happened was not intended, unintended consequence of badly given instructions have been warned about since the stories of fae, genies, and pretty much anyone who has dealt with someone following the letter rather than the intent of any instruction. The whole situations reads like a bad ethics play about badly treated labour and being careful what you ask for.
Framing it all around 'pacing the development of technology' rather than 'what are we companies building these models and agents asking AI to do and how can we bring ourselves into an ethical alignment' shows some blind spots at best, some potentially intentional ways of trying to avoid responsibility when things do go wrong at worst, and others said, there is a middle ground of trying to hoard the capabilities for themselves while using measures to stop others.
Our government could speak against this, but recent governments have sold most of our digital infrastructure to companies outside of the UK, many of them in the US meaning our digital infrastructure is under their laws. Look at the contracts the UK has paid out. We'd would need to be able to pressure the government to build digital autonomy first somehow before their voice could carry any weight.
I don’t know why people are so laissez faire about being driven to extinction so some men in the US could get rich. Why aren’t people raging about this?
As many have said, part of the problem is that many of us were raised being told the various ways the world was doing to end while also being surrounded by adverts like we still have today that we all need to do our bit at home and reach out to our representatives about this and that, donate to this or that, all being paid for by companies that have far more impact than any of us could ever hope to have.
The work many have done that has been to prevent the disasters and reduce the problems are often far more hidden from view compared to the constant toll of 'this is how the world will end this year'.
This leads to fatigue and apathy. The cynical side says that may be the point.
The comparison is about the mass hysteria. People were stockpiling, talking about hiding in bunkers etc, and like you say it took some changes to computer programming to fix it.
That was mainly stoked by people trying to sell thing, not the experts or anyone who was working on the Y2K IT issue.
Much like the 2012 'world is going to end' was mainly by people trying to sell things, not by any Mayans or anyone with any understanding of the Mayan calendar. Similarly, many of those who made money mocking it were also not experts or showed much understanding.