Like many in silicon valley Lanier has some batshit luxury beliefs:
Data dignity is not just for white-collar roles. Consider what might happen if A.I.-driven tree-trimming robots are introduced. Human tree trimmers might find themselves devalued or even out of work. But the robots could eventually allow for a new type of indirect landscaping artistry. Some former workers, or others, might create inventive approaches—holographic topiary, say, that looks different from different angles—that find their way into the tree-trimming models. With data dignity, the models might create new sources of income, distributed through collective organizations. Tree trimming would become more multifunctional and interesting over time; there would be a community motivated to remain valuable.
🙃
Also I don't think he adequately takes into account that these new models are, according to some, showing emergent properties. He gives it a brief nod here:
It’s easy to attribute intelligence to the new systems; they have a flexibility and unpredictability that we don’t usually associate with computer technology. But this flexibility arises from simple mathematics.
This is how complex systems work - simple components interact to create more complex emergent phenomena which are not predictable purely from looking at the constituent parts. You have to run the model to see what it does. These occur all over nature and the human world, from the properties of materials, evolving organisms, slime moulds, ant colonies, ecosystems, weather and climate, consciousness, language, economies, civilisations ... Complex systems also tend to be extremely sensitive to small external changes, which makes them even more unpredictable.
He throws in the fact that current AI systems have an 'unpredictability' but he never goes back to discuss it properly.
I don't think it adds much to argue about whether AI is 'intelligent' or what 'intelligence' means. Intelligent or not, it's a big new complex system that is apparently showing emergent properties. That makes it more than just a big collaborative mashup. It makes it more than the sum of its parts. It makes it unpredictable and not entirely under rational human control.
That, along with the general collective shittiness of the human species, is what worries me.