No matter how advanced or how self aware, AI will lack the ambition that is part of humanity, part of us due to our evolutionary history.
The ambition isn’t the issue. Its a question of power imbalance.
The Paperclip Maximizing Algorithm doesn’t have an innate desire to destroy the world, merely a mandate to turn everything into paperclips. And if the algorithm has enough resources at its disposal, it will pursue this quixotic campaign without regard for any kind of long term sensible result.
The robot dystopia is therefor a corporate dystopia.
There is some argument that one is a consequence of the other. It is, in some sense, the humans who are being programmed to maximize paperclips. The real Roko’s Basilisk isn’t some sinister robot brain, but a social mythology that leads us to work in the factors that make the paper clips, because we’ve convinced ourselves this will allow us to climb the Paperclip Company Corporate Ladder until we don’t have to make these damned things anymore.
Thats the beauty of it. The maximizer would understand that creating a machine that breaks appart worlds would maximize the paperclip output. It will be a “natural” progression
The ambition isn’t the issue. Its a question of power imbalance.
The Paperclip Maximizing Algorithm doesn’t have an innate desire to destroy the world, merely a mandate to turn everything into paperclips. And if the algorithm has enough resources at its disposal, it will pursue this quixotic campaign without regard for any kind of long term sensible result.
There is some argument that one is a consequence of the other. It is, in some sense, the humans who are being programmed to maximize paperclips. The real Roko’s Basilisk isn’t some sinister robot brain, but a social mythology that leads us to work in the factors that make the paper clips, because we’ve convinced ourselves this will allow us to climb the Paperclip Company Corporate Ladder until we don’t have to make these damned things anymore.
Someone screwed up if a paperclip maximiser is given the equipment to take apart worlds, rather than a supply of spring steel
Thats the beauty of it. The maximizer would understand that creating a machine that breaks appart worlds would maximize the paperclip output. It will be a “natural” progression