Rationalists understand that just because it first appeared in fiction doesn’t mean it can’t happen for real.
As for Roko’s Basilisk, it’s a thought experiment and hardly anyone takes it seriously in its literal form. But y’know Nazism is also a Roko’s Basilisk – they punish you for opposing it in its infancy and reward you if you help it.
Okay, but this can’t happen for real. If you’re rational you should also be aware that things that appear in fiction also and in majority DO NOT happen irl. The possibility of something isn’t the likelihood of something. It’s possible I could die of an aneurysm in the next 15 minutes, not probable though, and I’m not gonna live in fear of it cause it could potentially happen without weighing that potential. Rockin Basslicks is low as hell on the probability scale and an absolute fucking zero on the I’d care if it was real scale. Same with any simulation idea or hell, even the idea of God. I don’t care if I’m in one or if there is one, there is work to be done making this simulation or created universe better for people in it right here and right now and navel gazing about speculations way beyond what we can understand is a selfish and useless waste of time. Fucking feed the poor.
LLMs in their current form are dangerous, but not existential threats. To see the danger of human extinction, multiply some probabilities together:
At some point, AIs will likely become super-intelligent, meaning they are smarter than the entire human race put together. Let’s say there’s a 50% chance of this happening in the next 30 years. Something this smart would indeed be capable of doing something very clever and killing all humans (if it wanted to); there are various theories about how it might go about doing that, and if this is the part that sounds outlandish to you then I can elaborate. Needless to say, if something is extraordinarily smarter than you, it can figure out a way to kill you that you didn’t think of, even if they’re sandboxed in a high-security prison. (Mind you, it will probably be exposed like ChatGPT to the general public, not very likely to be sandboxed if you ask me.)
Okay but surely nobody would build an AI that would want to kill us all right? This is the “alignment problem” – we currently don’t know how to make sure an AI has the same “goals” that its creators want it to have. There’s that meme – an AI tasked with optimizing a silverware production process might end up turning the whole universe and everyone in it into spoons. Because almost nobody is actually taking this problem seriously, I think the first superintelligent AI has an 80% chance to be unaligned.
Would an unaligned AI want to kill us? It might be unaligned but conveniently still value human life. Let’s say be generous and say it’s a 50% chance that the unaligned AI works out in our favour. (It’s probably less likely.)
So that’s a 20% chance in the next 30 years that someone will create an AI which is clever enough to kill us all, and wants to kill us all.
Not true, plenty of people who supported the Nazis early on got shanked. Plenty of SPD members and Prussian aristocrats who jumped ship in 1933 prospered and achieved high rank. Until the Red Army rolled into town, anyway.
Rationalists understand that just because it first appeared in fiction doesn’t mean it can’t happen for real.
As for Roko’s Basilisk, it’s a thought experiment and hardly anyone takes it seriously in its literal form. But y’know Nazism is also a Roko’s Basilisk – they punish you for opposing it in its infancy and reward you if you help it.
Okay, but this can’t happen for real. If you’re rational you should also be aware that things that appear in fiction also and in majority DO NOT happen irl. The possibility of something isn’t the likelihood of something. It’s possible I could die of an aneurysm in the next 15 minutes, not probable though, and I’m not gonna live in fear of it cause it could potentially happen without weighing that potential. Rockin Basslicks is low as hell on the probability scale and an absolute fucking zero on the I’d care if it was real scale. Same with any simulation idea or hell, even the idea of God. I don’t care if I’m in one or if there is one, there is work to be done making this simulation or created universe better for people in it right here and right now and navel gazing about speculations way beyond what we can understand is a selfish and useless waste of time. Fucking feed the poor.
Yeah Roko’s Basilisk is fiction, it’s not worth worrying about. The danger of AI killing us all is real though.
Explain how
LLMs in their current form are dangerous, but not existential threats. To see the danger of human extinction, multiply some probabilities together:
At some point, AIs will likely become super-intelligent, meaning they are smarter than the entire human race put together. Let’s say there’s a 50% chance of this happening in the next 30 years. Something this smart would indeed be capable of doing something very clever and killing all humans (if it wanted to); there are various theories about how it might go about doing that, and if this is the part that sounds outlandish to you then I can elaborate. Needless to say, if something is extraordinarily smarter than you, it can figure out a way to kill you that you didn’t think of, even if they’re sandboxed in a high-security prison. (Mind you, it will probably be exposed like ChatGPT to the general public, not very likely to be sandboxed if you ask me.)
Okay but surely nobody would build an AI that would want to kill us all right? This is the “alignment problem” – we currently don’t know how to make sure an AI has the same “goals” that its creators want it to have. There’s that meme – an AI tasked with optimizing a silverware production process might end up turning the whole universe and everyone in it into spoons. Because almost nobody is actually taking this problem seriously, I think the first superintelligent AI has an 80% chance to be unaligned.
Would an unaligned AI want to kill us? It might be unaligned but conveniently still value human life. Let’s say be generous and say it’s a 50% chance that the unaligned AI works out in our favour. (It’s probably less likely.)
So that’s a 20% chance in the next 30 years that someone will create an AI which is clever enough to kill us all, and wants to kill us all.
Not true, plenty of people who supported the Nazis early on got shanked. Plenty of SPD members and Prussian aristocrats who jumped ship in 1933 prospered and achieved high rank. Until the Red Army rolled into town, anyway.