Another user already touched the Bayesian point, so I’m not going to follow that rabbit.
I was on board with this concern since around 2016 or so, long before LLMs, and I don’t have a vested interest in AI.
Ok? AI will become Skynet is such a popular idea that has permeated society since before I was even born, and I’m guessing before you were, or at least was something you were exposed to in your early years. It’s frankly not an original thought you came up with in 2016, but rather something you and everyone else has inherited from popular media. Saying we need to slow research on AI to align it with “human values” still allows for this idea that we can control AI to not kill us. Moreover, it allows for the idea that only large companies can align the AI to human values, and the “human values” they are currently aligning it with have nothing to do with saving humanity. Instead, the human values are to reinforce dominant classes in society, accelerate climate change through forcing scale as the only path forward (at least until deepseek dropped), and spark mass layoffs as white collar work is automated away.
We’re not going to create a paper clip machine that kills us all because it wants to simply make paper clips. We’re going to make a sophisticated bullshit generator whose primary role is to replace labor. Hopefully, I don’t need to spell out what this means in a capitalist society which is currently free falling into fascism. We’re reaching a point where LLMs have slightly preferable error rates at scale than human workers, and that’s the real danger here.
a moratorium would advance both goals. Frankly, if people see AI as an existential threat, that should be a great boon for other anti-AI parties, no?
I’m all for a Butlerian jihad, mount up. I’m not going to join you for a Yudkowskian Jihad, though.
In my view, the danger remains that if the only concern being talked about is AI will kill us all in some fantastical war or apocalyptic scenario, it creates a “hero” (i.e. Sam Altman or some other ghoul) who alone can fix it. The apocalypse argument is not currently pushing anyone towards any moratorium on AI development, but rather just creating a subfield of “alignment” which is more concerned with making sure LLMs don’t say mean things, follow the narrative, and don’t suggest people use irons to smooth out the wrinkles in their balls.
Global warming made sense to me when I was 8 too, but it’s a common talking point among conservatives that it’s ludicrous to suggest that humans could have an impact on something as large as the planet as a whole.
This part is tangential, but it actually helps as an allegory to this issue. Exxon new in the late 70s the effects their production would have, that climate change was due to our use of fossil fuels. Rather than act accordingly and pivot away, they protected their profits and muddied the waters by bringing these talking points to media and conservative outlets. Conservatives didn’t organically think this is ridiculous, they were told it was absurd by media empires, and they ate it up and spread it.
I get the feeling you are here in good faith, so if you want to read more about the very real, current, actually happening dangers of AI, I would point you to Atlas of AI, Resisting AI, and the work of Bender and Gebru.
Those are good points, I’ll take a look at the resources you suggested. I think my counter-argument to you right now can basiclaly be summed up as: I do agree that the danger of AI you are talking about is serious and is the more current and pressing concern, but that doesn’t really invalidate the X-risk factor of AI. I am not saying that X-risk is the only risk, and your point warning about a “hero” (which I agree with!) also doesn’t invalidate the concern. I mean, if it turns out that only a heroic space agency can save us from that asteroid, does that mean the threat from the asteroid isn’t real?
Following the asteroid analogy, I view it as this: If there’s a 20% chance that an asteroid could hit us in 2050, does that supplant the threat of climate change today?
I’m not trying to say that AI systems won’t kill us all, just that they are using to directly harm entire populations right now and the appeal to a future danger is being used to minimize that discussion.
Another thing to consider: If an AI system does kill us all, it will still be a human or organization that gave it the ability to do so, whether that be through training practices, or plugging it in to weapons systems. Placing the blame on the AI itself absolves any person or organization of the responsibility, which is in line with how AI is used today (i.e. the promise of algorithmic ‘neutrality’). Put another way, do the bombs kill us all in a nuclear armageddon or do the people who pressed the button? Does the gun kill me, or does the person pulling the trigger?
I think we completely agree – there can be both a 20% threat of extinction and also the threat of climate change
no, I don’t agree with this; that’s like saying the threat of the asteroid is used to supplant the threat of climate change. The X-risk threat of AI does not invalidate the other threats of AI, and I disagree with anyone who thinks it does. I have not seen anyone use the X-risk threat of AI to invalidate the other threats of AI, and I implore you to not let such an argument sway you for or against either of those threats, which are both real (!).
I do not blame the gun, I blame the manufacturer. I am calling for more oversight over AI companies and for people who research AI to take this threat more seriously. If an AI apocalypse happens, it will of course be the fault of the idiotic AI development companies who did not take this threat seriously because they were blinded by profits. What did I say that made you think I was blaming the AI itself?
Another user already touched the Bayesian point, so I’m not going to follow that rabbit.
Ok? AI will become Skynet is such a popular idea that has permeated society since before I was even born, and I’m guessing before you were, or at least was something you were exposed to in your early years. It’s frankly not an original thought you came up with in 2016, but rather something you and everyone else has inherited from popular media. Saying we need to slow research on AI to align it with “human values” still allows for this idea that we can control AI to not kill us. Moreover, it allows for the idea that only large companies can align the AI to human values, and the “human values” they are currently aligning it with have nothing to do with saving humanity. Instead, the human values are to reinforce dominant classes in society, accelerate climate change through forcing scale as the only path forward (at least until deepseek dropped), and spark mass layoffs as white collar work is automated away.
We’re not going to create a paper clip machine that kills us all because it wants to simply make paper clips. We’re going to make a sophisticated bullshit generator whose primary role is to replace labor. Hopefully, I don’t need to spell out what this means in a capitalist society which is currently free falling into fascism. We’re reaching a point where LLMs have slightly preferable error rates at scale than human workers, and that’s the real danger here.
I’m all for a Butlerian jihad, mount up. I’m not going to join you for a Yudkowskian Jihad, though.
In my view, the danger remains that if the only concern being talked about is AI will kill us all in some fantastical war or apocalyptic scenario, it creates a “hero” (i.e. Sam Altman or some other ghoul) who alone can fix it. The apocalypse argument is not currently pushing anyone towards any moratorium on AI development, but rather just creating a subfield of “alignment” which is more concerned with making sure LLMs don’t say mean things, follow the narrative, and don’t suggest people use irons to smooth out the wrinkles in their balls.
This part is tangential, but it actually helps as an allegory to this issue. Exxon new in the late 70s the effects their production would have, that climate change was due to our use of fossil fuels. Rather than act accordingly and pivot away, they protected their profits and muddied the waters by bringing these talking points to media and conservative outlets. Conservatives didn’t organically think this is ridiculous, they were told it was absurd by media empires, and they ate it up and spread it.
I get the feeling you are here in good faith, so if you want to read more about the very real, current, actually happening dangers of AI, I would point you to Atlas of AI, Resisting AI, and the work of Bender and Gebru.
Those are good points, I’ll take a look at the resources you suggested. I think my counter-argument to you right now can basiclaly be summed up as: I do agree that the danger of AI you are talking about is serious and is the more current and pressing concern, but that doesn’t really invalidate the X-risk factor of AI. I am not saying that X-risk is the only risk, and your point warning about a “hero” (which I agree with!) also doesn’t invalidate the concern. I mean, if it turns out that only a heroic space agency can save us from that asteroid, does that mean the threat from the asteroid isn’t real?
Following the asteroid analogy, I view it as this: If there’s a 20% chance that an asteroid could hit us in 2050, does that supplant the threat of climate change today?
I’m not trying to say that AI systems won’t kill us all, just that they are using to directly harm entire populations right now and the appeal to a future danger is being used to minimize that discussion.
Another thing to consider: If an AI system does kill us all, it will still be a human or organization that gave it the ability to do so, whether that be through training practices, or plugging it in to weapons systems. Placing the blame on the AI itself absolves any person or organization of the responsibility, which is in line with how AI is used today (i.e. the promise of algorithmic ‘neutrality’). Put another way, do the bombs kill us all in a nuclear armageddon or do the people who pressed the button? Does the gun kill me, or does the person pulling the trigger?
On each of your paragraphs:
I think we completely agree – there can be both a 20% threat of extinction and also the threat of climate change
no, I don’t agree with this; that’s like saying the threat of the asteroid is used to supplant the threat of climate change. The X-risk threat of AI does not invalidate the other threats of AI, and I disagree with anyone who thinks it does. I have not seen anyone use the X-risk threat of AI to invalidate the other threats of AI, and I implore you to not let such an argument sway you for or against either of those threats, which are both real (!).
I do not blame the gun, I blame the manufacturer. I am calling for more oversight over AI companies and for people who research AI to take this threat more seriously. If an AI apocalypse happens, it will of course be the fault of the idiotic AI development companies who did not take this threat seriously because they were blinded by profits. What did I say that made you think I was blaming the AI itself?