Can you explain how Singer is right-wing? I never really thought about whether he was left or right wing before, since I’m not sure it’s relevant to his most famous philosophical arguments. But I see in 1999 he wrote a book about how evolutionary psychology should be incorporated into left-leaning politics, indicating he at least self-identifies as left or did at the time.
Sounds like we basically agree. I’d be interested to see what alt-right views Scott Alexander agrees with. I’ve perused r/sneerclub but they seem kinda kooky when it comes to SSC. Like, for instance, I know he is in favour of “”“eugenics”“” but his version of that is totally different from the Nazi version to the point where I just roll my eyes when people make that comparison. It’s hard for me to separate the actually valid criticism of him from the people who can’t get past stuff like that.
(Edit: just to be 100% clear to people: it’s not master race stuff, it’s more like optimizing alleles in-vitro and at the most extreme end he might be in favour of persuading meth addicts not to have children unintentionally if they are at risk of certain serious drug-induced diseases (consensually). I’m on the fence about it. It’s very Peter Singer.)
idk it just was phrased in a disagree-y kinda way. Thought maybe they’d misread.
On each of your paragraphs:
I think we completely agree – there can be both a 20% threat of extinction and also the threat of climate change
no, I don’t agree with this; that’s like saying the threat of the asteroid is used to supplant the threat of climate change. The X-risk threat of AI does not invalidate the other threats of AI, and I disagree with anyone who thinks it does. I have not seen anyone use the X-risk threat of AI to invalidate the other threats of AI, and I implore you to not let such an argument sway you for or against either of those threats, which are both real (!).
I do not blame the gun, I blame the manufacturer. I am calling for more oversight over AI companies and for people who research AI to take this threat more seriously. If an AI apocalypse happens, it will of course be the fault of the idiotic AI development companies who did not take this threat seriously because they were blinded by profits. What did I say that made you think I was blaming the AI itself?
That’s true. So I think we need to come up with cleverer ways to make people see Musk as a threat. Just because he is a Nazi doesn’t mean pointing at him and crying wolf is the best way to fight.
Are you… disagreeing with me? Or are you reaffirming what I’m saying? I am not sure I understand the point of this comment.
Yes that’s right (that it’s only reliable as the prior probabilities that go into it).
Look at this another way, using the perspective you just shared: before applying bayesian reasoning, one might think that AI as an X-risk sounds super fantastical, and assign it ultra-low probability. But when you break it into constituent components like I did, it starts to sound much more plausible. We’re replacing how one feels intuitively about a certain (improbable-seeming) event with how one feels intuitively about other (more plausible) events. That isn’t a fallacy, that’s actually good off-the-cuff reasoning. Now we can look at whichever of those sounds the most implausible and break it down further.
My goal here isn’t to actually find the exact probability of an AI apocalypse, it’s to raise a warning flag that says “hey, this is more plausible than you might initially think!”
Those are good points, I’ll take a look at the resources you suggested. I think my counter-argument to you right now can basiclaly be summed up as: I do agree that the danger of AI you are talking about is serious and is the more current and pressing concern, but that doesn’t really invalidate the X-risk factor of AI. I am not saying that X-risk is the only risk, and your point warning about a “hero” (which I agree with!) also doesn’t invalidate the concern. I mean, if it turns out that only a heroic space agency can save us from that asteroid, does that mean the threat from the asteroid isn’t real?
I’m more convinced by SSC/ACX’s arguments about AI doom. I agree that Yudkowsky is weird, and I don’t like his political trend in recent years either – though I wouldn’t consider ACX a reactionary just because he tolerates fascists leaving comments, but that’s really not germane. Anyway, it sounds like you think Yudkowsky’s perspective on AI doom is not entirely invalid, so that sounds like you also believe there is a chance that AI could be an x-risk?
I do pride myself on being a silly person :)
Oops my bad.
I think we have a semantics disagreement over what “intelligence” means so let’s just put that word away. Machines can already solve problems that are out of human reach (this has been known for a very long time – take the 4-colour theorem for instance, which I think is a pretty clear example of a machine outputting something that wasn’t input). It doesn’t really matter if an AI is “intelligent” if it is capable of writing code to advance a goal that it was given or analyzing a situation and producing a good recommendation about how to proceed in order to advance a goal that was given. LLMs are already capable of this, and if we imagine in 10 years AIs might be 100x better at it, that is already too scary for me.
By the way, I don’t really know what I did to deserve that hostility.
Probabilities do work like that, if you believe in Bayesian probability anyway. The more we learn about the situation more accurate a probability we can get. The Drake equation works exactly in this way.
It may be true that the discourse is promoted by people who want more funding for AI, but that does not invalidate the point. I was on board with this concern since around 2016 or so, long before LLMs, and I don’t have a vested interest in AI. And there is overlap between commonplace concerns about AI and existential ones – for instance, a moratorium would advance both goals. Frankly, if people see AI as an existential threat, that should be a great boon for other anti-AI parties, no?
Global warming made sense to me when I was 8 too, but it’s a common talking point among conservatives that it’s ludicrous to suggest that humans could have an impact on something as large as the planet as a whole.
LLMs in their current form are dangerous to society, but not existential threats
I’m assuming you think LLMs are not dangerous to society? But they’re already putting people out of work. And they can cause havoc on social media. And they just piss me off.
LLMs in their current form are dangerous to society, but not existential threats. To see the danger of human extinction, multiply some probabilities together:
At some point, AIs will likely become super-intelligent, meaning they are smarter than the entire human race put together. Let’s say there’s a 50% chance of this happening in the next 30 years. Something this smart would indeed be capable of doing something very clever and killing all humans (if it wanted to); there are various theories about how it might go about doing that, and if this is the part that sounds outlandish to you then I can elaborate. Needless to say, if something is extraordinarily smarter than you, it can figure out a way to kill you that you didn’t think of, even if they’re sandboxed in a high-security prison. (Mind you, it will probably be exposed like ChatGPT to the general public, not very likely to be sandboxed if you ask me.)
Okay but surely nobody would build an AI that would want to kill us all right? This is the “alignment problem” – we currently don’t know how to make sure an AI has the same “goals” that its creators want it to have. There’s that meme – an AI tasked with optimizing a silverware production process might end up turning the whole universe and everyone in it into spoons. Because almost nobody is actually taking this problem seriously, I think the first superintelligent AI has an 80% chance to be unaligned.
Would an unaligned AI want to kill us? It might be unaligned but conveniently still value human life. Let’s say be generous and say it’s a 50% chance that the unaligned AI works out in our favour. (It’s probably less likely.)
So that’s a 20% chance in the next 30 years that someone will create an AI which is clever enough to kill us all, and (by unhappy accident) wants to kill us all. You can put your own numbers on all of these things and get your own probability. If you put very low numbers (like 0.1%) on any of these steps, you should ask yourself if in 2010 you thought it was likely that AI would be where it is today.
Edit: yes I know it sounds absurd and like fantasy, but a lot of real things sound absurd at first. One of the most pervasive arguments against global warming is that it sounds absurd. So if you’re going to disagree with this, please at least have a better reason than “it sounds absurd.”
I agree with you but apparently I said it more elegantly and got 33 upvotes instead of 76 downvotes. I guess what you’re not mentioning is that the real neo-nazis really are here this time.
Yeah Roko’s Basilisk is fiction, it’s not worth worrying about. The danger of AI killing us all is real though.
My absolute favourite thing is women doing things that “only men do.”
(P.S. am a woman)
I was taught when I was young that you don’t call something that isn’t really genocide genocide, someone that isn’t a nazi a nazi, etc. because it dilutes the meaning of the word. I do think that Israel is committing a genocide, and I do think that Musk is a neo-nazi; I’m not diluting these terms to say this.
This makes me feel sad about my job, but i upvote anyway