First, let me say that what broke me from the herd at lesswrong was specifically the calls for AI pauses. That somehow ‘rationalists’ are so certain advanced AI will kill everyone in the future (pDoom = 100%!) that they need to commit any violent act needed to stop AI from being developed.

The flaw here is that there’s 8 billion people alive right now, and we don’t actually know what the future is. There are ways better AI could help the people living now, possibly saving their lives, and essentially eliezer yudkowsky is saying “fuck em”. This could only be worth it if you actually somehow knew trillions of people were going to exist, had a low future discount rate, and so on. This seems deeply flawed, and seems to be one of the points here.

But I do think advanced AI is possible. And while it may not be a mainstream take yet, it seems like the problems current AI can’t solve, like robotics, continuous learning, module reuse - the things needed to reach a general level of capabilities and for AI to do many but not all human jobs - are near future. I can link deepmind papers with all of these, published in 2022 or 2023.

And if AI can be general and control robots, and since making robots is a task human technicians and other workers can do, this does mean a form of Singularity is possible. Maybe not the breathless utopia by Ray Kurzweil but a fuckton of robots.

So I was wondering what the people here generally think. There are “boomer” forums I know of where they also generally deny AI is possible anytime soon, claim GPT-n is a stochastic parrot, and make fun of tech bros as being hypesters who collect 300k to edit javascript and drive Teslas*.

I also have noticed that the whole rationalist schtick of “what is your probability” seems like asking for “joint probabilities”, aka smoke a joint and give a probability.

Here’s my questions:

  1. Before 2030, do you consider it more likely than not that current AI techniques will scale to human level in at least 25% of the domains that humans can do, to average human level.

  2. Do you consider it likely, before 2040, those domains will include robotics

  3. If AI systems can control robotics, do you believe a form of Singularity will happen. This means hard exponential growth of the number of robots, scaling past all industry on earth today by at least 1 order of magnitude, and off planet mining soon to follow. It does not necessarily mean anything else.

  4. Do you think that mass transition where most human jobs we have now will become replaced by AI systems before 2040 will happen

  5. Is AI system design an issue. I hate to say “alignment”, because I think that’s hopeless wankery by non software engineers, but given these will be robotic controlling advanced decision-making systems, will it require lots of methodical engineering by skilled engineers, with serious negative consequences when the work is sloppy?

*“epistemic status”: I uh do work for a tech company, my job title is machine learning engineer, my girlfriend is much younger than me and sometimes fucks other dudes, and we have 2 Teslas…

  • Evinceo@awful.systems
    link
    fedilink
    English
    arrow-up
    8
    ·
    edit-2
    1 year ago

    Content Warning: Ratspeak

    spoiler

    Let’s say that tomorrow, they build AGI on HP/Cray Frontier. It’s human equivalent. Mr Frontier is rampant or whatever and wants to improve himself. In order to improve himself he will need to create better chips. He will need approximately 73 thousand copies of himself just to match the staff of TSMC, but there’s only one Frontier. And that’s to say nothing of the specialized knowledge and equipment required to build a modern fab, or the difficulty of keeping 73 thousand copies of himself loyal to his cause. That’s just to make a marginal improvement on himself, and assuming everyone is totally ok with letting the rampant AI get whatever it wants. And that’s just the ‘make itself smarter’ part, which everything else is contingent on; it assumes that we’ve solved Moravec’s paradox and all of the attendant issues of building robots capable of operating at the extremes of human adaptability, which we have not. Oh and it’s only making itself smarter at the same pace TSMC already was.

    The practicalities of improving technology are generally skated over by aingularatians in favor of imagining technology as a magic number that you can just throw “intelligence” at to make it go up.

    • self@awful.systemsM
      link
      fedilink
      English
      arrow-up
      17
      ·
      1 year ago

      What I’m trying to get at is that the practicalities of improving technology are generally skated over by aingularatians in favor of imagining technology as a magic number that you can just throw “intelligence” at to make it go up.

      this is where the singularity always lost me. like, imagine, you build an AI and it maxes out the compute in its server farm (a known and extremely easy to calculate quantity) so it decides to spread onto the internet where it’ll have infinite compute! well congrats, now the AI is extremely slow cause the actual internet isn’t magic, it’s a network where latency and reliability are gigantic issues, and there isn’t really any way for an AI to work around that. so singulatarians just handwave it away

      or like when they reach for nanomachines as a “scientific” reason why the AI would be able to exert godlike influence on the real world. but nanomachines don’t work like that at all, it’s just a lazy soft sci-fi idea that gets taken way too seriously by folks who are mediocre at best at understanding science

      • swlabr@awful.systems
        link
        fedilink
        English
        arrow-up
        15
        ·
        1 year ago

        (To be read in the voice of an elementary schooler who is a sore loser at make believe): Nuh-uh! My AGI has quantum computers, so it doesn’t get slow from the internet, and, and, and, it builds robots, with jetpacks, and those robots have tiny robots that can go in your brain and and and make your brain explode, and if you say anything mean about me or the AGI it’ll take your brain and clone it and put wires in it and make you think youre getting like, wedgied and stuff, but really youre not but you think you are because it’s really good at making you think it

                  • froztbyte@awful.systems
                    link
                    fedilink
                    English
                    arrow-up
                    9
                    ·
                    1 year ago

                    slight queasiness like a reverse body high, tinged with experience of time distortion (“the fuck? since 2000?! WHY”) and dissociation

                    some memory loss about the experience as well, with any recall about the details of the visions entirely vacant from my mind. it’s like seeing the machine elves but my brain didn’t want to remember

      • Evinceo@awful.systems
        link
        fedilink
        English
        arrow-up
        8
        ·
        1 year ago

        Indeed, if distributed computing worked as well as singulatarians fear everyone would be using Beowulf clusters for their workloads instead of AWS.

        • froztbyte@awful.systems
          link
          fedilink
          English
          arrow-up
          5
          ·
          1 year ago

          Can I live in this world? Please? Pretty please with a cherry on top?

          It sounds so much less frustrating than this pile of mistakes with Pike’s shitty ideas at every fucking api and datamodel

      • Steve@awful.systems
        link
        fedilink
        English
        arrow-up
        7
        ·
        1 year ago

        but nanomachines don’t work like that at all, it’s just a lazy soft sci-fi idea that gets taken way too seriously by folks who are mediocre at best at understanding science

        Let’s call this Crichtonitis.

        • Steve@awful.systems
          link
          fedilink
          English
          arrow-up
          7
          ·
          1 year ago

          not a joke btw. literal plot of Prey which he followed with his climate change denial book State of Fear

      • BrickedKeyboard@awful.systemsOP
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        2
        ·
        1 year ago

        Serious answer not from yudnowsky: the AI doesn’t do any of that. It helps people cheat on their homework, write their code and form letters faster, and brings in revenue. AI owner uses the revenue and buys gpus. With the GPUs they make the AI better. Now it can do a bit more than before and then they buy more GPUs and theoretically this continues until the list of tasks the AI can do includes “most of the labor in a chip fab” and GPUs become cheap and then things start to get crazy.

        Same elementary school logic but I mean this is how a nuke works.

        • self@awful.systemsM
          link
          fedilink
          English
          arrow-up
          15
          ·
          1 year ago

          wait, so the AI is just your fears about capitalism?

          Same elementary school logic but I mean this is how a nuke works.

          what. no it isn’t

        • skillissuer@discuss.tchncs.de
          link
          fedilink
          English
          arrow-up
          4
          ·
          1 year ago

          your imaginary nukes explode forever. in reality, nuke stops exploding when it either runs out of plutonium or is dispersed too much. the energy of nuke is not infinite, it’s large, but most importantly it’s all contained in the device from the beginning

          your example also fails at the step of “getting more money forever”, when VC funding runs out or gets dispersed too much entire charade grinds to halt (because SV startups are shielded from commercial failure by that VC money). that state is sometimes called “AI winter”

          • BrickedKeyboard@awful.systemsOP
            link
            fedilink
            English
            arrow-up
            1
            arrow-down
            2
            ·
            edit-2
            1 year ago

            Did this happen with Amazon? The VC money is a catalyst. It’s advancing money for a share of future revenues. If AI companies can establish a genuine business that collects revenue from customers they can reinvest some of that money into improving the model and so on.

            OpenAI specifically seems to have needed about 5 months to go to 1 billion USD annual revenue, or the way tech companies are valued, it’s already worth more than 10 billion intrinsic value.

            If they can’t - if the AI models remain too stupid to pay for, then obviously there will be another AI winter.

            https://fortune.com/2023/08/30/chatgpt-creator-openai-earnings-80-million-a-month-1-billion-annual-revenue-540-million-loss-sam-altman/

            • skillissuer@discuss.tchncs.de
              link
              fedilink
              English
              arrow-up
              4
              ·
              edit-2
              1 year ago

              from what i understand openai runs all of their products at loss, so when vc money runs out things could get interesting

              and with recession coming, there could be less vc money to begin with

      • BrickedKeyboard@awful.systemsOP
        link
        fedilink
        English
        arrow-up
        1
        arrow-down
        2
        ·
        1 year ago

        I agree completely. This is exactly where I break with Eliezer’s model. Yes obviously an AI system that can self improve can only do so until it’s either (1) the best algorithm that can run on the server farm (2) finding a better algorithm takes more compute than is worth the investment in current compute

        That’s not a god. You do this in an AI experiment now and it might crap out at double or less the starting performance and not even be above the SOTA.

        But if robots can build robots, and the current AI progress shows a way to do it (foundation model on human tool manipulation), then…

        Genuinely asking, I don’t think it’s “religion” to suggest that a huge speedup in global GDP would be a dramatic event.