https://www.theverge.com/2023/7/10/23790132/google-memo-moat-ai-leak-demis-hassabis

In this link this discussion:

The memo, which was obtained by SemiAnalysis from a public Discord server, says that neither Google nor OpenAI have what they need to succeed in the AI industry. Instead, the researcher claims “a third faction has been quietly eating our lunch”: open-source AI models that the researcher says are “faster, more customizable, more private, and pound-for-pound more capable.”

WHO IS THE THIRD FACTION?

  • navigatron@beehaw.org
    link
    fedilink
    arrow-up
    20
    ·
    1 year ago

    Go a level deeper, beyond this news about news, and read the moat memo.

    The third faction is the open source community.

    The memo has an entire timeline section, dedicated to showing the speed at which the open source community absorbed and iterated on the leaked facebook model, LaMMa.

    The memo puts a lot of emphasis on how google and co are building new models from scratch, over months, with millions of dollars - and yet open source is building patches, in days, with only a few hundred dollars - and the patches stack, and are easily shareable.

    The open source models, through these patches, are getting better faster than google can re-architect and re-train new models from scratch.

    The main point of the memo is that google needs to change their strategy, if they want to stay “ahead” (some would argue they’re already behind) of the competition.

  • j4k3@lemmy.world
    link
    fedilink
    arrow-up
    19
    ·
    edit-2
    1 year ago

    The open source researchers that took the meta weights and ran with them, and then superseded them, was how I read it. It was the overall innovations that made patching as effective if not more than recompiling from scratch.

    I am curious how long this memo lasts as a real leak. Like I haven’t tried digging below the surface of information as presented, but most of this is indistinguishable from magic in actual practice. Like even open source, does not mean I have a chance in hell of compiling on my own, and modifying is completely absurd at this point. I wouldn’t mind learning, but I know I’m out of my depth on that one. Could this leak be a false flag or public passivation for political reasons? Making the publicly available options sound advanced and capable takes a lot of pressure off of proprietary efforts right as they are hitting a larger public focus.

    • sachasage@lemmy.world
      link
      fedilink
      arrow-up
      9
      ·
      1 year ago

      I believe the memo is saying meta will benefit in the end because they can utilise community innovation already built in their own architecture

  • Zeth0s@lemmy.world
    link
    fedilink
    arrow-up
    8
    ·
    1 year ago

    If you read the memo they talk about meta as possible benefiter of open source advances

  • dr_catman@beehaw.org
    link
    fedilink
    arrow-up
    4
    arrow-down
    1
    ·
    edit-2
    1 year ago

    huggingface is full of models not created by a large corporation. Maybe they mean those?

      • dr_catman@beehaw.org
        link
        fedilink
        arrow-up
        3
        ·
        1 year ago

        Yes! …well, with a caveat.

        Huggingface models (even the ones published to the platform by Google or fb) are available for anyone to use, but they’re not quite intended to be chatbots; they’re more like pretrained machine learning models for data science pipelines.

        Still, if you know python or are willing to learn, you can use huggingface however you want. You’re totally allowed to download a model, open it in python, and ask questions to it. Or feed it a long text and ask for summaries etc.

        • Obi@sopuli.xyz
          link
          fedilink
          arrow-up
          1
          ·
          1 year ago

          Yeah no I guess I’m more looking for the open-source end-products I could use similarly to chatGPT or midjourney etc. I did learn a bit of python a while back but I’m no coder.

      • philoko
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Yeah, Oobabooga let’s you easily download models from HuggingFace and has a chat interface.

  • nonearther
    link
    fedilink
    arrow-up
    5
    arrow-down
    2
    ·
    1 year ago

    A wild guess, but could it be Microsoft?

    I mean they are already creating use cases and integrating them in their apps, especially their business suite.

    While Google and OpenAI are ahead in AI innovation, I feel Microsoft is getting ahead in implementation and actual end user scenarios.

    Microsoft office offerings are still much larger than Google’s, plus they capture Dev’s needs with GitHub CoPilot and VS/ VSCode integrations.

    I feel like by the time Google will be ready or OpenAI’s contract with MS ends, both of these companies will lag sufficiently behind MS in real world.

    • Zeth0s@lemmy.world
      link
      fedilink
      arrow-up
      7
      arrow-down
      1
      ·
      edit-2
      1 year ago

      Microsoft currently lack the internal know how on AI. It is behind. They are the best understanding the average user/company needs, but they currently depend on openai. Either the buy openai (risking to destroy it because Microsoft is relatively bad at real research and cutting edge engineering) or they need to invest a lot more than Google.

      If you read the memo, the 3rd player is meta. Microsoft is not even considered.

      Microsoft will for sure make more money than anyone else, because that’s their job, making more money than anyone. They are a bad tech company, but a great money printer company

  • LachlanUnchained@lemmyunchained.net
    link
    fedilink
    arrow-up
    4
    arrow-down
    1
    ·
    edit-2
    1 year ago

    Meta. They are smashing it with small customisable, specialised models.

    Listen to the latest lex fridman podcast with zuck. I found it quite fascinating.

    And then there’s worrying about not wearing the same thing twice. My god! Don’t know how you do it.

  • BananaTrifleViolin@kbin.social
    link
    fedilink
    arrow-up
    1
    ·
    edit-2
    1 year ago

    Instead, the researcher claims “a third faction has been quietly eating our lunch”: open-source AI models that the researcher says are “faster, more customizable, more private, and pound-for-pound more capable.”

    The sentence you quote says who the 3rd faction is: Open-source AI models.