Kenn White (@kennwhite@mastodon.social)
mastodon.social
Attached: 3 images Incredible research at BlackHat Asia today by Tong Liu and team from the Institute of Information Engineering, Chinese Academy of Sciences (在iie.ac.cn 的电子邮件经过验证) A dozen+ RCEs on popular LLM framework libraries like LangChain and LlamaIndex - used in lots of chat-assisted apps including GitHub. These guys got a reverse shell in two prompts, and even managed to exploit SetUID for full root on the underlying VM!
  • self@awful.systemsEnglish
    13·
    7 months ago

    all of the developers I know at AI-related startups identify as researchers, regardless of their actual role

    the underlying hardware, OS, VMs etc.

    no, let’s not blame unaffiliated systems engineers for this dumb shit, thanks

    • cm0002@lemmy.worldEnglish
      67·
      7 months ago

      no, let’s not blame unaffiliated systems engineers for this dumb shit, thanks

      Oh, yea sorry I forgot AI models actually run in a vacuum and needs no supporting code or infrastructure to make it usable to the average user so it doesn’t even need non-AI best security practices! Process isolation? OS hardening? Pfft who needs it

      • flere-imsaho@awful.systemsEnglish
        10·
        7 months ago

        i wouldn’t touch the llm stuff with a barge pole unless i was expressly told to do so, and if i’ve been told to do it, i’d look for another employer (which i’m currently doing, for tangentially-related reasons).

        and it’s not that i don’t care about the llms. i do care very much about them all ending in fiery pit of the deepest of hells.