What are your thoughts on #privacy and #itsecurity regarding the #LocalLLMs you use? They seem to be an alternative to ChatGPT, MS Copilot etc. which basically are creepy privacy black boxes. How can you be sure that local LLMs do not A) “phone home” or B) create a profile on you, C) that their analysis is restricted to the scope of your terminal? As far as I can see #ollama and #lmstudio do not provide privacy statements.

  • toastal
    link
    fedilink
    arrow-up
    1
    ·
    6 hours ago

    Wat. You are saying you can’t package Python application on a system level? That means the language’s package managament is broken. Nix unlike most package managers can do a reasonable job juggling multiple version of packages at the same time & stuff still breaks, & more frequently than anything in any other language other than Haskell.

    There was also the SolarWind attack, Colorama, JarkaStealer, Cobo, pywx, Dropbox, PyTorch 2023. Zero-days galore.

    • lime!@feddit.nu
      link
      fedilink
      English
      arrow-up
      1
      ·
      3 hours ago

      you can, you just need to use the built-in tooling to build a self-contained application like a zipapp. if package maintainers did that it would break less. but they don’t, and the problem there is how big the packaging tooling is.

      like, imagine trying to install a js-based program the same way as you describe, pulling each dependency from apt. it would break immediately.

      also, none of those listed ones used the package index as the vector. solarwind had their own infrastructure compromised, colorama was a typosquatting attack, jarkastealer was malicious from the word go. the list goes on, none of these are packaging system failures.

      he problem npm has had for ages is that people are taking over legitimate packages and adding malicious code into them downstream from development using npm, and that the js ecosystem favours many small dependencies which makes the attack surface huge.

      now, if all of those you listed was due to revival hijacking it would be pypi’s problem, but that only works if the original dev removes their package. as it is, all of it is upstream of pypi. it’s bad opsec by the devs.