What are your thoughts on #privacy and #itsecurity regarding the #LocalLLMs you use? They seem to be an alternative to ChatGPT, MS Copilot etc. which basically are creepy privacy black boxes. How can you be sure that local LLMs do not A) “phone home” or B) create a profile on you, C) that their analysis is restricted to the scope of your terminal? As far as I can see #ollama and #lmstudio do not provide privacy statements.
Meant to be glue but is used in all sorts of places it probably shouldn’t. The way libraries are handled & pinned leads to lots of breakage—a couple applications I have overlays to disable testing since stuff gets merged into Nixpkgs with failing tests so frequently that I is better to just turn it off & deal with failures at runtime.
The ultralytics thing was massive last month https://snyk.io/blog/ultralytics-ai-pwn-request-supply-chain-attack/. These have been coming with regularity—even worse than npm.
I would at least agree Lua is a better place to start—at least for a dynamic scripting language. It is not a complicated language & it even supports tail recursion which you can’t say about far too many languages.
python dependencies, like all scripting language dependencies, must not be installed via the system package manager. yes python’s package management is bad, but if package maintainers for nix are not following best practices then honestly that’s their problem, not the tooling’s. this is python packaging 101.
also, malicious PRs being accepted due to ml people being famously bad at actual software engineering is not a “supply chain attack”. and they are definitely not worse than npm, because the problem wasn’t in pypi. pypi is historically really good at preventing this sort of thing, but what can you do when the actual, well-formed release approved and pushed by the actual maintainers has a cryptominer in it?