Software engineer (video games). Likes dogs, DJing + EDM, running, electronics and loud bangs in Reservoir.

  • 18 Posts
  • 466 Comments
Joined 3 years ago
cake
Cake day: June 11th, 2023

help-circle

  • Strongly recommend looking into Proxmox hypervisor as your base OS. Then install as many Linux VMs/LXCs as you like, with backups/snapshots all taken care for you. I’m on an Intel 8th gen CPU and have 3-4 VMs running, including Jellyfin, Home Assistant OS, an assortment of random docker images and a dedicated headless Debian VM for various AI projects. The CPU is more than enough, your 12th gen target will breeze through it. Get more RAM than you think you need - it will become the bottleneck for what you can do, not your CPU. I’m running 64GB and don’t regret it!

    For video encoding/decoding like Jellyfin/Frigate, make sure you stick with Intel CPU for hardware codec support via QuickSync, otherwise you’ll need to add a GPU.











  • LLM ASICs are different to NPUs. The idea is to encode a specific model’s weights (all billions of them) as a gigantic resistor network on the chip itself. Analog, where ohms law becomes the processor, and no RAM is necessary - the chip IS the model. It would pave the way for cost effective, fast, low-power local inference. Given the analog circuitry, there would be issues like interference, temperature sensitivity and so on, but we’re seeing models get good results even when heavily quantised, so the idea has legs. The big disadvantage is you’re stuck with that one model forever, so we’re likely to see these first appear in non-consumer applications like automotive, defence etc. A pretty exciting concept that solves some of the environmental issues around AI.

    More info for the curious: https://taalas.com/the-path-to-ubiquitous-ai/


  • I’m really interested to see what comes of the LLM ASIC idea - it worked wonders for hardware video codecs. I could see the smartphone industry picking up again on annual releases worthy of upgrades if they included dedicated AI chips not beholden to a cloud subscription. I think we’re reaching the point that the frontier models are becoming “good enough” to use for a while before uploading to the next.






  • We created an AI usage code of conduct at our workplace. One of the golden rules is “Don’t make other people’s jobs harder.”

    As an example, we tried getting AI to generate weekly bug reports from analysing the most common crashes and asserts for the week, assigned to the right person with relevant context. It was a great idea, but too often the bug description and even the title required way too much jargon and comprehension to get through, and often the findings weren’t quite right anyway, so it was only useful as a marker of “look at this crash please”.

    These walls of text people share with each other are another great example. If you’re going to do that, you really have to ask it to talk as simply and as briefly as possible with no context assumed. And then read it/correct it yourself!