• 8 Posts
  • 568 Comments
Joined 10 months ago
cake
Cake day: March 2nd, 2024

help-circle
  • Obviously it depends on your GPU. A crypto mine, you’ll leave it running 24/7. On a recent macbook, an LLM will run at several tokens per second, so yeah for long responses it could take more than a minute. But most people aren’t going to be running such an LLM for hours on end. Even if they do – big deal, it’s a single GPU, that’s negligible compared to running your dishwasher, using your oven, or heating your house.


  • jsomaetoMemesPatience is a virtue
    link
    fedilink
    arrow-up
    2
    arrow-down
    1
    ·
    1 day ago

    Exactly. Talking. Violence isn’t going to make more leftists.

    That said, call me paranoid but I think three-letter organizations are the main obstacle to organizing. I don’t know what to do about that.


  • I don’t have a source for that, but the most that any locally-run program can cost in terms of power is basically the sum of a few things: maxed-out gpu usage, maxed-out cpu usage, maxed-out disk access. GPU is by far the most power-consuming of these things, and modern video games make essentially the most possible use of the GPU that they can get away with.

    Running an LLM locally can at most max out usage of the GPU, putting it in the same ballpark as a video game. Typical usage of an LLM is to run it for a few seconds and then submit another query, so it’s not running 100% of the time during typical usage, unlike a video game (where it remains open and active the whole time, GPU usage dips only when you’re in a menu for instance.)

    Data centers drain lots of power by running a very large number of machines at the same time.









  • It’s obvious that the secular trend in English towards shorter sentences will tend to reduce the frequency of periods, at least in the case of works where commas and periods are rarely used as part of numbers and similar non-phrasal symbols.

    Shouldn’t that be increase?