I’m looking at starting a service that involves hosting a lot of LLM models, which are often going to be 16GB+ (compressed). I did a bit of searching for cloud storage providers with cheap egress, and the cheapest I could find is $0.01 per GB, which would still be $0.16+ per download.

How do sites like Huggingface or CivitAI do it? Lots of VC funding?

  • minorninth@lemmy.worldEnglish
    3·
    2 years ago

    Would BitTorrent be an option?

    There are also ad-supported download sites.

    You could offer users a choice: BitTorrent, ads, or $1 for an instant download.

    • omegastickOPEnglish
      1·
      2 years ago

      We’d be offering a service training models for users, so I don’t see bittorrent working. Each file would be unique to the user and probably only downloaded once. We might be able to give free downloads of a LoRA (kind of a very compressed “diff” of the training run), along with tools to download and merge it with the base model from Huggingface, then charge for a download of the full model maybe.