Hello !piracy@lemmy.dbzer0.com

I intend to upload an archive of a website I scraped for the last few weeks. The torrent is about 4 TB large and contains 400000 folders which in turn contain 40 .jpg’s on average (plus some metadata).

Should I just create the torrent with the files as they are right now, or should I put the individual folders in archives (or maybe even the entire torrent?)?

Thanks in advance

  • cesium@sh.itjust.worksOPEnglish
    4·
    1 year ago

    Makes sense. What about individual folders? I’m concerned that the massive amount of small files could slow down the download quite a bit. Unfortunately, I’m not too familiar with the BitTorrent protocol, so I don’t know if this would have an impact.

    • zodo123@lemmy.worldEnglish
      13·
      1 year ago

      Small files don’t impact BitTorrent transfers. They’re sent in pieces of a fixed size, and those pieces can contain multiple files.

      • XTornadoEnglish
        1·
        1 year ago

        The protocol not, the clients…not sure nowadays but in the past some would freak out when listing the files… as in freeze for minutes. Hopefully is not an issue nowadays no idea.

    • fkn@lemmy.worldEnglish
      0·
      1 year ago

      Depends on final size of the folder zips. 20-30mb? Sure. 200-300? Nah.