• koavf
    28·
    4 years ago

    Did you ever determine if it was a denial of service attack or just an influx from Hacker News (hi!)

    • DessalinesOPMA
      14·
      4 years ago

      Seemed to be a combination, we don’t have any major monitoring set up, but looking at our nginx logs, it was 55k requests in a short period of time. We posted to reddit a few weeks ago and hit the front page of some linux / OSS related subs, but the server held up fine.

      Luckily adding an nginx rate limiter fixed most of the issues, as the problem wasn’t with rust, but with some clients w/o caching (IE bots of some kind), requesting all the files constantly.

        • DessalinesOPMA
          3·
          4 years ago

          Limiter? I think we have 10 posts per hour, 2 communities / sign-ups per hour.

    • Volodymyr
      10·
      4 years ago

      That’s why I came here anyway. Or maybe DDOS was also attracted by the sudden exposure on HN?

    • Hal
      8·
      4 years ago

      Ahh… the Hacker News hug…

  • stax
    17·
    4 years ago

    Just gotta say that I really appreciate the work you are doing

  • nutomicMA
    15·
    4 years ago

    All good now, the site might just load slower than usual now (we need to rework the database).

    • Panzerfaust
      101·
      4 years ago

      I have seen that. One of the users said that Lemmy “… must be getting the HN hug of death …”.

  • DessalinesOPMA
    112·
    4 years ago

    We have > 2200 connections to the server right now, its a DDOS. Rust seems to be handling it fine, but the nginx is having issues.

    • stanleypar11
      82·
      4 years ago

      2200 does not seem like a DDoS in my opinion, I would expect hundreds of thousands in that case

      • eneville
        4·
        4 years ago

        I’m reading it as the 2200 connections are between nginx and the rust component, not 2200 to the edge nginx.

      • zodvik
        2·
        4 years ago

        I agree to this as well.

        I’m surprised nginx is having issues. I’ve personally handled nginx instances at more than an order of magnitude of connections.

        There should be good low-hanging fruits in configuration to exploit to help handle more load.

    • DessalinesOPMA
      6·
      4 years ago

      We definitely will never use Cloudflare, they’re an absolutely awful company that’s managed to weasel their way as a snoop in the middle to nearly half of all US internet communication.

  • DessalinesOPMA
    71·
    4 years ago

    I’m guessing we got x-posted to the donald or something.

    • schrutefarms
      16·
      4 years ago

      I just made an account and came from HN. I hope it is just a good ole regular hug of death!

      • SirLotsaLocks
        9·
        4 years ago

        Welcome, I like this because it feels like HN but it has other communities as well if you get what I mean. Hope you enjoy it here :)

    • DessalinesOPMA
      51·
      4 years ago

      It is, luckily I have the DB fixes finished, and should be deployed to here within a day or so.

  • Certified_Awful_Post
    32·
    4 years ago

    Site breaking so much it took ten minutes to post this (if it goes through this time).

  • lem135
    11·
    4 years ago

    hmmm…isnt p2p kinda prone to those attacks 🤔

    • lem135
      1·
      4 years ago

      I understood lemmy is decentralized p2p network how is this possible then? ELI5?

      • NXL
        3·
        4 years ago

        this OP post is a month old. It wasn’t being DDOSd it was trending on HackerNews. Also lemmy isn’t p2p its federated. This means it still works like a traditional server but can connect to other servers. In other words it’s Decentralized but not Distributed.