• andscape@feddit.it
    link
    fedilink
    arrow-up
    6
    arrow-down
    1
    ·
    1 year ago

    In order to avoid this restriction you would need a global instance discovery mechanism, which is extremely hard to implement without a central server that keeps a list of all instances in the network. And if you do implement instance discovery through a central server you really are losing the whole point of decentralization.

    Additionally, it’s good that each instance does not federate with everyone else by default. If it did, it would have to process all activity and keep a local copy of all the content in the entire network. This would be insanely inefficient, and make it prohibitively expensive to run even a tiny instance with 1 user and no communities.

    Decentralization isn’t useless if you can’t immediately see everything in the network, come on… We’re just spoiled by centralized services.

    • sugar_in_your_tea@sh.itjust.works
      link
      fedilink
      arrow-up
      3
      ·
      1 year ago

      you would need a global instance discovery mechanism

      I don’t think you do. Instances should merely reach out to other instances it’s federated with periodically to get a list of communities and some of their metadata. Ideally, they could ask all of those other instances to notify it when a community is added, modified, or deleted, and then store that metadata.

      That should be pretty easy to implement, and maybe it already has, idk.

      • andscape@feddit.it
        link
        fedilink
        arrow-up
        1
        ·
        edit-2
        1 year ago

        Sure, but this isn’t finding new instances, just new communities on known instances. Indeed, this is not difficult to implement. The reason it’s not done already is for resource economy. A lot of instances are already struggling to scale, making them process and store a lot more content with little value for most users of the instance isn’t feasible for a lot of servers right now.

        • sugar_in_your_tea@sh.itjust.works
          link
          fedilink
          arrow-up
          1
          ·
          1 year ago

          A list of communities isn’t “a lot more content.” Just run it once daily and the problem is solved. Instances don’t need to store posts, just community names available on that instance.

          If the issue is finding new instances, I think it’s fine for it to take some time the first time someone tries to find a community on that instance. But after that, it should immediately have a list of communities from that instance.

    • Martineski@lemmy.fmhy.ml
      link
      fedilink
      arrow-up
      3
      ·
      1 year ago

      Ummmm… All you need is some bots on each instance that automatically will interact once with communities known on lemmyverse.net and boom, you have unlocked full federation for every user on those instances. You are not losing any privacy through that, it just skips the steps where user has to manually index a sublemmy before it federates and makes platform more usable.

      • andscape@feddit.it
        link
        fedilink
        arrow-up
        4
        arrow-down
        1
        ·
        1 year ago

        Sure, but now this system has a dependency on the “centralized” lemmyverse.net service. And also your instance now has to receive and store a copy of almost the entire network’s content. Lots of instances are already struggling to sustain the load, this would make the problem even worse.

        If a single instance decides that it can sustain the increased load and doesn’t mind depending on lemmyverse.net sure, nothing’s stopping them. But it shouldn’t be the default behavior for all instances.

        • Martineski@lemmy.fmhy.ml
          link
          fedilink
          arrow-up
          1
          ·
          1 year ago

          If load is that big of a problem then pretty much all instances are in a big trouble because all you need is ONE person that decides to run some bots and it’s done anyway.

        • interdimensionalmeme
          link
          fedilink
          arrow-up
          1
          arrow-down
          1
          ·
          1 year ago

          Yet bittorrent dht is 20 years old. How can this supposedly decentralized service be unable to self organize. Is Lemmy some kitchen napkin high school fair project ?

          • andscape@feddit.it
            link
            fedilink
            arrow-up
            1
            ·
            1 year ago

            Social media and torrents are pretty damn different. There’s a reason no federated platform has implemented automatic discovery, even ones with much more resources than Lemmy, like Mastodon.

            I don’t know why you folks keep pointing at missing features and saying “Lemmy doesn’t have this pretty advanced network feature, so it’s not really decentralized”, or “it cannot organize”, or “it’s useless”… It’s basically two people’s passion project that only blew up in the past month because reddit fucked up. You’re not paying for it, are you? So I really don’t see how this attitude is warranted.

            • interdimensionalmeme
              link
              fedilink
              arrow-up
              1
              ·
              1 year ago

              The tech is right there, it’s 20 years old. I’m pointing at it in response to people saying “this is too hard, we can’t have 700 instances sharing a few kilobytes of text !, You’re asking too much”

              • andscape@feddit.it
                link
                fedilink
                arrow-up
                1
                ·
                1 year ago

                Look, if you understand what a DHT is I also expect you to understand the amount of effort it would take to implement such a feature. The fact that the tech exists does not mean you can just plug it in and go. It took the devs weeks to move from websockets to HTTP, a feature like this would take months and take time away from a lot of other important work.

                • interdimensionalmeme
                  link
                  fedilink
                  arrow-up
                  1
                  ·
                  1 year ago

                  The pseudocode for this feature is as follows Cron job to be run at regular intervals (say top of the hour) Query database for all content (posts, comments) between last packet and now, place in packet. Query database for all moderation actions ( votes, censure, blocking, pinning, etc ) between last packet and now, place in packet. Take packet, name with name on instance plus date range. Export as dht enable magnet link torrent Forward magnet url to all known federated instances

      • sneakyninjapants@sh.itjust.works
        link
        fedilink
        arrow-up
        1
        ·
        1 year ago

        That’s certainly an option. Probably pretty useful too. Though I think it would put quite a load on server resources to have every community in the threadiverse cached though.

          • sneakyninjapants@sh.itjust.works
            link
            fedilink
            arrow-up
            1
            ·
            1 year ago

            I don’t know enough about how the instances interact on the activitypub protocol to comment on that. I see that Mastodon has over 11,000 instances running, and haven’t heard of instabilities, but they also are fundamentally different applications so idk how comparable they are.

      • interdimensionalmeme
        link
        fedilink
        arrow-up
        1
        arrow-down
        2
        ·
        1 year ago

        How is that not part of base code. Lemmy is completely unusable until this is fixed. The clock is ticking on Reddit’s implosion. If this isn’t fixed, the Reddit userbase will go back to Reddit for another 20 years. Please don’t let Lemmy as useless as Mastodon, this is clearly design sabotage by silicon Valley big tech.

        • imaqtpie@sh.itjust.works
          link
          fedilink
          arrow-up
          2
          arrow-down
          1
          ·
          1 year ago

          Calm down bro. Your negative energy is harshing the vibe here. Let’s try that again

          I see, thanks for the info. The devs should prioritize adding a feature like that into the base code. It would really help reddit migrants get set up here.

          See? I liked a couple of your comments in the other thread about downvotes being public information, but you’re sounding pretty defeatist and frankly, miserable in the past few hours. If you don’t want the reddit userbase to go back, the best thing you can do is be a user that other users enjoy interacting with.

          Unless you wanna step in and start coding this thing 🤣