Genuine inquiry . Maybe I am not experienced enough with the various federated platforms but I am an avid user of matrix, and have dabbled in lemmy. From what I have seen is federation is on the path to decentralization but not fully there. It creates fiefdom, little kingdoms . Great yes you may find one that suites you better, but users now can end up isolated to their island, switch island sure but now you are isolated for the previous island and maybe others. Its stupid. On matrix you need to know the other island(server) to even find its rooms(communities). Some rooms block users from one server while others block users of other servers. You either have to run multiple accounts or accept the limits. Add in you are at the mercy of your home server, you can lose your account have it immitated, and more. The performance is horrible not sure why, but content is slow to update and spread. Matrix has the problem because of its design most people are on the matrix.org server and so the point of federation is largely lost. They are moving to p2p where it seems the solutions for federation now dont apply.

Anyway why is federation not stupid? Are these problems only with Matrix? Cause I look at lemmy and it seems far worse.

    • lemm1ngsOP
      7·
      3 years ago

      Thanks for your excellent response. I find it a bit contradictory you assert federations is not to get stuck in echo chambers but also to make little groups. Not that I am against either. For me federation is more about censorship resistance. To that end I think of regimes that shut down the internet or block things like whatsapp or even signal when the ‘citizenry’ become an organized problem. Which is why things like the matrix p2p effort is important to me. Devices that can self organize into a continually changing platform is very powerful. I also see in federation control of data but that is less important for me than censorship resistance.

      There must be many ways this can all be done, discoverability shouldn’t be a problem.

      I know some are trying to build forums on matrix. I wonder if the lemmy devs have considered intergrating with matrix. Matrix seems to be on an exponential curve now with their 30million investment and the german health system selecting them

    • Ferk
      7·
      3 years ago

      But the point of Instances and federation is not about engagement and discoverability. It’s about making intimate communities, unique to their own. Like clubs. The objective is to make connections/friendship with people, focusing on your own interests and objectves

      Isn’t that already possible without federation? I don’t understand the purpose of having the servers federate if having intimate communities were really the point.

      You can even use something like OpenId to have consistent user authentication across different servers without the need of each server having to federate among themselves. They’ll still be intimate communities that anyone can host and each user would still only need one OpenId account to access them all.

      To me the point of decentralization is the openness of the protocol that allows me to access content as I choose. Ideally, I should be able to host my own instance without creating any community or inviting anyone else, yet still be able to federate with other instances to consume the content through my own interface and cache it (so that, for example, I can access to it from my intranet even if I go offline, or the other server goes offline). Ultimatelly, if enough people did that then it starts to become closer to a P2P network. I think it wouldn’t be a bad idea to go for a hybrid model were you can have peer nodes and dedicated servers at the same time, which is what I believe Matrix is going for.

        • Ferk
          3·
          3 years ago

          having an openId account, you’d have to open a bunch of different websites to access their communities

          Not necessarily.

          It’s also possible to set up ways to access communities from one place without needing federation. Nowadays you can build webapps that communicate with different servers. In theory you don’t need server-to-server communication (which is what makes it federated), you can have one client communicate with multiple servers.

          An example of this is how in the webapp from Matrix (https://riot.im/app/#/login) you can change the account (or even create an account) for any matrix server (it doesn’t have to be matrix.org) without needing to have different websites for different instances.

          In fact, that model (separating the frontend website from the server) is more flexible in terms of UI than current state in lemmy. I cannot choose to use https://lemmy.ml/login if my account is not @lemmy.ml …I’m forced to use the website of the instance I have an account with, and if I have multiple accounts in different servers I do have to use different websites. Whereas in riot.im/app the website is not even hosting a matrix server and you can use any account you want.

            • Ferk
              3·
              3 years ago

              If it uses OpenID accounts for the authentication then nobody really exclusively “belongs” to A or B.

              You can send messages to both server B and server A. Both servers will identify you based on your OpenId identity and it would allow you to post in both servers, without the servers needing to speak between themselves.

              It’s like sending emails to a server and have the server publicly display the messages. You don’t need to be registered in that server, the server just needs to receive your email and if they don’t have your address in a blocklist, show it in the appropriate manner.

              You still can post in both places and people can still connect to those places and see your messages. It’s basically sharing the same account and the same interface across communities hosted in different servers.

              Yes, it’s an alternative to do the same thing. This is why I’m saying you don’t need federation for that goal.

                • Ferk
                  2·
                  3 years ago

                  You’ll have alot of wasted info data that cannot be pruned, comparing it to fediverse where federated data can be deleted later. you’d have two copies that can’t be deleted.

                  I don’t understand. Why would you have two copies? In my example there’s no federation. If I send a message to server B it’ll only be in server B. You don’t have to duplicate the data. I expect this would produce actually less duplication than if it was federated, since you don’t need server-to-server caching.

                  No it’s not, I’m saying that you’d have to implement complicated solutions to a problem that federation solves it more simply.

                  It’s actually simpler. Federation requires server-to-server communication.

                  That really sounds like centralization. You don’t get to communicate with other servers then people would rather be in one place where everyone is there

                  You can communicate with both server A and server B, without having to go through the process of creating an account in server A and server B.

                  This is the same way as how you can post in https://<server-A>/c/fediverse and in https://<server-B>/c/fediverse with an account from <server-C>. The only difference is that in this case <server-C> would be an OpenID provider, specialized only in authentication, and not really a content provider.

                  You could apply the same argument of why would people want to use https://<server-B>/c/fediverse if everyone is using https://<server-A>/c/fediverse

                  • lemm1ngsOP
                    1·
                    3 years ago

                    I like what you are saying with openid but without caching or duplication the network scales poorly and content can go missing.

      • nutomicA
        3·
        3 years ago

        To me the point of decentralization is the openness of the protocol that allows me to access content as I choose. Ideally, I should be able to host my own instance without creating any community or inviting anyone else, yet still be able to federate with other instances to consume the content through my own interface and cache it (so that, for example, I can access to it from my intranet even if I go offline, or the other server goes offline). Ultimatelly, if enough people did that then it starts to become closer to a P2P network. I think it wouldn’t be a bad idea to go for a hybrid model were you can have peer nodes and dedicated servers at the same time, which is what I believe Matrix is going for.

        The server needs to be online to receive activities from remote instances, so it should stay online (they will retry sending a few times). The solution is to cache things on the client when it is online. That would actually be pretty simple to implement, because each post, user etc has a specific id. Then you also need to cache anything that your client sends, and transfer it to the server once its available.

        Honestly this solution is better than p2p, as long as your are using the internet and not something like bluetooth.

        • Ferk
          1·
          3 years ago

          Sure, but then you need a heavier client. And at that point, why do you even need federation between servers? Just have the client do the work, connect the client to multiple servers instead of having the traffic go server-to-server in federation.

          Imho, this defeats the point of federating.

          Another advantage of hosting your own instance (even without hosting any community) is that then you are the one in control of your identity (same reason why someone might want to host their own email servers). Ideally I should be able to participate in a third party instance without being worried of my entire identity disappearing if that instance disappears, or if the owner of the instance doesn’t like me and wants me removed from his server. I’d much rather just be banned from communication with that instance rather than lose my entire identity in the whole federated network.

          Even the OpenId example I gave that is not using federation is more flexible in this regard. There you really can own your identity and participate in multiple communities without being “owned” by them.

          If that flexibility is taken away, then it’s not very different from not having a federation at all. In Mastodon at least the federation makes sense for threads of toots that are truly cross-server (as in, none of the servers actually owns the entire conversation, or has admin privileges over the other). But in the case of a reddit-like post where communities are owned by a particular instance, I expect the server hosting the OP has authority over what comments are done in that post, or what posts are done in that community. So in the end the control of the conversation is centralized, even if the messages come from other servers. At that point they might as well come directly from the clients.

          • nutomicA
            2·
            3 years ago

            You really underestimate how hard it would be to write something like Lemmy as true p2p. You can already connect to multiple servers from Lemmur. And of course you can host your own instance for use with any client, if you are worried about an instance disappearing.

            In Lemmy federation actually makes a lot of difference, for example your instance may not show posts or comments in a remote community if the author’s instance is blocked. Votes are also not counted from blocked instances.

            • Ferk
              2·
              3 years ago

              You can already connect to multiple servers from Lemmur.

              But that’s using separate identities, right?

              I wasn’t comparing it to p2p. And I agree p2p is hard. What I was talking about is using something similar to OpenID to abstract the authentication and share an identity across servers. Then each server can have whatever forms of moderation they prefer. The servers would use standardized APIs for the client and identity server, so users can use any compatible client and any compatible identity provider (OpenID style).

              • nutomicA
                3·
                3 years ago

                Sorry, I still had another comment by @lemm1ings in mind and didnt notice that you’re a different user.

                The main point of federation is that you dont need multiple accounts on different instances. Instead you can communicate with people on different instances from a single account. I am not familiar with openid, but if someone else wants to implement that in lemmy, we would probably accept it.

              • lemm1ngsOP
                1·
                3 years ago

                or Matrix could just solve p2p really well. The matrix.org situation has been mentioned by the p2p dev as motivation for p2p. The side benefit is matrix p2p solutions should work for other projects.

            • Abbé
              2·
              3 years ago

              Anyway. hopefully the dendrite server can solve this problem, hosting that hog of server is costly.

              Lately Synapse has been pretty light on resources, and performant, and being actively developed reference implementation has its perks. Hopefully Dendrite, Conduit, et. al. will catch up, but Matrix in general is not as lightweight as XMPP, due to former’s focus on being able to evade censorship

                • Abbé
                  2·
                  3 years ago

                  With user joined to ~30 rooms with ~200 users (average), and none of them is of the order of Matrix HQ, following is the ps aux output (sixth column is RSS, physical memory in use in KiBs) on a 4G RAM VPS (2 cores), which also hosts XMPP, gitea, fossil-scm, et. al. services. I used to run it all by itself on 2G RAM VPS, but consolidated ~2 months ago:

                  postgres 29897   0.0  3.7  191284 153736  -  Ss    2Jul21     28:15.26 postgres: synapse synapse 127.0.0.1(5828)  (postgres)
postgres 37308   0.0  3.7  191284 153264  -  Ss    2Jul21     27:54.09 postgres: synapse synapse 127.0.0.1(16807)  (postgres)
synapse  45418   0.0  9.0 1246748 375220  -  Ss    2Jul21   2264:01.80 /usr/local/bin/python3.8 -m synapse.app.homeserver --daemonize -c /usr/local/etc/matrix-synapse/homeserver.yaml
postgres 45433   0.0  3.7  191284 153096  -  Ss    2Jul21     28:21.45 postgres: synapse synapse 127.0.0.1(12404)  (postgres)
postgres 46529   0.0  3.7  191284 153644  -  Ss    2Jul21     28:06.31 postgres: synapse synapse 127.0.0.1(31560)  (postgres)
postgres 47450   0.0  3.7  191284 153972  -  Ss    2Jul21     28:16.65 postgres: synapse synapse 127.0.0.1(30342)  (postgres)
postgres 47929   0.0  3.7  191284 152948  -  Ss    2Jul21     28:20.01 postgres: synapse synapse 127.0.0.1(8337)  (postgres)
postgres 48024   0.0  3.7  196404 153296  -  Ss    2Jul21     28:22.18 postgres: synapse synapse 127.0.0.1(43996)  (postgres)
postgres 54977   0.0  3.7  191284 153656  -  Ss    2Jul21     28:16.74 postgres: synapse synapse 127.0.0.1(8192)  (postgres)
postgres 55034   0.0  3.7  191284 153468  -  Ss    2Jul21     28:20.01 postgres: synapse synapse 127.0.0.1(48984)  (postgres)
postgres 74852   0.0  3.7  191284 153328  -  Ss    2Jul21     28:10.76 postgres: synapse synapse 127.0.0.1(44007)  (postgres)

                  Disk Usage (media) is 1.6G, and load average is ~0.33

                  It can be tuned further, but didn’t spend time on it so far. Although 2 years ago, I could not run it on 2G RAM VPS. Federation traffic kept DoSing the server, once you join any room. Lately, situation has improved a lot.

              • lemm1ngsOP
                1·
                3 years ago

                What censorship evasion are you refering to with Matrix internal or external?

                • Abbé
                  1·
                  3 years ago

                  The room does not live on one server in particular. So, if server A is blocked, then one can still participate in the same room through a different server B.

          • lemm1ngsOP
            1·
            3 years ago

            Signal’s centralization is a huge weak point proven with the servers crash a while back.

            Wanting decentralized tech is more a libertarian vs authoritarian thing than left and right. I really dont want to pull politic politics into this though cause the tech politics alone is enough :-)