I’ve been looking for a free Reddit alternative and preferably one that was federated. I’m not really sure how federation works with this though. A lot of similar sites are just personal projects that people made as a hobby that lack a lot of important features or the interface was really ugly.
I haven’t seen how to moderate communities though but the Github page says this can be done, which I consider important since I want moderation to be done by communities and users rather then admins. If there’s a quarantine feature similar to Reddit that would be useful too so I don’t just have to ban communities.
I know, that’s why I said “should”.
I’ll repeat what I said: If you really really don’t want to have blocklists, and you do want to rely on user-instances being trust-worthy… then well… I guess in that case the current federated model that uses whitelists makes sense. But then it’ll be a closed network, not comparable with open networks like XMPP / Matrix, where self-hosting is not only possible, but encouraged.
Not only did I never say or imply that federation is incompatible with self-hosting, I actually gave notorious federated protocols (XMPP & Matrix) as example of protocols that encourage self-hosting (unlike Lemmy).
And btw, those other projects actually make it a point to try and minimize the creation of big nodes (Matrix in particular is pushing hard to try and make matrix.org smaller, to the point that they are experimenting with a hybrid P2P model now). Imho, these are examples of federation done right. Lemmy, as it stands with its current design and whitelisting model, is not.
I don’t understand how did you reach the conclusion that I’d want to get rid of user instances. I don’t even understand why getting rid of user-instances would even make what I said true.
The Google/Facebook example was a way to showcase how user-instances do not serve the content, the same way Google/Facebook do not serve the content of the websites that use them as auth methods.
Ah. I see where there’s confusion.
I don’t think it’s required for the user instance to “provide a front-end that serves content from various community-instances”. That’s the job of the client software, not of the user-instance. The user-instance can be just a simple server that handles user accounts and nothing else. It could be theoretically used for other types of services, it might not even be that much related to Lemmy in particular. It might even be just an “OpenID Connect” instance, maybe with some customizations if required.
You could have client software that’s completelly web based, using localstorage to keep its local cache (in addition to the remote community-instance having its cache) without having the server who provides the webclient cache or serve any of the community content. An example of this is https://riot.im/app a Matrix client on the web that you can use to access any Matrix instance and consume content via sending requests directly to the instance where the content is hosted. riot.im is not the server, matrix.org (or whichever server you specify when connecting) is. The web client is not responsible of the content you access through it, because it does not serve nor host that content. And the user-instance also does not. Only the community-instance would be responsible of its own content, and because it’s not federated it will not need to take responsability of content hosted by other community-instances.
Of course the same server that runs a user-instance can have also a web client and can even run its own community-instance, all at the same time. But the point is that each of those is a separate module and they don’t really federate (as in, no two instances of the same type communicate with something that isn’t a service I can replace. And because they don’t federate and don’t cache third party content you remove the biggest reason to do whitelisting/allowlisting.
You are not leaving lemmy.ml if you are still consuming its content. You depend on it still. Interacting with lemmy.ml through a third party is just a way to use lemmy.ml. The difference between interacting with lemmy.ml directly is just a technical detail.
It does not give you much of real end-user advantage. The only difference is that the user details will be hosted in a difference instance, but you don’t need federation for that, as we have already discussed.
I don’t understand how Lemmy protocol encourages to not self host. I’m not much familiar with Matrix, but to me it seems that the only difference is that a Lemmy community is only hosted on one instance while a Matrix room is duplicated on each of the users’s home server.That’s probably very naive, but wouldn’t it mean that in practice one would need more memory to self host a Matrix 1 user-instance?
If it’s about the fact that it uses whitelisting instead of blacklisting, I already told you that I’m also in favor of the latter. But this should only have impact on the time it takes to federate, not on the state of an instance once it’s set up. After all, the only difference is that the other instance’s owner needs to press “ok” before federating.
I think we misunderstood each other about what “managing a user account” means. I define the user-instance as a server where the user has an account, and who federates with the appropriate community-instances so as to serve their content to the user.
Note that I mistakenly used the word “frontend” here, but I never thought that this instance was the same as the client. Sorry about that ! In your Matrix example, matrix.org is definitely what I would refer to as the user-instance.
I noticed later that by “managing a user account” you probably only meant “provide an ID who allows to avoid setting up a new password”. Is that right? That thing is not an “instance” of the networking software, as it doesn’t know anything about the software in question, and only provide some kind of token that allows to create an account on the community instance in question. (the account may be as simple as a pseudonym and a reference to the ID, but may also possibly contain additional info or keep track of the user’s activity on the instance)
This brings me back to my previous comment. Say the user-instance (as defined above) is replaced with a simple ID-provider, and serving the content from the community instances becomes the client’s job. This means that you don’t have a cross-instance profile anywhere on the internet, and need to manually connect to any instance where you have an account each time you connect on a new machine. Right?
Now it seems to me that you not wanting user-instances (I still think you don’t want them if they are defined as above, but please correct me if that’s wrong !) is one of the reasons why you would invoke meta-communities as a way to keep track of a cross-instance list of communities.
Again, I don’t agree that’s the main reason, but we discussed that already.
Sure, but remember that white/blacklistings on the Fediverse are not transitive. So I can federate with both lemmy.ml and with other people who are on instances that don’t federate with lemmy.ml. Unless some instances federate only with lemmy.ml, one is never incentivized to stay on lemmy.ml if they want to leave it.
Note that this also works in the context of your original proposal of cross-instance categories : if you follow a category that is open to lemmy.ml, most content that you consume there will come from lemmy.ml. In fact, except if your instance blacklists lemmy.ml. But I’m not sure about why stopping to interact with the biggest instance would become a goal per se.
I don’t like whitelisting either, but if I opened my own instance with the current state of things, I would actually apply whitelisting on my server myself to limit what I federate with.
The problem is that the way the federation works in Lemmy would force my instance to cache and publicly serve content from any server it federates with. And I don’t want to host illegal content and get in trouble.
So the issue is that whitelisting is the only way to safely host a lemmy instance without requiring close eye maintenance. This is what I see as the biggest problem.
If it’s that simple, the whitelisting would not be very useful to protect against bad instances. It’ll either be inefficient (if not enough control is applied) or be a burden (if too much control is applied). The line that divides those two is diffuse and it’s likely we’ll get both false negatives and false positives, so there’ll be mistakes either way, ending up both being a burden and inefficient.
It also complicates things to add a human factor to the process of setting up your instance. You’ll have to do some public relations with the other instances and the other instances might want to keep an eye on your instance which adds to the maintenance burden, specially if lemmy ever becomes popular and it ever reaches numbers in the hundreds of instances.
With an email server or an XMPP server, self-hosting is enough to communicate with everyone. But with lemmy I’ll have to do work for every instance, at that point why not just create an account in that instance, which might actually be instantaneous. I’d rather make a multi-account client that abstracts my identities as if they were one instead of hosting my own server to have control of my identity.
It would defeat the whole point if I meant for the user management instance to federate with instances of its own class. This whole branch on the conversation thread was about me wanting to explore the idea of federation not being needed to do what Lemmy is doing right now, that instead of federating you can separate the services in a more “microservice”-ish like model (but not as extreme), making it not only simpler but also more modular and reusable.
Yes, more or less. It’d act as an authentication server (I explicitly linked OpenID’s wikipedia page right after mentioning “user account management”) and it’d also allow users and its sessions to be created/removed/edited (again, that’s what “managing” means when we talk data). Any operation that related to the user and not the content. It does not view/add/remove/edit (manage) any kind of community content, that would be content management, not user management.
Not necessarily, there could be user metadata stored in the user service. Things like name, profile, personal website… but also what instances the user has a token with. I expect the token exchanged is long lasting so the user-instance needs to keep track on its side about this already anyway. Much in the same way as how Paypal keeps “authorizations” for websites the user has allowed payment to be automatic, the user-instance would allow the client to know of those websites and be able to access them with the token without manually having to connect (done from whatever the client is, without serving the content). Also the notifications system could be part of that instance, or it could be its own separate service entirely.
That’s the kind of instance I’d be happy to self-host, because it’ll be mainly personal data related to my user. I’m not so interested in hosting communities.
It’s the other way around. Don’t you remember what started this “federation” topic?
To me cross-instance “topics” (either tags or meta-communities) would be the one thing that would make federation useful. Without that, I don’t see the benefit of federation. Which is why I was talking about separating the user management (and notifications) from the content management, so that you can achieve shared accounts without having to federate.
Sorry if I’m repeating myself (I think I’ve said that same paragraph in almost every comment, with different words) but I don’t know how else to explain it.
Again: “You are not leaving lemmy.ml if you are still consuming its content.”
If you are using the non-transitive property to consume lemmy.ml content, then you are not leaving lemmy.ml, even if you are connecting to it through a different instance. Even if you don’t have an account in lemmy.ml, you did not really left it. Since we don’t have a cross-instance “topic” to subscribe to, and since you don’t want “duplicated” communities, you have to keep relying on lemmy.ml as the central node where most of the active communities are.
It doesn’t have to be a goal.
But in that case just embrace the centralization of those communities, what’s the advantage of federating? The things you want to share cross instance, share them by having a separate service handle them. Only if there’s a reason to have community-servers directly communicate with each other (eg. cross-instance “topics”) does federating across them become useful, imho.
So you say that as soon as you create an instance, everyone can see its frontpage, and that there is no way to create an instance where the only thing that outsiders can access is your profile? That would solve your public serving problem, right?
What other difference are there?
Ok I was probably wrong in saying “setting up” there, as if you had to decide on day one what you federate with. How it would actually work is that you can vet any new instance that you encounter (e.g. via a crosspost, or via a new community there that is announced on some instance you already federate with) before sending it a federation request, that it then would have to accept.
In fact, do you know what happens when a user from an instance that I don’t federate with comments a post that I see? Is it simply hidden? Does it say “comment from remote instance hidden”? Or does my instance’s admin receive a suggestion to federate with them? Or maybe in Lemmy federation is only about which communities I can access, but doesn’t hide comments from unfederated (with me) instances?
Filtering the content a user has access to and constructing their frontpage can be seen as part of managing their account. As I said, we misunderstood each other’s definition of managing the user account, that’s ok.
We said already several times, federation also offers the advantage of being able to delegate filtering tasks to the instance you register on. Each user doesn’t have to allow/block each (community-)instance by themselves, but can leave that to an instance admin that they trust.
It also allows the instance to store its users’ history (or outbox in activitypub language).
If the content is served to you by your user instance, it also means to don’t have to communicate with the community-instance every time you want to read content from it. This means that the community-instance cannot track your (passive) activity, which I think is also valuable.
What about crossposting?
Again, the federated network can have one-user nodes, but federation offers the possibility to delegate tasks that you don’t want to do yourself.
Again, you can completely leave it in the sense you mentioned by registering on an instance that doesn’t (plan to) federate with it. So you can leave it, and keep interacting with the rest of the network, the part that interacts both with you and with lemmy.ml.
(This is of course not the only acceptable definition of “leaving” in that context. Isn’t one user closing their gmail account to register with another provider leaving it? Would you say “you don’t leave gmail if you can still consume content from it”?)
Yes, essentially. It’s too much of a problem having to publicly host content from other people’s instances that you might have not much control on, unless you activelly monitor them constantly, which is why I think it makes sense lemmy instances use whitelisting. Even if whitelisting is an obstacle in federation.
However, you cannot do what you proposed to solve the same problem when it happens in community instances. As long as community instances federate, whitelisting is the safest option. This is why I propose they should rather not federate among themselves.
Between whitelisting vs blocklisting? I’m not sure if there are any other significant differences than what we already discussed.
As far as I understand, instances are unaware of what instances not in the whitelist might have commented, because they do not host that content. In the words of a lemmy dev: " your instance may not show posts or comments in a remote community if the author’s instance is blocked. Votes are also not counted from blocked instances " …I expect this means the comment will simply not appear at all, but I haven’t tested.
And I already answered that this can be done with shared blocklists. Anything you want to share, see if it can be made made independent, instead of assuming cross-instance federation is the only way.
Can’t the feed just reference to community content hosted in community servers without having to host the content itself? The client may load that content in the feed requesting it from the community servers directly, without it having to be cached in the user-instance.
Or, alternatively, request separately the history feed for each community instance the user has interacted with within the given timeframe (the user instance should know which ones) and merge their history client side.
You can already do that client-side. The client can cache all the requests on local storage so it doesn’t request the same data every time it wants to read it. This actually was suggested by one of the lemmy devs before.
The network will be significantly crippled though, since most active communities are on lemmy.ml.
If gmail has a significant majority of the network of email and you still have the need to communicate primarily with gmail accounts, then yeah, you still depend on gmail, even if it’s not who hosts your email account. This gives a lot of power to gmail over the email protocol, even if you don’t have a google account, your emails will get into google’s hands.
Imho, one of the points for federation is trying to avoid depending on big central nodes. If one node is down, disappears or becomes “evil”, the impact is not as big as long as the federated network is well balanced.
Of course, I agree that if we separate community instances from user instances, there is no need for community instances to federate with each other. They only need to federate with user instances.
In turn, user instances may or may not federate with each other. That’s the rest of the discussion. If they don’t, it means there is no user-to-user interaction independently from communities, right? For example no direct message feature.
So you claim there should be a separate service for each community-independent feature : one for providing ID, maybe one for DM, one for sharing blocklists, … That is indeed another way to do it, I don’t think I ever claimed federated user instances was the only one. It is still one way, and it has its own disadvantages but also advantages over the one you advocate for.
Maybe true if you want to check a specific content, what I mostly meant was that if you want to check an updated community feed, the community instance is aware you do that request. The user instance is an intermediate who doesn’t have to tell which user is checking the feed.
But even for the former use, it is not always true : what if you are using a webclient from a shared computer?
First, same argument as before : you need to tell every community instance in you subscription that you are checking your history.
Also, that means basically that you lose your own content when that instance shuts down.
For this latest two points, the idea is really that you can trust a single (ideally small scale) user-instance instead of vetting each community instance that hosts content that you are interested in. Can be useful if most communities are on an instance that one day turns evil :)
Not negating that.
Not having an account is still an advantage though, as google doesn’t know anything about what you exchange with people who don’t use it. Federation allows you to
get that level of “leaving the main instance”
keep in touch with people still on the main instance
keep in touch with people who completely left it, without exposing them to it.
Yes. That’s the gist of it. I feel we are reaching an understanding, I’m hoping our comments will start getting smaller :P (although I do enjoy the conversation).
Also note: imho, there’s a big difference between serving content through federation for 1-on-1 private communication that can be encrypted end-to-end where not even the server knows what data is being exchanged, and hosting public content for everyone to see. I suspect this is why XMPP / Matrix can more easily federate openly without leaning towards whitelisting. This is why I’m not so worried about 1-on-1 conversations. In fact, just use XMPP/Matrix (or even email!) for private communication, no need to reinvent the wheel. The user-instance could even just be an XMPP/Matrix server and make use of their pubsub layer for the notifications on new replies.
This is true, but there can be ways to mitigate that too. If the community is public, the API to retrieve the community feed doesn’t need to be authenticated, you might only need to authenticate the request if you want to submit content or if the community is private. And in both of those cases, even if there was an intermediary, I expect there should be a way for the original instance that hosts the community to make sure the user has the rights to do that operation on their instance, so they would have to know either way.
They still can know your IP, but that’s a different level and using the same IP doesn’t mean you are the same person. Specially when using shared computers or moving around. And you’d still have that issue if you host your own user instance to cache your accessed content.
Personally I don’t have a problem with the community-instance knowing I’m accessing the content, specially if that’s the cost of mitigating whitelisting. After all, the whole internet works this way already.
My motivation for federation is in the decentralized and open nature of it. I’m ok with having data (as well as knowledge of me accessing that data) across smaller servers, because as long as they are small and I don’t use them primarily, they won’t be able to know what I do outside of their domain, only what I do when accessing that server. The issue is when not only there’s one predominant server hosting most of the content, but also there’s whitelisting that makes federating with it and self-hosting troublesome.
That depends on what you compare it with. The 3 points you enumerate can be done without needing communities to federate among themselves.
Note that in this section of the conversation we are not comparing “closed walled gardens” vs “federation”. The point we were discussing in this branch of the conversation was: would it be bad if one instance (ie. lemmy.ml… or gmail.com for email) is central to a substantial majority of the activity?
When I said “embrace the centralization”, I didn’t mean making it a walled garden, I meant replacing the federation between communities with shared services. With user accounts being shared it wouldn’t be any more of a closed network than it already is (it might actually be less closed if it leads to removing whitelisting). I’d mean communities will be accepted as a centralized aspect that lives in one instance, even if other aspects (like user accounts) are decentralized. Essentially this would remove pressure from the community instance to have to serve third party content since that’s the requirement for federation. And with that pressure removed, I think there’s much less of a reason to do whitelisting (the only reason left would be avoiding toxic users, but that can be done in other ways as discussed, or at the very least the whitelisting could be limited to content submission rather than content access, so you don’t need to create different accounts for accessing different networks that don’t connect with each other).
The alternative to “embracing the centralization” is actually pushing for decentralization. Trying to fight the social snowball effect that might make big instances more attractive to use than small ones and implementing more cross-instance ways to access the content so that the system is more decentralized. Attempt to minimize the impact of a community “belonging” to one instance, so that smaller communities in smaller instances can compete for user attention in fairer conditions.
I also think we are coming close to the end of this nice but long conversation.
We both understand that what a user instance does can be done by several small services. The latter gives you more control, but the former gives you administrative simplicity. I guess which one is desirable is a matter of taste. It may be nice indeed if it was possible to do those things separately, but there would still be combined offers.
There’s also the question of whether the community instance need to serve the user instance (name we give to the above combination of services) or directly to the client. The advantage of the former is that the community instance doesn’t know anybody checked the content in question, as it was regularly served to the user instance (from what I understand fron activitypub, the content is sent to the instance with the indication of who can access it). As you point out, the request may not need authentification, which makes it less dramatic, but I’m sure data recombination can get creative.
Note that aside from the theoretical question of which is the best protocol, being able to talk to a user instance is necessary if Lemmy wants to connect with the rest of the fediverse someday.
A disadvantage of federation is that one may end up serving undesired content. You mention whitelist as a crucial point here, but again it only changes the speed of connection with unknown servers, thereby only helping when discovering evil instances. When a trusted instance turns evil, it really doesn’t change anything. So the usefulness of whitelisting depends on the rate of appearance of new evil instances. I don’t know how much it is in practice, but that most of the fediverse use blacklist might indicate it’s not that dramatic.
To be honest, I’m not a big fan of Twitter-like services (I find there’s way too much noise in such feeds), so I never really explored the Fediverse surrounding activitypub too much. Any criticism I have on Lemmy current structure is likely to extend to other parts of the Fediverse where similar approaches apply. So I guess you could see my points as rants on that approach for public cross-publishing in general, rather than Lemmy in particular. I guess I picked the target I feel the most interest towards and that I think has the most potential to become something I could use a lot.
If Mastodon (and the activitypub fediverse in general) has the same approach of cross-publishing and forces instances to serve the content from third parties, I think it’s risky that they don’t have a stronger policy towards whitelisting that imposes a tight control over what instances are allowed to have their content served through them, imho it’s asking for trouble to appear. So I think whitelisting was a justified approach in case of lemmy.
In my mind, the issue with whitelisting/blocklisting is the limitation on the reach of the federation. People would have to create multiple accounts to access networks that don’t federate, rather than using one account that can access it all. It makes sense for an internet forum to ban users, block them. But it would make no sense for an internet forum to explicitly block their (legit) users from visiting other forums.
So, ideally, it would be best to find a solution that allows users to access any instance that’s technically compliant with the protocol without having to impose the burden on each instance host to carry responsibility for the content of other instances.