I already get rate-limited like crazy on lemmy and there are only like 60,000 users on my instance. Is each instance really just one server or are there multiple containers running across several hosts? I’m concerned that federation will mean an inconsistent user experience. Some instances many be beefy, others will be under resourced… so the average person might think Lemmy overall is slow or error-prone.
Reddit has millions of users. How the hell is this going to scale? Does anyone have any information about Lemmy’s DB and architecture?
I found this post about Reddit’s DB from 2012. Not sure if Lemmy has a similar approach to ensure speed and reliability as the user base and traffic grows.
https://kevin.burke.dev/kevin/reddits-database-has-two-tables/
Pretty much everything but the database can be infinitely scaled. The database can be somewhat scaled by adding multiple read-only instances. But you’re always going to reach a limit where throwing more resources at the write database gives diminishing returns. At that stage you need to shard to increase performance which can be done but it’s a similar architecture to having multiple federated instances, so why not just spin up a new instance instead of growing an old one? Another way to lighten the load on instances is with efficient used of relays.