A couple days ago, someone posted on /0 (the meta community for the Divisions by zero) that the incoming federation from lemmy.world (the largest lemmy instance by an order of magnitude) is malfunctioning. Alarmed, I started digging in, since a federation problem with lemmy.world will massively affect the content my community can see. As always [...]
Very interesting read, thank you!
I (self)host a lot of stuff as well as developing and deploying some of my software via docker containers and dabbled in Full-Stack territory quite a few times.
Exposing stuff to the internet still scares the shit out of me. Debugging sucks. There’s so much that can go wrong, every layer multiplicates the possibilities of stuff that can wrong or behave in a way not expected. Your journey describes the pain of debugging perfectly. Yeah, in hindsight, it’s often something that probably should have been checked first. But that’s hindsight for you.
And that’s not even accounting for staying ahead of the game while securing your 24/7 publicly accessible service, running on ever-changing software, with infrastructural requirements you basically have no control over. In your spare time.
Hosting something for yourself can be a lot of fun, hosting something for other, potentially many thousand, people makes you kind of responsible. That can be rewarding and fun at times as well, but is also a prime source for headaches.
Deploying stuff is the easy part, knowing what to do when stuff inevitably breaks is where it is at. Therefore, IMHO, it’s probably a good thing that most Lemmy admins at least know where to ask/start when shit hits the fan. This unfortunately leads to more centralization, but for good reasons: teams of volunteers taking care of fewer instances will almost always lead to a better experience than a lot of lone wolfs curating a lot of small instances. Improving scalability, monitoring and documentation is always nice, but will never replace a capable admin such as yourself.