Heyho,
as a PostgreSQL guy, i’m currently working an tooling environment to simulate load on a lemmy instance and measure the database usage.
The tooling is written in Go (just because it is easy to write parallel load generators with it) and i’m using tools like PoWA to have a deep look at what happens on the database.
Currently, i have some troubles with lemmy itself, that make it hard to realy stress the database. For example the worker pool is far to small to bring the database near any real performance bootlenecks. Also, Ratelimiting per IP is an issue.
I though about ignoring the reverse proxy in front of the lemmy API and just spoof Forwarded-For headers to work around it.
Any ideas are welcome. Anyone willing to help is welcome.
Goals if this should be:
- Get a good feeling for the current database usage of lemmy
- Optimize Statements and DB Layout
- Find possible improvements by caching
As your loved core devs for lemmy have large Issue Tracker backlog, some folks that talk rust fluently would be great, so we can work on this dedicated topic and provide finished PR’s
Greatings, Loki (@tbe on github)
Yes, i see this issue also. I would assume that the statements used here, tend to get very bad plans, due to overhang (specific id’s will have far more entries then others).
This is one of the reasons for my current setup.
But when it comes to optimizing databases, i think i’m pretty skilled in it, and i have seen much worse scenarios (billing systems, processing > 100.000.000 entries per billing run, with tough time constraints).
Just now they found out that Lemmy is falling over with 300 comment threads.
Thx for the heads up. Now, we are talking database ;)