What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Major Changes

This version brings major optimizations to the database queries, which significantly reduces CPU usage. There is also a change to the way federation activities are stored, which reduces database size by around 80%. Special thanks to @phiresky for their work on DB optimizations.

The federation code now includes a check for dead instances which is used when sending activities. This helps to reduce the amount of outgoing POST requests, and also reduce server load.

In terms of security, Lemmy now performs HTML sanitization on all messages which are submitted through the API or received via federation. Together with the tightened content-security-policy from 0.18.2, cross-site scripting attacks are now much more difficult.

Other than that, there are numerous bug fixes and minor enhancements.

Support development

@dessalines and @nutomic are working full-time on Lemmy to integrate community contributions, fix bugs, optimize performance and much more. This work is funded exclusively through donations.

If you like using Lemmy, and want to make sure that we will always be available to work full time building it, consider donating to support its development. No one likes recurring donations, but they’ve proven to be the only way that open-source software like Lemmy can stay independent and alive.

Upgrade instructions

Follow the upgrade instructions for ansible or docker. There are no config or API changes with this release.

This upgrade takes ~5 minutes for the database migrations to complete.

You may need to run sudo chown 1000:1000 lemmy.hjson if you have any permissions errors.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

  • @AlexWIWA
    link
    English
    1247 months ago

    80% reduction is crazy! Good job!

    • @nutomicMA
      link
      English
      1007 months ago

      That particular improvement is actually mine. Lemmy was storing a lot of federation data which was completely unused so I removed it. However the 80% improvement is actually overstated, because not all data was migrated to the new table. So the db will grow a bit bigger over time, but still much smaller than before.

      Phiresky made a lot of other sql optimizations which make Lemmy snappier and reduce CPU usage on the server. We don’t have any benchmarks in that regard, but server load on lemmy.ml has gone down a lot since upgrading.

      • Char [she/her]
        link
        fedilink
        English
        287 months ago

        Thanks for the clarification! Your stewardship along with everyone collaborating in work and donations makes for a wonderful project. Really demonstrates the power of people working together to provide spaces outside of the control of corporate interests that are destroying other platforms.

      • fmstrat
        link
        fedilink
        English
        67 months ago

        Do we need to run a full vacuum after upgrade? And for that matter, is the data not used truncated during upgrade?

    • @nutomicMA
      link
      English
      597 months ago

      It was actually easy because a lot of data was stored which was never used at all.

      • Wren
        link
        fedilink
        English
        467 months ago

        Shhh let me compliment youuuu

      • @netburnr
        link
        English
        117 months ago

        I was saving that data, you never know when it might come in handy.

        • @natanael
          link
          English
          1
          edit-2
          5 months ago

          deleted by creator

  • @wiki_me
    link
    English
    66
    edit-2
    7 months ago

    Update on lemmy finances (not including cryptocurrencies)

    patreon: $1,591/month

    liberapay: $374.22 per week (about 1609 per month)

    open collective: $2082 (29/6/2023 -> 29/7/2023)

    Assuming 63K active users , the per user monetization of 0.08 dollar per user (Reddit’s revenue per monthly user is roughly $1.19).

    Estimated developer salary for the two main developers is about 2600$, estimated median salary for developer in the US is about 10K a month.

    For comparison firefish made about 1424$ ((29/6/2023 -> 29/7/2023) with an active users count of 11868 (or 8146 if you don’t count calckey, which i think is important because they added a pop up asking for donation, but i don’t know if that is after the name change) so that gives a per user monetization of 0.11 dollar per user ( or 0.17 not counting calckey).

    Corrections are welcomed.

  • phiresky
    link
    fedilink
    English
    417 months ago

    I want to clarify that most/ a lot of the DB work of this release actually came from others e.g. nutomic, RocketDerp, Sh4d ;)

  • YⓄ乙
    link
    fedilink
    English
    297 months ago

    🙏 Thank you Devs.

    Kind People, please donate .

    • @JaneDoe
      link
      English
      67 months ago

      As a donor I had to spend some amount of time finding where to donate, that could be more explicit

  • @MarsAgainstVenus@fedimav.win
    link
    fedilink
    English
    26
    edit-2
    7 months ago

    Upgraded without issue (other than manually restarting docker after upgrading)! Thanks!

    I did it with Lemmy-Easy-Deploy by simply running sudo ./deploy.sh -y

  • @Die4Ever@programming.dev
    link
    fedilink
    English
    24
    edit-2
    7 months ago

    well my Lemmy backups did exceed 10MB, but the 0.18.3 database optimizations brought me under 10MB again lol

    11M Jul 28 22:35 bak-lemmy-2023-07-28_22.35.52.zip
    8.8M Jul 28 22:39 bak-lemmy-2023-07-28_22.38.59.zip
    

    (hosting an instance with communities but no users doesn’t need much hardware at all btw)

    • @maegul
      link
      English
      97 months ago

      Willing to share any numbers? How many communities and how many resources?

    • Andreas Fornberg
      link
      fedilink
      English
      3
      edit-2
      7 months ago
      1,4G jul 29 19:15 lemmy_dump_2023-07-29_19_01_51.sql.gz
      262M jul 30 12:58 lemmy_dump_2023-07-30_12_56_35.sql.gz
      
  • @ohai@subsubd.com
    link
    fedilink
    English
    167 months ago

    docker compose down && docker compose pull && docker compose up -d update was flawless. nice!

    • Chris Ely
      link
      fedilink
      11
      edit-2
      7 months ago

      You can pull new images while the old ones continue running happily. That docker-compose down command is completely unnecessary.

      @ohai
      @dessalines

    • RoundSparrow
      link
      English
      67 months ago

      docker compose down &&

      Looks like the & parsing in code blocks is another bug in 0.18.3 like the post title one. I’ll add your comment to the GitHub issue.

  • Sky Cato
    link
    fedilink
    English
    157 months ago

    Please make Lemmy easily crawable by search engines so people can get useful search results

    • @morrowind
      link
      English
      217 months ago

      It is, the problem is search engines aren’t designed to deal with federation. Ty search-lemmy.com for one that is

  • RoundSparrow
    link
    English
    13
    edit-2
    7 months ago

    Heads-up that a new issue on GitHub reporting this problem upgrading from 0.18.2 to 0.18.3:

    thread 'main' panicked at 'Couldn't run DB Migrations: Failed to run 2023-07-08-101154_fix_soft_delete_aggregates with: syntax error at or near "trigger"', crates/db_schema/src/utils.rs:221:25 note: run with 'RUST_BACKTRACE=1' environment variable to display a backtrace

    https://github.com/LemmyNet/lemmy/issues/3756

    If you use “Lemmy from scratch” install,

    you may be on an older PostgreSQL that Ubuntu distro provides, not the same as the Docker install method.

    • katie
      link
      fedilink
      English
      107 months ago

      Yes, I had to upgrade from Postgres 13 to get this working. I’m not sure if there’s a minimum Postgres version specified anywhere in the docs, I took a quick look but couldn’t find it.

      For anyone else running Debian like me, you can add the official Postgres repo and use the pg_upgrade script to migrate to the new version.

      • RoundSparrow
        link
        English
        97 months ago

        I’m not sure if there’s a minimum Postgres version specified anywhere in the docs, I took a quick look but couldn’t find it.

        I do not find it in the “From Scratch Install” documentation. I will work to revise those documents this weekend to warn people that 0.18.3 and later is incompatible. Thank you again for sharing.

  • @anon_water
    link
    English
    137 months ago

    Thank you for all of your hard work devs! I’m a supporter with monthly payments.