Hi all. I just setup a new install from scratch on FreeBSD (write up to come once I work out all the issues) but I can’t seem to subscribe to remote communities.

I’m running the backend and UI versions 0.17.4 and you can see the instance at https://discuss.petersanchez.com

Anytime I try to search for any community (ie, !lemmy_support@lemmy.ml) it says not found. Logs show errors about values, etc. I try via full URL or via the local instance url (`/c/lemmy_support@lemmy.ml) and it’s the same. I was able to subscribe to 1 remote community but about 10 others fail to be found.

Here’s an example of an error from the logs:

2023-06-30T00:10:30.187507Z ERROR HTTP request{http.method=GET http.scheme="http" http.host=discuss.petersanchez.com http.target=/api/v3/ws otel.kind="server" request_id=bfe8111b-8a78-4bc8-8e04-d4a91ff08837 http.status_code=101 otel.status_code="OK"}: lemmy_server::api_routes_websocket: couldnt_find_object: error decoding response body: missing field `properties` at line 1 column 158
   0: lemmy_apub::fetcher::search::search_query_to_object_id
             at /home/lemmy/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lemmy_apub-0.17.4/src/fetcher/search.rs:17
   1: lemmy_apub::api::resolve_object::perform
           with self=ResolveObject { q: "!lemmy_support@lemmy.ml", auth: Some(Sensitive) }
             at /home/lemmy/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lemmy_apub-0.17.4/src/api/resolve_object.rs:21
   2: lemmy_server::root_span_builder::HTTP request
           with http.method=GET http.scheme="http" http.host=discuss.petersanchez.com http.target=/api/v3/ws otel.kind="server" request_id=bfe8111b-8a78-4bc8-8e04-d4a91ff08837 http.status_code=101 otel.status_code="OK"
             at /home/lemmy/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lemmy_server-0.17.4/src/root_span_builder.rs:16

I can access my users json details with curl -H 'Accept: application/activity+json' https://discuss.petersanchez.com/u/peter just fine.

Here is my nginx config:

https://paste.sr.ht/~petersanchez/2bd686389f7d0bdd5e8645fcedbae65717344885

I’m thinking this may be the issue because I pieced this together based on past ansible-lemmy changes as the newer version seems to be geared towards 0.18.0 and doesn’t appear to pass into the UI (port 1234) any more.

Any ideas? Thanks in advance!

Apologies - This is a repost. Seems my last question got lost in all the turbulence over the last 4 days but the issue continues here.

  • Dru5k1@lem.dru5k1.com
    link
    fedilink
    English
    arrow-up
    1
    ·
    1 year ago

    The fact that you can subscribe to one community suggests that your lemmy setup works correctly and the issue may be external, maybe DNS. Are you able to provide more logs? What does your lemmy.hjson look like? That one community you’ve subscribed to, does it show a green subscribed button and load its posts, votes and comments?

    Oh and really dumb question, have you enabled federation in the server settings?

    • lemmy_newbieOP
      link
      fedilink
      English
      arrow-up
      1
      ·
      1 year ago

      Thank you for the help so far! Another user successfully sent me a message so it seems some things are working. I do have federation enabled in settings (see image)

      Do I need to specifically list all the instances I want to federate with? That seems counter intuitive as by default I’d like to federate with all instances and just block the ones that give me a headache.

      My lemmy.hjson looks like this:

      {
        database: {
          password: "NOT REAL PASSWD"
        }
        # replace with your domain
        hostname: discuss.petersanchez.com
        bind: "127.0.0.1"
        federation: {
          enabled: true
        }
        # remove this block if you don't require image hosting
        pictrs: {
          url: "http://localhost:8080/"
        }
      }
      

      I don’t even know what other options are available for this file. I can’t seem to find any docs for it. Let me know if something should be added/removed/changed.

      DNS seems OK on the system when trying to resolve all the given communities from a server shell. I know ActivityPub isn’t for the faint of heart but in my toying around with it in the past, it was straight forward enough, just different software use different fields for different purposes. So that can create confusion in federation but in this case I’m dealing with Lemmy to Lemmy so I assume it should work heh.

      Oh and the 1 community I was able to actually find is on kbin.social and it says subscribe pending:

      Thanks again for the help so far!

    • Dru5k1@lem.dru5k1.com
      link
      fedilink
      English
      arrow-up
      1
      ·
      edit-2
      1 year ago

      Right, I’ve moved to my PC and your post makes more sense now.

      It seems like the problem section of your logs is couldnt_find_object: error decoding response body: missing field properties at line 1 column 158. I’m thinking it may have something to do with your ngnix.conf because it might be implying that it can’t see anything at all, rather than just a missing field property.

      I’ve shared my nginx.conf below in the hope that, by comparison you can get your instance to federate. (This is from my instance that is federating fully and correctly).

      worker_processes auto;
      events {
          worker_connections 1024;
      }
      http {
          upstream lemmy {
              # this needs to map to the lemmy (server) docker service hostname
              server "lemmy:8536";
          }
          upstream lemmy-ui {
              # this needs to map to the lemmy-ui docker service hostname
              server "lemmy-ui:1234";
          }
      
          server {
              # this is the port inside docker, not the public one yet
              listen 1236;
              listen 8536;
              # change if needed, this is facing the public web
              server_name localhost;
              server_tokens off;
      
              gzip on;
              gzip_types text/css application/javascript image/svg+xml;
              gzip_vary on;
      
              # Upload limit, relevant for pictrs
              client_max_body_size 20M;
      
              add_header X-Frame-Options SAMEORIGIN;
              add_header X-Content-Type-Options nosniff;
              add_header X-XSS-Protection "1; mode=block";
      
              # frontend general requests
              location / {
                  # distinguish between ui requests and backend
                  # don't change lemmy-ui or lemmy here, they refer to the upstream definitions on top
                  set $proxpass "http://lemmy-ui";
      
                  if ($http_accept = "application/activity+json") {
                    set $proxpass "http://lemmy";
                  }
                  if ($http_accept = "application/ld+json; profile=\"https://www.w3.org/ns/activitystreams\"") {
                    set $proxpass "http://lemmy";
                  }
                  if ($request_method = POST) {
                    set $proxpass "http://lemmy";
                  }
                  proxy_pass $proxpass;
      
                  rewrite ^(.+)/+$ $1 permanent;
                  # Send actual client IP upstream
                  proxy_set_header X-Real-IP $remote_addr;
                  proxy_set_header Host $host;
                  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
              }
      
              # backend
              location ~ ^/(api|pictrs|feeds|nodeinfo|.well-known) {
                  proxy_pass "http://lemmy";
                  # proxy common stuff
                  proxy_http_version 1.1;
                  proxy_set_header Upgrade $http_upgrade;
                  proxy_set_header Connection "upgrade";
      
                  # Send actual client IP upstream
                  proxy_set_header X-Real-IP $remote_addr;
                  proxy_set_header Host $host;
                  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
              }
          }
      }
      
      • Dru5k1@lem.dru5k1.com
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        I’ve had a look at your instance and the community you’ve said you’re subscribed to… you’re not, there’s no posts or anything. It may show subscribed but it’s lying to you.

        The UI version is showing as unknown version which is strange. I also can’t sign up which is doubly strange. So, there’s definitely an issue with your setup. I’d also guess that there’s an issue with network connectivity between your instance and the www. I can’t say anymore without further info. I’m sorry I couldn’t be of more help. I’m happy to help out more though, from one instance owner to another, drop me a message over at https://iam.dru5k1.com/mastodon. You know the saying two heads are better than one.

      • lemmy_newbieOP
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        Thank you for sharing your config. Mine was nearly exactly identical. I updated some of the headers (yours were slightly different than mine) but that doesn’t seem to have made any difference. I will try enabling federation debug and see what the logs show