Hi all. I just setup a new install from scratch on FreeBSD (write up to come once I work out all the issues) but I can’t seem to subscribe to remote communities.

I’m running the backend and UI versions 0.17.4 and you can see the instance at https://discuss.petersanchez.com

Anytime I try to search for any community (ie, !lemmy_support@lemmy.ml) it says not found. Logs show errors about values, etc. I try via full URL or via the local instance url (`/c/lemmy_support@lemmy.ml) and it’s the same. I was able to subscribe to 1 remote community but about 10 others fail to be found.

Here’s an example of an error from the logs:

2023-06-30T00:10:30.187507Z ERROR HTTP request{http.method=GET http.scheme="http" http.host=discuss.petersanchez.com http.target=/api/v3/ws otel.kind="server" request_id=bfe8111b-8a78-4bc8-8e04-d4a91ff08837 http.status_code=101 otel.status_code="OK"}: lemmy_server::api_routes_websocket: couldnt_find_object: error decoding response body: missing field `properties` at line 1 column 158
   0: lemmy_apub::fetcher::search::search_query_to_object_id
             at /home/lemmy/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lemmy_apub-0.17.4/src/fetcher/search.rs:17
   1: lemmy_apub::api::resolve_object::perform
           with self=ResolveObject { q: "!lemmy_support@lemmy.ml", auth: Some(Sensitive) }
             at /home/lemmy/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lemmy_apub-0.17.4/src/api/resolve_object.rs:21
   2: lemmy_server::root_span_builder::HTTP request
           with http.method=GET http.scheme="http" http.host=discuss.petersanchez.com http.target=/api/v3/ws otel.kind="server" request_id=bfe8111b-8a78-4bc8-8e04-d4a91ff08837 http.status_code=101 otel.status_code="OK"
             at /home/lemmy/.cargo/registry/src/index.crates.io-6f17d22bba15001f/lemmy_server-0.17.4/src/root_span_builder.rs:16

I can access my users json details with curl -H 'Accept: application/activity+json' https://discuss.petersanchez.com/u/peter just fine.

Here is my nginx config:

https://paste.sr.ht/~petersanchez/2bd686389f7d0bdd5e8645fcedbae65717344885

I’m thinking this may be the issue because I pieced this together based on past ansible-lemmy changes as the newer version seems to be geared towards 0.18.0 and doesn’t appear to pass into the UI (port 1234) any more.

Any ideas? Thanks in advance!

Apologies - This is a repost. Seems my last question got lost in all the turbulence over the last 4 days but the issue continues here.

  • terribleplan@lemmy.nrd.liEnglish
    2·
    1 year ago

    Not sure how much it helps, but that log is of your server getting bad JSON from a remote server, so your nginx config is unlikely to be the culprit there, and FWIW your nginx is quite similar to mine and I am still on 0.17.4.

    I often have to search multiple times or wait a very long time before I get results, and I have to unsub/resub multiple times before it goes through on massively overloaded instances (.world, .ml, beehaw, etc). For the sake of debugging you could try searching for a community on a smaller host so you can be sure it isn’t the remote’s fault. !admin_blog@lemmy.nrd.li if you want to use one on my instance.

    edit: Just looking at this post on lemmy.ml indicates that it is still massively overloaded as I am missing all of the other comments for this thread on my instance.

    edit2: A pending subscribe basically just means that your local isntance knows you want content from that community, not that the remote has accepted your subscription and not that anything will actually be federated. All of the Kbin magazines I have tried to subscribe to have never gotten out of “pending” and have never federated anything to me despite retrying multiple times.

    • lemmy_newbieOPEnglish
      1·
      1 year ago

      Thank you. Yea, I’ve compared with quite a few nginx.conf’s now and I’m not quite sure what the issue could be with my nginx config. Trying the community on your instance, I get this error:

      couldnt_find_community: Failed to resolve actor for admin_blog@lemmy.nrd.li

      I assume this is not DNS related because I can resolve lemmy.nrd.li without issue from the server where lemmy is running.

      • terribleplan@lemmy.nrd.liEnglish
        1·
        1 year ago

        Yeah, that (probably) isn’t a DNS resolution problem. “Failed to resolve actor” basically means it was trying to look up the actor for the community (a community is just a Group in ActivityPub parlance, so it does a WebFinger request to https://lemmy.nrd.li/.well-known/webfinger?resource=acct%3Aadmin_blog%40lemmy.nrd.li in this case to find that Group), and that request is failing. It could be some sort of network issue, I think I’ve seen someone with a similar issue due to missing their ca certs.

        Make sure you are doing all of your debugging from inside the lemmy container not just the server if you’re using containers.

        edit: Also, I had to individually load your comment with a search so I could reply, so .ml is running pretty far behind in terms of federation…

        • lemmy_newbieOPEnglish
          1·
          1 year ago

          Thanks for confirming. I am running lemmy from Supervisord and not using any containers. I’m debugging via it’s log output. This is so strange as I can’t really see anything else wrong or missing from nginx config.

          Are you running 0.18.0? I ask because I notice in lemmy-ansible now the config is different and there seems to be no proxy-ing for the UI, just the backend. Not quite sure how that is supposed to work now. I think I read that 0.18.0 has some changes from web socket to API calls but not sure if that means the UI is totally bypassed now?

          • terribleplan@lemmy.nrd.liEnglish
            1·
            1 year ago

            My server is on 0.17.4 for the moment as I am waiting for them to add back captcha support with 0.18.1 before I upgrade. AFAIK the only thing you might change_ as far as nginx would be you don’t need to do the connection upgrade stuff any more since the websockets got ripped out, all the endpoints and stuff are still the same and the routing of frontend vs backend is still basically “certain routes, POSTs, and anything with Accept of the apub content type(s) go to backend, everything else goes to frontend”. You can see the current version of their nginx config file here: https://github.com/LemmyNet/lemmy/blob/main/docker/nginx.conf