Matt Cloy (@cloy@techhub.social)
techhub.social
So I made a "pending review" decision on the fediverse full-text search engine we wrote - uses the public API, which means it can't be defederated, and it fetches from a range of dynamic IPs, so please don't try relying on IP blocks, filling out robots.txt is the solution for hosts, or as a user set your profile to do-not-index on mastodon and/or add #noindex to your bio). Available under login ONLY to *verified* instance moderators (and only searching federated instances of that mod). I.E. if the server is defederated from your instance, their mods can't search the commons for anything. *Constructive* feedback on this welcome (including thoughts on adding watch-phrases for flagging abuse patterns for review, making robots.txt-banned instances public, or anything else that improves moderation), please let me know NOW not later. Would rather a discussion before the cat is out of the bag than afterwards. #fediblock (because I know that hashtag will get me feedback) #flameproofpantstime

So, this appears to be a third party site that aggregates information from multiple instances via public API and then allows full text searching on them. To be implemented in a way that doesn’t allow defederation, yet claims to respect blocking.

Technical details are still vague

  • @0x1C3B00DA
    1•1 year ago

    I think full text search should be built into all fediverse software. We have privacy controls on posts and accounts. As long as implementations respect that, I don’t see the issue. Eventually, people are going to build these without asking for discussion so, to me, it makes more sense to build it into the software in a privacy respecting way to prevent the need for third party indices.

    • AdaOP
      1•1 year ago

      Yeah, full text search is already built in to many fediverse platforms. Misskey and its forks come with full text search out of the box for example, but they are limited by the federation reach of the instance they run on. This is something else, because it is a search tool trying to search the fediverse independent of blocklists and federation reach.

      • @0x1C3B00DA
        1•1 year ago

        a search tool trying to search the fediverse independent of blocklists and federation reach.

        Isn’t that just a basic search engine? I also don’t understand the issue with that. As a user, you can post privately or even mark your whole profile as #noindex. Instances can also setup a robots.txt to prevent indexing of some or all of the instance.

        • AdaOP
          1•1 year ago

          The difference is that current full text searches only searches content that has federated to them, because they are built in to the instance. So blocking etc limits their scope. Those searches are also only available to members of the instance.

          This tool will make searches available to anyone, and rather than searching content that has already federated to the server, it will be actively pulling in data explicitly to search, in a way that isn’t restrained by blocklists.

          Full text search already exists, but it’s limited in scope. This is aiming to increase the scope.