Australian urban planning, public transport, politics, retrocomputing, and tech nerd. Recovering journo. Cat parent. Part-time miserable grump.

Cities for people, not cars! Tech for people, not investors!

  • 11 Posts
  • 4 Comments
Joined 2 years ago
cake
Cake day: November 5th, 2022

help-circle





  • @makeasnek On a broader note, I think possibly the best approach for decentralised, open-sourced web search might be an evolution on the SearXNG model.

    At the top of the funnel, you have meta search engines that query and aggregate results from a number of smaller niche search engines.

    The metasearch engines are open source, anyone with a spare server or a web hosting account can spin one up.

    For some larger sites that are trustworthy, such as Wikipedia, the site’s own search engine might be what’s queried.

    For the Fediverse and other similar federated networks, the query is fed through a trusted node on the network.

    And then there’s a host of smaller niche search engines, which only crawl and index pages on a small number of websites vetted and curated by a human.

    (Perhaps on a particular topic? Or a local library or university might curate a list of notable local websites?)

    (Alternatively, it might be that a crawler for a web index like Curlie.org only crawls websites chosen by its topic moderators.)

    In this manner, you could build a decent web search engine without needing the scale of Google or Microsoft.