Salta al contenuto
Citiverse è uno spazio aperto a tutte le comunità. Se vuoi aprire un gruppo locale o una sezione per la tua organizzazione, puoi contattare gli amministratori: pagina dei contatti.

is this federated search engine concept good, the fediverse??

Fediverse
26 10 7
  • scritto ultima modifica di adniem@thelemmy.club
    #1

    Here is a brief concept: It is the federated/fediverse Web search engine (this may have been based on a older post i made)

    But how is it federated, that part i hope i can explain it well,

    How It May Work:

    1: Instead of a crawler, it uses the older method of A Hand curated database of links and websites (By the instance mods), And no it is not like a lemmy link aggregation community, it is designed to replace DuckDuckGo, Google, Startpage, Kagi and more,

    1.1: it is that the server admins add a entry to a database with that entry having a Link and a Name and other properties, which turns into search result, you see, and the database of Entries federate so you have more entries

    2: The instances federate, so you have more possible search results

    3: you can upvote or downvote specific search results (so if a page was SEO spam you get less of it)

    4: it has integration with the rest of the fediverse

    A Why:

    Web Index's, And Crawlers, could be locked down so privacy first search engines would be locked down and have issues

    which would cause panic, unlike social media, you cannot easily have an alternative

    A Notice:

    THIS CONCEPT HAS NO AI INVOLVED IN ITS MAKING WHATSOEVER

    i hope you understood the concept

    Tags For Mastodonians:

    #technology #fediverse #foss #opensource #freesoftware #ideas #concepts

  • Here is a brief concept: It is the federated/fediverse Web search engine (this may have been based on a older post i made)

    But how is it federated, that part i hope i can explain it well,

    How It May Work:

    1: Instead of a crawler, it uses the older method of A Hand curated database of links and websites (By the instance mods), And no it is not like a lemmy link aggregation community, it is designed to replace DuckDuckGo, Google, Startpage, Kagi and more,

    1.1: it is that the server admins add a entry to a database with that entry having a Link and a Name and other properties, which turns into search result, you see, and the database of Entries federate so you have more entries

    2: The instances federate, so you have more possible search results

    3: you can upvote or downvote specific search results (so if a page was SEO spam you get less of it)

    4: it has integration with the rest of the fediverse

    A Why:

    Web Index's, And Crawlers, could be locked down so privacy first search engines would be locked down and have issues

    which would cause panic, unlike social media, you cannot easily have an alternative

    A Notice:

    THIS CONCEPT HAS NO AI INVOLVED IN ITS MAKING WHATSOEVER

    i hope you understood the concept

    Tags For Mastodonians:

    #technology #fediverse #foss #opensource #freesoftware #ideas #concepts

    scritto ultima modifica di
    #2

    its all open source! go ahead and build your example, no one is stopping you 🥰 id love to see your implementation

  • its all open source! go ahead and build your example, no one is stopping you 🥰 id love to see your implementation

    scritto ultima modifica di
    #3

    but maybe the thing that needs a how is how we may stop evil instances willing to ruin everything

  • but maybe the thing that needs a how is how we may stop evil instances willing to ruin everything

    scritto ultima modifica di
    #4

    what?

    you put out the changes you think should be made.

    go ahead, make the changes! Its all open source. free to do as you please.

    Put a code change in to support your idea!

  • what?

    you put out the changes you think should be made.

    go ahead, make the changes! Its all open source. free to do as you please.

    Put a code change in to support your idea!

    scritto ultima modifica di adniem@thelemmy.club
    #5

    wait i have the concept

    instances can have karma if their results get constant downvotes they get put onto a low karma list for the admin to see

  • wait i have the concept

    instances can have karma if their results get constant downvotes they get put onto a low karma list for the admin to see

    scritto ultima modifica di
    #6

    go ahead :)?

  • go ahead :)?

    scritto ultima modifica di
    #7

    but what should i learn to make such a project

    like what after HTML, CSS, And Javascript

  • but maybe the thing that needs a how is how we may stop evil instances willing to ruin everything

    scritto ultima modifica di
    #8

    Most ActivityPub software allows you to defederate, no? You can always implement a way to defederate known bad actors.

  • Most ActivityPub software allows you to defederate, no? You can always implement a way to defederate known bad actors.

    scritto ultima modifica di
    #9

    but the question is

    how do you find bad actors in the first place

  • but the question is

    how do you find bad actors in the first place

    scritto ultima modifica di
    #10

    While we are talking about software that does not exist, there are Mastodon blocklists that can be gathered. At the very least, the FreeDNS blocklist seems like a reasonable bare minimum.

    If bad-faith actors are a huge concern -- and for this idea, this is a reasonable concern -- you can look into the idea of an island network.

  • While we are talking about software that does not exist, there are Mastodon blocklists that can be gathered. At the very least, the FreeDNS blocklist seems like a reasonable bare minimum.

    If bad-faith actors are a huge concern -- and for this idea, this is a reasonable concern -- you can look into the idea of an island network.

    scritto ultima modifica di
    #11

    maybe a karma system for instances where if instances get downvoted a lot they are put onto that

    and so does instances that have a large percent of users downvoting

    and a rate limiter for downvoting and upvoting

  • maybe a karma system for instances where if instances get downvoted a lot they are put onto that

    and so does instances that have a large percent of users downvoting

    and a rate limiter for downvoting and upvoting

    scritto ultima modifica di
    #12

    You just have to make sure somehow that no network of bots is downvoting all the stuff they don't like

  • Here is a brief concept: It is the federated/fediverse Web search engine (this may have been based on a older post i made)

    But how is it federated, that part i hope i can explain it well,

    How It May Work:

    1: Instead of a crawler, it uses the older method of A Hand curated database of links and websites (By the instance mods), And no it is not like a lemmy link aggregation community, it is designed to replace DuckDuckGo, Google, Startpage, Kagi and more,

    1.1: it is that the server admins add a entry to a database with that entry having a Link and a Name and other properties, which turns into search result, you see, and the database of Entries federate so you have more entries

    2: The instances federate, so you have more possible search results

    3: you can upvote or downvote specific search results (so if a page was SEO spam you get less of it)

    4: it has integration with the rest of the fediverse

    A Why:

    Web Index's, And Crawlers, could be locked down so privacy first search engines would be locked down and have issues

    which would cause panic, unlike social media, you cannot easily have an alternative

    A Notice:

    THIS CONCEPT HAS NO AI INVOLVED IN ITS MAKING WHATSOEVER

    i hope you understood the concept

    Tags For Mastodonians:

    #technology #fediverse #foss #opensource #freesoftware #ideas #concepts

    scritto ultima modifica di
    #13

    Honest question: who tf has time to hand-curate results over a crawler or index?? That's insane! I would host, except that's a lot of work.

    Maybe a way to query search results quantities to then itemize your results? Idk. But, that's just asking for a bad result, my $.02.

  • Honest question: who tf has time to hand-curate results over a crawler or index?? That's insane! I would host, except that's a lot of work.

    Maybe a way to query search results quantities to then itemize your results? Idk. But, that's just asking for a bad result, my $.02.

    scritto ultima modifica di
    #14

    well i think that's counting the entire web which nobody does visit the entire web

    but maybe i do some ways to bulk add (such as probably Regex)

    and if i think you misunderstood it isn't you curate the results for "Dogs" but you just do make entries for example an image of a dog and you do add some proprieties, and the search engine tries to find it when you search "Dogs"

  • its all open source! go ahead and build your example, no one is stopping you 🥰 id love to see your implementation

    scritto ultima modifica di
    #15

    I don't think OP is talking about adding a feature to Lemmy, this is a separate idea for a fediverse platform

  • Here is a brief concept: It is the federated/fediverse Web search engine (this may have been based on a older post i made)

    But how is it federated, that part i hope i can explain it well,

    How It May Work:

    1: Instead of a crawler, it uses the older method of A Hand curated database of links and websites (By the instance mods), And no it is not like a lemmy link aggregation community, it is designed to replace DuckDuckGo, Google, Startpage, Kagi and more,

    1.1: it is that the server admins add a entry to a database with that entry having a Link and a Name and other properties, which turns into search result, you see, and the database of Entries federate so you have more entries

    2: The instances federate, so you have more possible search results

    3: you can upvote or downvote specific search results (so if a page was SEO spam you get less of it)

    4: it has integration with the rest of the fediverse

    A Why:

    Web Index's, And Crawlers, could be locked down so privacy first search engines would be locked down and have issues

    which would cause panic, unlike social media, you cannot easily have an alternative

    A Notice:

    THIS CONCEPT HAS NO AI INVOLVED IN ITS MAKING WHATSOEVER

    i hope you understood the concept

    Tags For Mastodonians:

    #technology #fediverse #foss #opensource #freesoftware #ideas #concepts

    scritto ultima modifica di
    #16

    Are you suggesting that users manually add websites and keywords to a database?

  • Are you suggesting that users manually add websites and keywords to a database?

    scritto ultima modifica di
    #17

    yes but with some automation (such as Regex and probably more tho i haven't written that out)

  • Here is a brief concept: It is the federated/fediverse Web search engine (this may have been based on a older post i made)

    But how is it federated, that part i hope i can explain it well,

    How It May Work:

    1: Instead of a crawler, it uses the older method of A Hand curated database of links and websites (By the instance mods), And no it is not like a lemmy link aggregation community, it is designed to replace DuckDuckGo, Google, Startpage, Kagi and more,

    1.1: it is that the server admins add a entry to a database with that entry having a Link and a Name and other properties, which turns into search result, you see, and the database of Entries federate so you have more entries

    2: The instances federate, so you have more possible search results

    3: you can upvote or downvote specific search results (so if a page was SEO spam you get less of it)

    4: it has integration with the rest of the fediverse

    A Why:

    Web Index's, And Crawlers, could be locked down so privacy first search engines would be locked down and have issues

    which would cause panic, unlike social media, you cannot easily have an alternative

    A Notice:

    THIS CONCEPT HAS NO AI INVOLVED IN ITS MAKING WHATSOEVER

    i hope you understood the concept

    Tags For Mastodonians:

    #technology #fediverse #foss #opensource #freesoftware #ideas #concepts

    scritto ultima modifica di
    #18

    You may be interested in YaCy

  • You may be interested in YaCy

    scritto ultima modifica di
    #19

    but that's not YaCy

    it is a static databse, there's no crawler

  • but that's not YaCy

    it is a static databse, there's no crawler

    scritto ultima modifica di
    #20

    Without crawling a website, a search engine cannot know content of the website.

    You are effectively reinventing an web directory.

    Maybe it could be done as wiki or even federated one like ibis.wiki.

    Another prior art would be many pinboards


Citiverse è un progetto che si basa su NodeBB ed è federato! | Categorie federate | Chat | 📱 Installa web app o APK | 🧡 Donazioni | Privacy Policy

Il server utilizzato è quello di Webdock, in Danimarca. Se volete provarlo potete ottenere il 20% di sconto con questo link e noi riceveremo un aiuto sotto forma di credito da usare proprio per mantenere Citiverse.