Citiverse
  • Lovely #fediverse does anybody have a list of all the known user-agents for #activitypub software ?

    General Discussion
    6 13 0

    kc@social.coopK
    7
    0

    Lovely does anybody have a list of all the known user-agents for software ?

    I have some WAF rules to write to make sure my server can be seen behind a frontend that is only meant to battle AI crawlers and I rather not (knowingly) leave endpoints open to OpenAI

  • smallcircles@social.coopS
    96
    0

    @kc dunno if such a list exists, but it would be useful to have one.

    PS. I maintain the 3 fediverse-related curated delightful lists. There's a ton of projects, so it'd be a sizable task to draft such list. See https://delightful.coding.social

  • reiver@mastodon.socialR
    21
    0

    @kc

    I doubt it exists yet.

    But, I do suspect some Fediverse software doesn't set a User-Agent.

  • kc@social.coopK
    7
    0

    @smallcircles that's what I'm thinking. I see a lot of random go and other default package UAs, some are legit fediverse servers and others are skriptkiddies that zone will always be very gray.

    I guess the best is a little bit of a baseline to work with and the rest will come out in the logs.

    I'll check out your project, it looks interesting

  • fentiger@mastodon.socialF
    1
    0

    @kc @smallcircles There are quite a few instances out there that will refuse to answer requests unless they have a non-default User-Agent. I think it's generally done by blocking known defaults, rather than allowing known non-defaults.

  • kc@social.coopK
    7
    0

    @FenTiger @smallcircles normally yes, my WAF is paranoid psychotic particularly on aws, google cloud, azure, and digitalocean (99% of the internet's problems come from these ranges).

    Problem here is that there are instances that exist in those IP spaces, that's why I'm tweaking in user agents and a couple of other checks before it hits the line "fuck-bezos-gates-and-zuckerberg" 😅

  • kc@social.coopK
    7
    0

    @uriel I wish it was that easy, you'll be surprised how many "legitimate" users, UAs, and retail IP ranges are currently eating shit in my tarpit.

    The obvious ones are blocked on three levels at this point, I'm all about making new problems for the smarter ones (and for myself, obviously)

  • kc@social.coopK
    7
    0

    @uriel the idea is more understand what we know of in the fedi, open the doors to them, grab the new ones every so often from the logs to adapt, while not letting the capitalist pigs eat all the food in the room because the door wasn't closed enough

  • uriel@bbs.keinpfusch.netU
    6
    0

    @kc Sorry, I am not into crusades, of any sort.

  • julian@activitypub.spaceJ
    174
    0

    @kc@social.coop my understanding is there is no known standard for UA.

    NodeBB just sends NodeBB/ ()

    Pretty personalized for NodeBB 🙂

  • kc@social.coopK
    7
    0

    @julian accidental standard perhaps, it looks like most start (PACKAGE)/(version 0.0.0) - eg Plume/0.7.2, Mastodon/(a little bit more), Pixelfed/…

    That is a good starting point for me to test and see what sticks. Thanks !

  • julian@activitypub.spaceJ
    174
    0

    @kc@social.coop oh that's just the recommendation described here 🙂

  • kc@social.coopK
    7
    0

    @julian seems a lot of people followed it when developping 😅

    I caught a couple of randoms (I totally forgot about Friendica) in the bot validator after deploying the new rules, looks like it's safe enough to push in that format to avoid filtering out the fediverse


Citiverse è un progetto che si basa su NodeBB ed è federato! | Categorie federate | Chat | 📱 Installa web app o APK | 🧡 Donazioni | Privacy Policy

Il server utilizzato è quello di Webdock, in Danimarca. Se volete provarlo potete ottenere il 20% di sconto con questo link e noi riceveremo un aiuto sotto forma di credito da usare proprio per mantenere Citiverse.