Salta al contenuto
Citiverse è uno spazio aperto a tutte le comunità. Se vuoi aprire un gruppo locale o una sezione per la tua organizzazione, puoi contattare gli amministratori: pagina dei contatti.

TIL the fediverse basically DDOS's sites when you share a link.

Fediverse
141 78 0
  • I just took this random post from my frontpage: https://sh.itjust.works/post/68605733
    and made a manual request for the federated JSON-LD.

    {
      "@context": [
        "https://join-lemmy.org/context.json",
        "https://www.w3.org/ns/activitystreams"
      ],
      "type": "Page",
      "id": "https://sh.itjust.works/post/68605733",
      "attributedTo": "https://sh.itjust.works/u/derrickoswald",
      "to": [
        "https://lemmy.world/c/technology",
        "https://www.w3.org/ns/activitystreams#Public"
      ],
      "name": "Stratospheric airship flew from US to Japan and back",
      "cc": [],
      "content": "<p>It seems with an unmanned airship they should be using hydrogen for the ~8% increased payload capacity.</p>\n",
      "mediaType": "text/html",
      "source": {
        "content": "It seems with an unmanned airship they should be using hydrogen for the ~8% increased payload capacity.",
        "mediaType": "text/markdown"
      },
      "attachment": [
        {
          "href": "https://arstechnica.com/gadgets/2026/10/trans-pacific-journey-tested-stratospheric-cell-tower-concept-in-japan/",
          "mediaType": "text/html; charset=utf-8",
          "type": "Link"
        }
      ],
      "image": {
        "type": "Image",
        "url": "https://sh.itjust.works/pictrs/image/df608ab3-b644-489d-b217-9224aa09e203.jpeg"
      },
      "sensitive": false,
      "published": "2026-10-19T20:16:37.383385Z",
      "language": {
        "identifier": "en",
        "name": "English"
      },
      "audience": "https://lemmy.world/c/technology",
      "tag": [
        {
          "href": "https://sh.itjust.works/post/68605733",
          "name": "#technology",
          "type": "Hashtag"
        }
      ]
    }
    

    Notice the

    "image": {
        "type": "Image",
        "url": "https://sh.itjust.works/pictrs/image/df608ab3-b644-489d-b217-9224aa09e203.jpeg"
      }
    

    Yes the image itself isn't sent over (iirc they never are?), but it's stored on sh.itjust.works. Other instances grab it from there, not the website the post links to.

    I did notice just now though that some Lemmy instances seem to just proxy the image through their servers, which I assume is to avoid CORS. But that's an instance choice, not a general fediverse issue.

    Mbin does the same as Lemmy, but it also includes the image in the attachment property. Probably for compatibility with some other platform.

    One thing I didn't notice before was that you're using Piefed. That's the one software I didn't check before writing my previous comment. It does seem like Piefed is the outlier here. I tried a post from piefed.ca and the image is nowhere to be seen. The image property is not present, it's not in the attachment array, and I can't see any other property that contains it.

    Edit: A piefed.social post I just tried did have an image, so I assume I'm just blind.

    scritto ultima modifica di
    #61

    Rimu posted a parent comment elsewhere regarding PieFed and Lemmy.

  • I mean, just how many instances are gonna ping the URL? Several hundred, low thousands at worst?

    Unless it happens to be some very large media content or a dynamic page with no caching, I'd think the automated federation hits shouldn't be too bad.

    The influx of users that might come after a viral post though, that's rough....

    scritto ultima modifica di
    #62

    It's just bad design. What happens when/if the fediverse scales up to many more instances?

  • they could include the link preview data as part of the federation

    scritto ultima modifica di
    #63

    Perhaps a modification to the ActivityPub specification?

  • There are two proposed solutions I have seen. I do not claim that this is exhaustive, just only limited to the proposals that I have stumbled across.

    FASP - Fediverse Auxiliary Service Provider

    The idea that mastodon has is to have a third-party service that does something like generate a link preview once and then share that to fediverse instances that need it. They have a whole spec they wrote up about these kinds of services that can be run to help out fediverse instances with different things like this, search, or discoverability. They call these FASPs - the repo with the spec is here.

    I don't yet know of any FASP that is live and running, so it is kind of a paper spec at the moment. I don't do too much on the microblogging side of the fediverse though, so I very well could be wrong.

    Federate link metadata and the post at the same time

    This comes from an issue on PieFed's repo. It is basically just an idea at this point, but it is a pretty basic one; have the originating instance fetch the link preview information and simply federate it out alongside the post itself.

    scritto ultima modifica di
    #64

    It could have been FAPS, what a missed opportunity.

  • It's almost like the software that powers the internet was designed for a different time and is only still used because there's monied interest in keeping it around. We need GNUnet.

    scritto ultima modifica di
    #65

    GNUnet helps building a new Internet

    GNUnet is a network protocol stack for building secure, distributed, and privacy-preserving applications.

    With strong roots in academic research, our goal is to replace the old insecure Internet protocol stack.

    GNUnet provides privacy-by design

    It provides [improving addressing, routing, naming and content distribution](https://www.w3.org/2014/strint/papers/65.pdf) in a technically robust manner - as opposed to ad-hoc designs in place today.

  • I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

    I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

    This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

    E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

    scritto ultima modifica di
    #66

    I have my blog address (which goes to my home server) in my Mastodon profile, and every time I post on Masto, my server gets a request from every Mastodon site that federates with my instance - i.e. hundreds of requests. It's no problem for my server, because my blog is a static site, but for anything that dynamically generates every single page impression (like worpress or joomla), it could be a problem. Caching settings might help, I suppose.

    Truth is, though, I get far more traffic from vulnerability scanners and AI crawlers that don't honour robots.txt (or aren't yet listed).

  • GNUnet helps building a new Internet

    GNUnet is a network protocol stack for building secure, distributed, and privacy-preserving applications.

    With strong roots in academic research, our goal is to replace the old insecure Internet protocol stack.

    GNUnet provides privacy-by design

    It provides [improving addressing, routing, naming and content distribution](https://www.w3.org/2014/strint/papers/65.pdf) in a technically robust manner - as opposed to ad-hoc designs in place today.

    scritto ultima modifica di
    #67

    GNUnet provides privacy-by design

    Money doesn't want us to have privacy.

  • I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

    I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

    This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

    E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

    scritto ultima modifica di
    #68

    Thanks for bringing this up. I think in general we need to work on optimisation of Activitypub and Threadiverse implementations of it.

  • There are two proposed solutions I have seen. I do not claim that this is exhaustive, just only limited to the proposals that I have stumbled across.

    FASP - Fediverse Auxiliary Service Provider

    The idea that mastodon has is to have a third-party service that does something like generate a link preview once and then share that to fediverse instances that need it. They have a whole spec they wrote up about these kinds of services that can be run to help out fediverse instances with different things like this, search, or discoverability. They call these FASPs - the repo with the spec is here.

    I don't yet know of any FASP that is live and running, so it is kind of a paper spec at the moment. I don't do too much on the microblogging side of the fediverse though, so I very well could be wrong.

    Federate link metadata and the post at the same time

    This comes from an issue on PieFed's repo. It is basically just an idea at this point, but it is a pretty basic one; have the originating instance fetch the link preview information and simply federate it out alongside the post itself.

    scritto ultima modifica di
    #69

    But what happens when a link target changes? For example, I've seen news articles change as the author gets new information. Might be an edge case, but a solution like this would always show the preview at the time of the initial post.

  • It's almost like the software that powers the internet was designed for a different time and is only still used because there's monied interest in keeping it around. We need GNUnet.

    scritto ultima modifica di
    #70

    Now this is interesting. Thanks for sharing

  • It's just bad design. What happens when/if the fediverse scales up to many more instances?

    scritto ultima modifica di
    #71

    I liked writing efficient code but every place I worked didn't care at all about it. All they wanted was a finished feature and a kanban note moved to finished.

  • It's almost like the software that powers the internet was designed for a different time and is only still used because there's monied interest in keeping it around. We need GNUnet.

    scritto ultima modifica di
    #72

    is there something that makes this different to 17 other proposals for a new internet or is it just a case of
    https://xkcd.com/927/

  • I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

    I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

    This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

    E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

    scritto ultima modifica di
    #73

    Is there a reason why the publishing instance is not downloading the preview image, and then submits it with the post to the other instances? This way they would not have to fetch it themselves.

  • Is there a reason why the publishing instance is not downloading the preview image, and then submits it with the post to the other instances? This way they would not have to fetch it themselves.

    scritto ultima modifica di
    #74

    Yeah, thats how it should be: one download, everyone else downloads from that server, not the original source.

    Seems abusable though

  • I have my blog address (which goes to my home server) in my Mastodon profile, and every time I post on Masto, my server gets a request from every Mastodon site that federates with my instance - i.e. hundreds of requests. It's no problem for my server, because my blog is a static site, but for anything that dynamically generates every single page impression (like worpress or joomla), it could be a problem. Caching settings might help, I suppose.

    Truth is, though, I get far more traffic from vulnerability scanners and AI crawlers that don't honour robots.txt (or aren't yet listed).

    scritto ultima modifica di
    #75

    that don’t honour robots.txt

    That's essentially all of them in my experience.

  • Did I say "shouldn't be"? Could have sworn i said "is"...

    scritto ultima modifica di
    #76

    The image isn't federated, but [I believe] it should be.

    But I am already thinking it shouldn't be because of hosting concerns.

  • Yeah, thats how it should be: one download, everyone else downloads from that server, not the original source.

    Seems abusable though

    scritto ultima modifica di
    #77

    Add metadata with a checksum of the og one with signature and let each client check that up.

  • Yeah, thats how it should be: one download, everyone else downloads from that server, not the original source.

    Seems abusable though

    scritto ultima modifica di
    #78

    Seems abusable though

    Exactly

  • Yeah, thats how it should be: one download, everyone else downloads from that server, not the original source.

    Seems abusable though

    scritto ultima modifica di
    #79

    Seems abusable though

    If one instance doesn't trust another, they shouldn't federate in the first place.

  • Add metadata with a checksum of the og one with signature and let each client check that up.

    scritto ultima modifica di
    #80

    If it's a signed checksum then maybe. But if the og instance is doing the abusing then it won't help.


Citiverse è un progetto che si basa su NodeBB ed è federato! | Categorie federate | Chat | 📱 Installa web app o APK | 🧡 Donazioni | Privacy Policy

Il server utilizzato è quello di Webdock, in Danimarca. Se volete provarlo potete ottenere il 20% di sconto con questo link e noi riceveremo un aiuto sotto forma di credito da usare proprio per mantenere Citiverse.