Salta al contenuto
Citiverse è uno spazio aperto a tutte le comunità. Se vuoi aprire un gruppo locale o una sezione per la tua organizzazione, puoi contattare gli amministratori: pagina dei contatti.

TIL the fediverse basically DDOS's sites when you share a link.

Fediverse
141 78 0
  • I don't really get it. The image is federated, there's no need for instances to refetch it. Just checked out some posts and indeed it's there, in the image property. With the URL pointing to the Lemmy instance.

    So if this is actually an issue, that's just bad programming and needs to be fixed.

    Edit: didn't notice OP uses Piefed. Unlike Lemmy and Mbin, it seems like Piefed doesn't federate it.

    Edit: actually I'm blind, Piefed does federate it too, sorry piefed

    scritto ultima modifica di
    #41

    The image is not federated.

  • https://en.wikipedia.org/wiki/Slashdot_effect but people were going in, looking at the link, sometimes overwhelming the server

    scritto ultima modifica di
    #42

    That's not at all the same thing. That's just the inability to handle large amounts of unexpected normal user traffic.

    This is essentially bot traffic driven by servers all independently but simultaneously fetching the same image, without even a single human necessarily looking at it.

  • That's not at all the same thing. That's just the inability to handle large amounts of unexpected normal user traffic.

    This is essentially bot traffic driven by servers all independently but simultaneously fetching the same image, without even a single human necessarily looking at it.

    scritto ultima modifica di
    #43

    From server's perspective the only difference between those is the user-agent. Traffic is the same

  • The image is not federated.

    scritto ultima modifica di
    #44

    the image should be federated

  • From server's perspective the only difference between those is the user-agent. Traffic is the same

    scritto ultima modifica di
    #45

    But what OP is talking about is a technical detail that can be fixed.

  • It was my first thought too but there are downsides to it. Trust is a small one given the nature of the system but compute and bandwidth costs could be an issue also, especially for larger instances.

    The first solution is more fragile but could be CDN'd and those are cheap.

    scritto ultima modifica di
    #46

    I don't think compute resources should be an issue, if anything it saves other instances resources.

    As far as I understand, right now, all instances are fetching this data individually, so they all have the same compute load from this. Doing it on one and sending it along with federation should keep resource useage pretty much the same on the originating instance, and less for all others.

    This would still increase bandwidth of course, but at least with all VPS providers I have used, bandwith is pretty cheap.

  • Lemmy does this to itself, too - when you create a post it federates out and then every Lemmy instance that receives it makes a GET request to retrieve the post from the original server. Even though it just received the exact same data. Some other fedi software is even worse - Akkoma does a GET of the id of every activity it receives, not just for posts but for Likes too.

    It's insane.

    Anyway, proper caching at the reverse proxy / Cloudflare level solves it. The same kind of caching setup would do the same for sites serving HTML, like NPR too.

    scritto ultima modifica di
    #47

    That's weird. Is that how it's supposed to work, or did they just implement it in a bad way?

  • From server's perspective the only difference between those is the user-agent. Traffic is the same

    scritto ultima modifica di
    #48

    The origin and purpose of the traffic is not the same. That's why they block one and not the other.

  • the image should be federated

    scritto ultima modifica di
    #49

    Did I say "shouldn't be"? Could have sworn i said "is"...

  • Do they block the useragent or the URL? NPR apparently blocks the former.

    scritto ultima modifica di
    #50

    You're talking about 2 different kinds of blocking here. NPR can't block a URL, that concept doesn't really exist at this level. They could block ip addresses, or user agents as you mentioned.

    A workplace can't block by user agent (at least not on any website using TLS), as this header is not readable by any intermediary devices.

  • The image is not federated.

    scritto ultima modifica di
    #51

    Are you sure? In your post you're talking about fetching an image for a link preview, not an image post.

    Usually, when I see an image post, the source is actually the pictrs of the source instance, so in that case I think it is federated.

  • Lemmy does this to itself, too - when you create a post it federates out and then every Lemmy instance that receives it makes a GET request to retrieve the post from the original server. Even though it just received the exact same data. Some other fedi software is even worse - Akkoma does a GET of the id of every activity it receives, not just for posts but for Likes too.

    It's insane.

    Anyway, proper caching at the reverse proxy / Cloudflare level solves it. The same kind of caching setup would do the same for sites serving HTML, like NPR too.

    scritto ultima modifica di
    #52

    How else can you be sure the content is valid though? Lemmy posts are sent as Announces from the community, not from the actor posting the post. The community could just fake a post if you just trust it.

  • they could include the link preview data as part of the federation

    scritto ultima modifica di
    #53

    How do you know that data is authentic?

  • You're talking about 2 different kinds of blocking here. NPR can't block a URL, that concept doesn't really exist at this level. They could block ip addresses, or user agents as you mentioned.

    A workplace can't block by user agent (at least not on any website using TLS), as this header is not readable by any intermediary devices.

    scritto ultima modifica di
    #54

    NPR can't block a URL

    I wasn't talking about NPR, I was talking about their employer...

  • NPR can't block a URL

    I wasn't talking about NPR, I was talking about their employer...

    scritto ultima modifica di
    #55

    You mentioned both in the same comment...

    NPR apparently blocks the former

    I'm just saying, their employer can't block like NPR does, because they are fundamentally different things. The one is a server receiving a request, the other is a firewall seeing encrypted packets.

    Edit: the other could also be a DNS resolver that just does not resolve certain domains.

  • Are you sure? In your post you're talking about fetching an image for a link preview, not an image post.

    Usually, when I see an image post, the source is actually the pictrs of the source instance, so in that case I think it is federated.

    scritto ultima modifica di
    #56

    No one was talking about an image post...

  • But what OP is talking about is a technical detail that can be fixed.

    scritto ultima modifica di
    #57

    Yes, but then a comment mentioned the "hug of death" - high spike of requests resulting from a link being shared on some site, and OP asked why centralized sites were also creating problems

    There was another post regarding previews some months/(knowing me probably a year) ago and personally I would go with federating previews. With a button for mods to refresh it from source for when something goes fucky

    But social networks have been creating random bursts of requests, this way or another, even in the times of centralized socials

  • No one was talking about an image post...

    scritto ultima modifica di
    #58

    I figured a post with a link with preview image wouldn't be the same in terms of event as an image post. Turns out it pretty much is, but also, I checked a few posts and cross-posts across instances, all of them have an image url of the original instance, not the one I'm viewing on. So if they are individually fetching this image, they're not using it.

    Edit: I was wrong, two of the crossposts just didn't respond to the request for activity, so I was looking at the wrong response data

  • I shared a link from NPR Tiny Desk but there was no image preview or video embed. After investigating this I learned that when you do that, each instance picks up the link and individually retrieves the link preview nearly simultaneously. For gargantuan sites like YT or Reddit, this is a trivial amount of traffic and not a problem, but for smaller sites like NPR this results in a sort of botnet DDOS attack on the site's servers. More here. Another example here . For that reason, NPR has blocked Fediverse servers like Mastodon.

    I have experienced this on my personal server as well (DDOS), but at the time didn't know why, exactly.

    This isn't the end of the world, but certainly is not great and hurts engagement with the content. Is there a solution for this? Perhaps using some sort of webtorrent service like PeerTube where they just fetch the information from another fediverse server instead of the source server? Maybe fetching embeds client-side? At least that would stagger the requests.

    E: Many people have replied comparing this to the Reddit "hug of death" or "The Slashdot effect". This is not the same at all. Those are legitimate site visits from humans that happen over the course of several hours. This is the result of several thousand servers all automatically fetching the same image simultaneously , and without any human involvement, and potentially next to zero human traffic to the site, which is why I compared it to a botnet or DDOS attack.

    scritto ultima modifica di
    #59

    It's almost like the software that powers the internet was designed for a different time and is only still used because there's monied interest in keeping it around. We need GNUnet.

  • The image is not federated.

    scritto ultima modifica di
    #60

    I just took this random post from my frontpage: https://sh.itjust.works/post/68605733
    and made a manual request for the federated JSON-LD.

    {
      "@context": [
        "https://join-lemmy.org/context.json",
        "https://www.w3.org/ns/activitystreams"
      ],
      "type": "Page",
      "id": "https://sh.itjust.works/post/68605733",
      "attributedTo": "https://sh.itjust.works/u/derrickoswald",
      "to": [
        "https://lemmy.world/c/technology",
        "https://www.w3.org/ns/activitystreams#Public"
      ],
      "name": "Stratospheric airship flew from US to Japan and back",
      "cc": [],
      "content": "<p>It seems with an unmanned airship they should be using hydrogen for the ~8% increased payload capacity.</p>\n",
      "mediaType": "text/html",
      "source": {
        "content": "It seems with an unmanned airship they should be using hydrogen for the ~8% increased payload capacity.",
        "mediaType": "text/markdown"
      },
      "attachment": [
        {
          "href": "https://arstechnica.com/gadgets/2026/10/trans-pacific-journey-tested-stratospheric-cell-tower-concept-in-japan/",
          "mediaType": "text/html; charset=utf-8",
          "type": "Link"
        }
      ],
      "image": {
        "type": "Image",
        "url": "https://sh.itjust.works/pictrs/image/df608ab3-b644-489d-b217-9224aa09e203.jpeg"
      },
      "sensitive": false,
      "published": "2026-10-19T20:16:37.383385Z",
      "language": {
        "identifier": "en",
        "name": "English"
      },
      "audience": "https://lemmy.world/c/technology",
      "tag": [
        {
          "href": "https://sh.itjust.works/post/68605733",
          "name": "#technology",
          "type": "Hashtag"
        }
      ]
    }
    

    Notice the

    "image": {
        "type": "Image",
        "url": "https://sh.itjust.works/pictrs/image/df608ab3-b644-489d-b217-9224aa09e203.jpeg"
      }
    

    Yes the image itself isn't sent over (iirc they never are?), but it's stored on sh.itjust.works. Other instances grab it from there, not the website the post links to.

    I did notice just now though that some Lemmy instances seem to just proxy the image through their servers, which I assume is to avoid CORS. But that's an instance choice, not a general fediverse issue.

    Mbin does the same as Lemmy, but it also includes the image in the attachment property. Probably for compatibility with some other platform.

    One thing I didn't notice before was that you're using Piefed. That's the one software I didn't check before writing my previous comment. It does seem like Piefed is the outlier here. I tried a post from piefed.ca and the image is nowhere to be seen. The image property is not present, it's not in the attachment array, and I can't see any other property that contains it.

    Edit: A piefed.social post I just tried did have an image, so I assume I'm just blind.


Citiverse è un progetto che si basa su NodeBB ed è federato! | Categorie federate | Chat | 📱 Installa web app o APK | 🧡 Donazioni | Privacy Policy

Il server utilizzato è quello di Webdock, in Danimarca. Se volete provarlo potete ottenere il 20% di sconto con questo link e noi riceveremo un aiuto sotto forma di credito da usare proprio per mantenere Citiverse.