@evan it was a very leading question coming from a guy in tech, forgive me if I have you confused with a scraper bro

@rey

You can read about me here: https://en.wikipedia.org/wiki/Evan_Prodromou

Feel free to ask me any questions you have about my work on the Fediverse.

@evan @rey i imagine a less combative version of the answer could be something like "yes, provided that everyone gets to consent to being part of that new thing"

@rnd @evan yes

given that 90% of "code that does new things on the Fediverse" consists of new, exciting ways to scrape content without consent, you may understand my concerns.

Thanks all. I see a lot of developers getting brigaded for launching new tools on the Fediverse. At best, they're insulted and their motives are questioned. At worst, they're brigaded on GitHub, the Fediverse, or in real life.

Many of the replies mentioned "scrapers" as a category of software that's not acceptable. I think this is misguided.

First, ActivityPub has a Web API that is used by federated servers to find out about people, the content they make, and what they've done. That's built in. It's on purpose. You have a lot of control of what is shared in this API. If you are concerned about what you share, you should check your software's privacy settings. There are a lot of knobs and levers.

I don't even think "scraper" is a reasonable name for this kind of software. We usually use that for screen-scraping software, which takes HTML output and grabs the structured data out of it.

Second, a lot of the things we use on the Fediverse depend on reading that API data. Like user search, content search, hashtag feeds, relays, backfilling conversations, and aggregate data.

Anyway, even for software that doesn't crawl the API, I think we need to give people the benefit of the doubt. People spend a lot of time and effort making that software -- usually open source, usually for free.

ActivityPub is made for a lot of different use cases. I really like when people explore use cases we haven't tried yet.

@evan
Hello brother, how are you? Could you please share my post and help me with a donation to provide treatment and food for my children? My husband is a prisoner and I have no provider. 🥹

Anyway, my answer is yes. It's OK for people to make code that does new things on the Fediverse.

@evan new things like what?

@evan No, that's not how it works. The fact that someone is able to *read* something I publish does not give them the right to republish it, aggregate it, use it to drive harassment mobs, incorporate it into slop-generation models in an attempt to mimic my writing style and deceive people, etc. etc. etc.

There is no explicit or implicit license to do anything with anyone's posts except display them temporarily, honoring any future block/deletion/etc. events, to users who the protcol indicates are allowed to see them.

@evan

@evan in the same way that people won't opt in to services that use their data to do things they don't care about, the developers making these services won't listen unless they are told forcefully that their presence is not welcome. I have the world's smallest violin for the people getting brigaded when they act like assholes. There are plenty of developers on Fedi doing new things and not getting brigaded. For example, I don't recall the GoToSocial devs getting brigaded for adding additional reply controls to their project, but they also weren't acting like jerks so I wouldn't expect them to be brigraded.

@dalias if a post is public, indexable and shareable, I think all this is explicitly allowed by the protocol.
Usage for AI training is not, but we all know it inevitably happens when any content is publicly available. @evan

@kleisli @evan It's not. That's why commercial social media has vile "Terms of Service" that demand you grant them license to basically do whatever they want with your writing/images/video, and sublicense them to other parties.

Respectable Mastodon instances do not do that. (The flagship tried and got such backlash they backed down.) This means there is no legal basis for doing *anything* with other people's posts. It's all potentially copyright infringement, relying on the author's intent and goodwill not to take action. But if you are doing stuff that clearly violates the author's intent and consent, you don't have any legal defense for that.

@bkardell If I knew what the new things were, they wouldn't be new.