The amount of FediFetcher instances scraping chaos.social is alarming. They all come from different Azure IPs because it's the recommended way to run it. Github reports: 1.361 deployments. We also see massive scraping from the TOR Network and scraping of RSS Feeds from SerendeputyBot. That sucks! #mastoadmin
Dieser Beitrag wurde bearbeitet. (11 Monate her)

teilten dies erneut

Als Antwort auf Martijn Vos

@mcv @heluecht @Tealk
I remember there being some browser extensions which would, upon viewing a thread in the browser, reach out and start pulling missing posts for that thread in real-time. It's unfortunate that it doesn't seem like most clients are built to do that. Having them pulled as needed would be preferable, I think, to pulling them just in case.
Als Antwort auf Martijn Vos

Yes, when ActivityPub was designed I was pushing for the distribution method that Diaspora uses (the first post of a thread is responsible for distributing comments). This is possible in ActivityPub using LD signatures, but unfortunately it hasn't taken off. Unfortunately, the commenting systems now distribute their comments on their own, leading to the known problems.
Als Antwort auf Leah

as small instance admin I used FediFetcher to keep up with Fedi. Without some means to fetch context, my users are literally blindfolded most of the time they encounter a post originating from a "foreign" instance.

Not saying it's ideal or you'd be wrong to be pissed about the situation.

I've stopped FediFetcher for now, to see how this all plays out, but we small ones desperately need ways to keep up here 😀

Dieser Beitrag wurde bearbeitet. (11 Monate her)
Als Antwort auf Leah

Yikes! that's not great.

And - speaking selfishly for a moment here - sad that you have decided to block FediFetcher, although I understand why.

Would it be helpful to put the origininator's instance into FediFetcher's UA, so that you can see if you can be more granular in your block and/or potentially even identify any bad actors, or would that result in a ‘CBA’ (or even worse ‘now we're just playing cat and mouse’) type reaction?

@hybridhavoc @Tealk

Als Antwort auf Michael

@michael @hybridhavoc

I primarily have 2 active accounts on my self-hosted instance.

I operate the popular @hashtaggames account. That account generates a lot of interaction each day, esp between 9pm and 2am est

Fedifetcher is my number one crawler.

Here are my stats this week, starting on Sunday 00:00utc

55,439 (16.09%) hits
504 (06.77%) visitors
96.2 MiB (13.01%) TX
FediFetcher

FYI, I'm not complaining–just giving a use case example.

Als Antwort auf Leah

@michael @hybridhavoc Just out of interest what endpoints is the traffic generally hitting?

As I'm watching the logs on my instance here I can see that the majority of Fedifetcher traffic is to the search endpoint (which makes sense, it's my server, and I'm using it)

Any other instances of fedifetcher requests that I'm seeing (other servers fetching my post contexts) are a fraction of my other traffic.

Als Antwort auf Michael 🇺🇦

you don't need a FEP for this, the tools are already in place (gts.superseriousbusiness.org/@…)

cc @leah

@Leah
Als Antwort auf tobi is writing bugs

There's a big difference in terms of load on the systems between adding yourself to the list of recipients and getting comments as they arrive, and having to make these requests yourself and requesting each one. It also doesn't help with new comments that you don't know about, when you want to get notified about new comments in the background.
Dieser Beitrag wurde bearbeitet. (11 Monate her)
Als Antwort auf tobi is writing bugs

@dumpsterqueer @heluecht funny thing is that replies collection actually isn't part of activitypub. it's part of activitystreams instead. there are no side effects for receiving a Create of an object with inReplyTo set. w3id.org/fep/7458 is a FEP that is relevant here, describing actually using the replies collection. combine with w3id.org/fep/0391 for special collection proofs (stamps).
Als Antwort auf infinite love ⴳ

@dumpsterqueer @heluecht and hey, while we're looking at properties from activitystreams, i'd perennially point toward `context` and w3id.org/fep/7888 as a "parallel path" to fep-7458. instead of recursing through replies collections, we could have a singular context collection which represents the moderated conversation as a flat list. this allows inReplyTo as simply metadata.
Unbekannter Ursprungsbeitrag

mastodon - Link zum Originalbeitrag

Stefan Bohacek

@oliphant One big downside, as you might realize, is that this only works for desktop browsers (maybe mobile Firefox?)

I use the official Android app quite a bit. Worth also pointing out that some mobile apps do have a way to pull down more replies, it would be nice to see the official apps implement this as well.

Als Antwort auf D3

@dnddeutsch @oliphant I'm not too sure. I mean, it will mainly increase the load on each server, that's for sure. But if it's only triggered by a user pressing a "fetch more replies" button, that's still better than what FediFetcher does, or even visiting the original page in your browser, which will also require loading CSS, JS, and other resources, as opposed to pulling in the post object via API.
@D3