The amount of FediFetcher instances scraping chaos.social is alarming. They all come from different Azure IPs because it's the recommended way to run it. Github reports: 1.361 deployments. We also see massive scraping from the TOR Network and scraping of RSS Feeds from SerendeputyBot. That sucks! #mastoadmin
Dieser Beitrag wurde bearbeitet. (11 Monate her)
teilten dies erneut
Patrick
Als Antwort auf Leah • • •Are you manually blocking these IPs once you discover them?
Leah
Als Antwort auf Patrick • • •Leah
Unbekannter Ursprungsbeitrag • • •Leah
Unbekannter Ursprungsbeitrag • • •Natasha Nox 🇺🇦🇵🇸
Als Antwort auf Leah • • •Leah
Als Antwort auf Natasha Nox 🇺🇦🇵🇸 • • •Tealk
Als Antwort auf Leah • • •But blocking that doesn't make much sense, does it? It is a function that is urgently needed and whether it is controlled via an external tool or built into Mastodon, the load will be the same.
I myself use an instance of FediFetcher on my infrastructure and would find it extremely bad if the tool stopped working.
Leah
Als Antwort auf Tealk • • •Michael 🇺🇦
Als Antwort auf Tealk • • •Tealk
Als Antwort auf Michael 🇺🇦 • • •Michael 🇺🇦
Als Antwort auf Tealk • • •Martijn Vos
Als Antwort auf Michael 🇺🇦 • •@Michael Vogel @Tealk @Leah (Cloudstylistin)
Is that what this is for? Then I understand completely. I find incomplete conversations a failure of ActivityPub. It needs a fix that doesn't cause too much overhead.
mögen das
hybrid havoc und Tealk mögen das.
hybrid havoc
Als Antwort auf Martijn Vos • • •I remember there being some browser extensions which would, upon viewing a thread in the browser, reach out and start pulling missing posts for that thread in real-time. It's unfortunate that it doesn't seem like most clients are built to do that. Having them pulled as needed would be preferable, I think, to pulling them just in case.
Martijn Vos
Als Antwort auf hybrid havoc • •@hybrid havoc @Michael Vogel @Tealk @Leah (Cloudstylistin)
Ueah, I'd prefer your own server to handle this with the server hosting the original post. Having every client do it is a bit much.
Michael 🇺🇦
Als Antwort auf Martijn Vos • • •hybrid havoc
Als Antwort auf Leah • • •Michael
Als Antwort auf hybrid havoc • • •@hybridhavoc
Unfortunately not. I would happily accept a pr for it, if you think it’d be useful though.
I’m amazed that FediFetcher is widely used. A shame that it’s still needed!
@leah is there anything that could be done to lessen the burden from your pov?
Leah
Als Antwort auf Michael • • •D3
Als Antwort auf Leah • • •as small instance admin I used FediFetcher to keep up with Fedi. Without some means to fetch context, my users are literally blindfolded most of the time they encounter a post originating from a "foreign" instance.
Not saying it's ideal or you'd be wrong to be pissed about the situation.
I've stopped FediFetcher for now, to see how this all plays out, but we small ones desperately need ways to keep up here 😀
D3
Als Antwort auf D3 • • •Leah
Als Antwort auf D3 • • •D3
Als Antwort auf Leah • • •FediFetcher for conversation context and GMF for hashtags people follow
We jump through some hoops to periodically provide GMF with an updated list of hashtags that are actually being followed, simply because we don't have a hard drive to hoard unread stuff 😬
Michael
Als Antwort auf Leah • • •Oh wow. That's a lot!
FWIW, given that FediFetcher by default mostly fetches replies to posts in people's home timelines, I do think that those fetched posts are probably looked at quite often. I certainly regularly do.
But I do really wish FediFetcher wasn't needed! It's such a pain that we miss so much context without it on small instances like mine.
@hybridhavoc
Leah
Als Antwort auf Michael • • •Michael
Als Antwort auf Leah • • •Yikes! that's not great.
And - speaking selfishly for a moment here - sad that you have decided to block FediFetcher, although I understand why.
Would it be helpful to put the origininator's instance into FediFetcher's UA, so that you can see if you can be more granular in your block and/or potentially even identify any bad actors, or would that result in a ‘CBA’ (or even worse ‘now we're just playing cat and mouse’) type reaction?
@hybridhavoc @Tealk
Leah
Als Antwort auf Michael • • •Michael
Als Antwort auf Leah • • •Leah
Als Antwort auf Michael • • •thank you 😀 I would build the UA like mastodon: (Mastodon/4.2.9; +https:///chaos.social/)
(Added one more slash to prevent recognition as URL)
Michael
Als Antwort auf Leah • • •Michael
Als Antwort auf Leah • • •OK, that's done now. UA's should have the format
FediFetcher (+mstdn.thms.uk; https[:]//go.thms.uk/ff)
It'll probably be a little while until all installs of FediFetcher have been updated though…
Let me know if that helps!
Leah
Als Antwort auf Michael • • •Paul Chambers🚧
Als Antwort auf Michael • • •@michael @hybridhavoc
I primarily have 2 active accounts on my self-hosted instance.
I operate the popular @hashtaggames account. That account generates a lot of interaction each day, esp between 9pm and 2am est
Fedifetcher is my number one crawler.
Here are my stats this week, starting on Sunday 00:00utc
55,439 (16.09%) hits
504 (06.77%) visitors
96.2 MiB (13.01%) TX
FediFetcher
FYI, I'm not complaining–just giving a use case example.
pieceofthepie
Als Antwort auf Leah • • •@michael @hybridhavoc Just out of interest what endpoints is the traffic generally hitting?
As I'm watching the logs on my instance here I can see that the majority of Fedifetcher traffic is to the search endpoint (which makes sense, it's my server, and I'm using it)
Any other instances of fedifetcher requests that I'm seeing (other servers fetching my post contexts) are a fraction of my other traffic.
Leah
Als Antwort auf pieceofthepie • • •pieceofthepie
Als Antwort auf Leah • • •ah, so not search then.
That is *quite* a lot of requests though :S
Leah
Als Antwort auf pieceofthepie • • •Michael 🇺🇦
Als Antwort auf Leah • • •tobi is writing bugs
Als Antwort auf Michael 🇺🇦 • • •you don't need a FEP for this, the tools are already in place (gts.superseriousbusiness.org/@…)
cc @leah
Post by tobi (they/them) is writing bugs , @dumpsterqueer@superseriousbusiness.org
superseriousbusiness.orgMichael 🇺🇦
Als Antwort auf tobi is writing bugs • • •infinite love ⴳ
Als Antwort auf tobi is writing bugs • • •fep/fep/7458/fep-7458.md at main
Codeberg.orginfinite love ⴳ
Als Antwort auf infinite love ⴳ • • •fep/fep/7888/fep-7888.md at main
Codeberg.orgStefan Bohacek
Unbekannter Ursprungsbeitrag • • •@oliphant One big downside, as you might realize, is that this only works for desktop browsers (maybe mobile Firefox?)
I use the official Android app quite a bit. Worth also pointing out that some mobile apps do have a way to pull down more replies, it would be nice to see the official apps implement this as well.
Leah
Unbekannter Ursprungsbeitrag • • •D3
Als Antwort auf Stefan Bohacek • • •Stefan Bohacek
Als Antwort auf D3 • • •D3
Als Antwort auf Stefan Bohacek • • •