By the way, I am now officially tied to a device which is too old to support Anubis.
-
@xyhhx mhm okay what browser do you use
I have the same configuration and I actually sort of understand it now
the most common blunder is faked User-Agent and Sec-Ch-Ua headers -
@xyhhx mhm okay what browser do you use
I have the same configuration and I actually sort of understand it now
the most common blunder is faked User-Agent and Sec-Ch-Ua headers -
By the way, I am now officially tied to a device which is too old to support Anubis. Not just in the "it takes too long" sense, but also being unable to install any version of Chromium that supports the required APIs.
This is probably what a lot of people which are less fortunate than you feel like. -
I think the biggest statement that I want to make here is this:
Software should be efficient, and fast. We've all forgotten what it means to be on a platform that is restricted ever since our computing resources started going up, and this is where it left us.
Those less fortunate are unable to view even the simplest pages because there is software in front that simply won't run on my device.
Our software isn't quick or snappy anymore, to the point where any software which IS quick or snappy markets itself as being so. It has become a marketing feature.
Sure, all our new fancy tech is quite nice, but let's not forget that not everyone is as fortunate.
I am very glad that there's tools which work even on the cheapest or oldest devices.@alexia can i ask a quick favour of you? (no worries if not, just ignore me or tell me to buzz off lol). if you visit my profile https://k.iim.gay/@kim in your browser, how long does the scraper deterrence there take to complete on your device? it's something we're testing embedding a very simplified form of in gotosocial as an optional defense for users (defaults to off). but depending on accessibility issues it's not something we're set in stone on keeping. we're ultimately just experimenting with making it easy for users to protect themselves, but only if it doesn't compromise on our core principles of accessibility, ease of deployment and low resource usage (at least on the server side, though you'll see from our extremely minimal web client we do care about that client side too)
-
@alexia can i ask a quick favour of you? (no worries if not, just ignore me or tell me to buzz off lol). if you visit my profile https://k.iim.gay/@kim in your browser, how long does the scraper deterrence there take to complete on your device? it's something we're testing embedding a very simplified form of in gotosocial as an optional defense for users (defaults to off). but depending on accessibility issues it's not something we're set in stone on keeping. we're ultimately just experimenting with making it easy for users to protect themselves, but only if it doesn't compromise on our core principles of accessibility, ease of deployment and low resource usage (at least on the server side, though you'll see from our extremely minimal web client we do care about that client side too)
@kim 13450ms on my ZTE Blade A34 (so, quad-core 1.6GHz)
on the Moto C Plus, it doesn't work at all. Just never loads, as I can't get a browser with the required APIs on this thing
(well except Firefox but it gets OOM'd too frequently) -
@xyhhx hm. Can you drop your User-Agent string and Sec-Ch-Ua headers?
You can use this to check Sec-Ch-Ua, found it on a whim: 51degrees.com/client-hints -
@kim 13450ms on my ZTE Blade A34 (so, quad-core 1.6GHz)
on the Moto C Plus, it doesn't work at all. Just never loads, as I can't get a browser with the required APIs on this thing
(well except Firefox but it gets OOM'd too frequently)@alexia I'm interested that it never loads. we don't actually use any browser crypto APIs, we use our own lil sha256 function in pure js. maybe it doesn't support service workers?
even the no-JS proof of work methods I've seen are super hacky. finding solutions to protect against LLM scrapers is such a pain in the ass.
thank you for doing this btw, very useful, and clearly i need to go do some more thinking on it
-
@alexia I'm interested that it never loads. we don't actually use any browser crypto APIs, we use our own lil sha256 function in pure js. maybe it doesn't support service workers?
even the no-JS proof of work methods I've seen are super hacky. finding solutions to protect against LLM scrapers is such a pain in the ass.
thank you for doing this btw, very useful, and clearly i need to go do some more thinking on it
@kim well supposedly Chromium 66 (which is roughly what I'm stuck on) should have service workers, I've ran it for a minute and it just sorta went nowhere so I assumed it'd never finish -
@sneexy Until the Pixel 7 arrives somewhere between 18th and 23rd of this month, I am stuck with:
- Moto C Plus (32bit, armv6, Android 7, modded)
- ZTE Blade A34 (64bit, armv7?, Android 13, stock, cannot unlock) -
@alexia@shrimp.starlightnet.work @carbonatedcaffeine@social.treehouse.systems I'm going to be trying my best to keep my current PC for that long, previous PC held strong for almost 7 years before I managed to earn some funds for a new one in 2021. Old hardware is still sitting here ofc, ready to serve as a retro machine or backup PC if needed (although I also have a steamdeck now)
@mitsunee @alexia @carbonatedcaffeine It's much easier to do that with PCs than with phones, since they're more repairable, don't require a battery, will run upstream kernels and so can be kept up to date without support from the vendor, and are generally more powerful than phones. I still regularly use a laptop from 2008 and it's perfectly usable, but a phone from that period would be almost unusable (especially without a new battery) and might not even be able to connect to the mobile network in some countries. -
@mitsunee @alexia @carbonatedcaffeine It's much easier to do that with PCs than with phones, since they're more repairable, don't require a battery, will run upstream kernels and so can be kept up to date without support from the vendor, and are generally more powerful than phones. I still regularly use a laptop from 2008 and it's perfectly usable, but a phone from that period would be almost unusable (especially without a new battery) and might not even be able to connect to the mobile network in some countries.
@noisytoot@berkeley.edu.pl @alexia@shrimp.starlightnet.work @carbonatedcaffeine@social.treehouse.systems yeah I honestly have no clue what I'll do if my current phone dies or stops getting updates entirely. There isn't a single manufacturer I trust with keeping up-to-date with security patches for longer than the first one to three years of a device's lifespan and I honestly don't even have the money to replace it currently
-
@mitsunee @alexia @carbonatedcaffeine It's much easier to do that with PCs than with phones, since they're more repairable, don't require a battery, will run upstream kernels and so can be kept up to date without support from the vendor, and are generally more powerful than phones. I still regularly use a laptop from 2008 and it's perfectly usable, but a phone from that period would be almost unusable (especially without a new battery) and might not even be able to connect to the mobile network in some countries.@noisytoot @mitsunee @carbonatedcaffeine
fun fact: one of the two devices (the ZTE) that I'm currently on until the used order arrives has a Unisoc SoC that is apparently supported by mainline linux :3 -
-
@noisytoot@berkeley.edu.pl @alexia@shrimp.starlightnet.work @carbonatedcaffeine@social.treehouse.systems yeah I honestly have no clue what I'll do if my current phone dies or stops getting updates entirely. There isn't a single manufacturer I trust with keeping up-to-date with security patches for longer than the first one to three years of a device's lifespan and I honestly don't even have the money to replace it currently
@mitsunee@mk.absturztau.be @alexia@starlightnet.work @carbonatedcaffeine@social.treehouse.systems Maybe a postmarketOS device that can run mainline Linux? Something with SDM845 is probably the best choice currently (or a PinePhone, but the hardware sucks).
-
@alexia can i ask a quick favour of you? (no worries if not, just ignore me or tell me to buzz off lol). if you visit my profile https://k.iim.gay/@kim in your browser, how long does the scraper deterrence there take to complete on your device? it's something we're testing embedding a very simplified form of in gotosocial as an optional defense for users (defaults to off). but depending on accessibility issues it's not something we're set in stone on keeping. we're ultimately just experimenting with making it easy for users to protect themselves, but only if it doesn't compromise on our core principles of accessibility, ease of deployment and low resource usage (at least on the server side, though you'll see from our extremely minimal web client we do care about that client side too)
-
In other words, I encourage you to look to other solutions.
Some that I know of in no particular order:
- go-away – alternative with No-JS challenges and more customizability
- Iocaine – actively poison AI scrapers, best paired with nam shub of enki configuration
I heard mumbles that Anubis is also getting No-Javascript methods eventually, but I don't know more than that.
Anubis has No-Javascript challenges, there's some more advanced ones in the paid offering
(feel free to add to that list, I had something bookmarked but I've lost it on my old phone)@alexia what do you mean by if you pay up? As far as I'm aware no-js support should be part of a recent anubis open source release already? (https://github.com/TecharoHQ/anubis/releases/tag/v1.20.0)
It's not enabled by default but ideally it'd be used for UAs reporting some old browser. -
@alexia what do you mean by if you pay up? As far as I'm aware no-js support should be part of a recent anubis open source release already? (https://github.com/TecharoHQ/anubis/releases/tag/v1.20.0)
It's not enabled by default but ideally it'd be used for UAs reporting some old browser.@jakob I already edited the post, I was corrected by Anubis' author. Be sure to view on the source -
@jakob I already edited the post, I was corrected by Anubis' author. Be sure to view on the source
-
A awoo@gts.apicrim.es shared this topic
