Most of the worst of the worst scrapers run Headless Chrome. Headless
Chrome is difficult for Anubis to combat because it follows all the
rules that browsers do. The worst of the worst scrapers also use
residential proxy services. Those residental proxy services charge
upwards of $1 per GB of data egressed or ingressed. The Prompt API makes
Chrome download a 4Gi or 16Gi machine learning model. When you ask it to
start downloading, it will _continue_ downloading even when you leave
the Anubis challenge page.
This will make the local model answer "why is the sky blue?" in an
absurt amount of detail, which wastes both bandwidth and scraper CPU
(some scraping companies charge via Chrome CPU too).
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(config): deprecate the report_as field for challenges
This was a bad idea when it was added and it is irresponsible to
continue to have it. It causes more UX problems than it fixes with
slight of hand.
Closes: #1310Closes: #1307
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(policy): use the new logger for config validation messages
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs(admin/thresholds): remove this report_as setting
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Xe Iaso