After deploying Anubis bot traffic is drastically reduced but I still
see a lot of requests from User-Agents that claim to be 'Opera' like so:
"Opera/9.90.(Windows NT 6.0; mt-MT) Presto/2.9.173 Version/10.00"
"Opera/8.46.(X11; Linux i686; fo-FO) Presto/2.9.161 Version/11.00"
Add 'Opera' to the generic-browser rule to also challenge them.
Signed-off-by: Michael Jeanson <[email protected]>
Signed-off-by: Xe Iaso <[email protected]>
Co-authored-by: Xe Iaso <[email protected]>
* Add documentation example for a NGINX configuration that demonstrates how to insert Anubis in the middle of a normal configuration.
Signed-off-by: Jeroen Massar <[email protected]>
* Add changelog entry
Signed-off-by: Jeroen Massar <[email protected]>
* docs/admin/installation: rephrasing and diagrams
Signed-off-by: Xe Iaso <[email protected]>
* docs/admin/installation: flatten down the nginx config
Signed-off-by: Xe Iaso <[email protected]>
* docs/admin/installation: other fixups and note the assumptions at play
Thanks @SuperSandro2000!
Signed-off-by: Xe Iaso <[email protected]>
---------
Signed-off-by: Jeroen Massar <[email protected]>
Signed-off-by: Xe Iaso <[email protected]>
Co-authored-by: Xe Iaso <[email protected]>
* Create anubis.freebsd
add freebsd rc.d script so can be run as a freebsd daemon
Signed-off-by: Paul Wilde <[email protected]>
* Update CHANGELOG.md
Signed-off-by: Paul Wilde <[email protected]>
---------
Signed-off-by: Paul Wilde <[email protected]>
* fix(fetch): improve error handling for Content-Type parsing
Signed-off-by: Jason Cameron <[email protected]>
* fix(fetch): rename OgHandledError to ErrOgHandled for statichcheck to like me
Signed-off-by: Jason Cameron <[email protected]>
---------
Signed-off-by: Jason Cameron <[email protected]>
- updates botPolicies with ips from the website
- adds the updated information to the `CHANGELOG.md` file
Signed-off-by: Xe Iaso <[email protected]>
Co-authored-by: Xe Iaso <[email protected]>
I've been keeping a list in my head for a while, but I think a canonical
location with most known instances could help others, e.g. for deciding
wheather to use Anubis or not and to get in contact with Anubis operators.
* feat: Add Open Graph tag support (og-tags)
Signed-off-by: Jason Cameron <[email protected]>
* Fix: Prevent nil pointer dereference in test (og-tags)
Signed-off-by: Jason Cameron <[email protected]>
* feat!: Implement Open Graph tag caching and passthrough functionality (WIP)
I'm going to sleep. currently tags are passed to renderIndex.
see https://github.com/TecharoHQ/anubis/issues/131
Signed-off-by: Jason Cameron <[email protected]>
* feat: Add configuration for air tool with build and logger settings
Signed-off-by: Jason Cameron <[email protected]>
* feat: Move OG tags to base template (og-tags)
Moves the Open Graph (OG) tags from the index template to
the base template. This allows OG tags to be set on any
page, not just the index. Also adds a
BaseWithOGTags function to the web package to allow
passing OG tags to the base template. Removes the
ogTags parameter from the Index function and template.
Signed-off-by: Jason Cameron <[email protected]>
* Delete CHANGELOG.md
Signed-off-by: Jason Cameron <[email protected]>
* feat: Add language attribute to HTML tag in template
Signed-off-by: Jason Cameron <[email protected]>
* fix(tests): Fix nil pointer ref
Signed-off-by: Jason Cameron <[email protected]>
* feat(og-tags): Add timeout to http client (og-tags)
Signed-off-by: Jason Cameron <[email protected]>
* style: fix line endings & indentation
Signed-off-by: Jason Cameron <[email protected]>
* style: add inspection comment for GoBoolExpressions in UnchangingCache
Signed-off-by: Jason Cameron <[email protected]>
* feat(og-tags): Implement Open Graph tag fetching and caching
Signed-off-by: Jason Cameron <[email protected]>
* fix(og-tags): Simplify Open Graph tag extraction logic
Signed-off-by: Jason Cameron <[email protected]>
* fix(og-tags): Add nil check in isOGMetaTag and enhance test cases
Signed-off-by: Jason Cameron <[email protected]>
* feat(og-tags): Add approved tags and prefixes for Open Graph extraction
Signed-off-by: Jason Cameron <[email protected]>
* test(og-tags): Update tests with approved tags and improve clarity
Signed-off-by: Jason Cameron <[email protected]>
* chore: Add changelog notes
Signed-off-by: Jason Cameron <[email protected]>
* fix: Improve stability of the target fetcher?
Signed-off-by: Jason Cameron <[email protected]>
* fix: Update template error handling and improve Open Graph tag integration
Signed-off-by: Jason Cameron <[email protected]>
* style: format files and remove deubg logs
Signed-off-by: Jason Cameron <[email protected]>
* feat: Credit CELPHASE for mascot design (og-tags)
Signed-off-by: Jason Cameron <[email protected]>
* feat: Credit CELPHASE for mascot design (og-tags)
Signed-off-by: Jason Cameron <[email protected]>
* feat: Allow twitter prefixed OG tags by default
Signed-off-by: Jason Cameron <[email protected]>
* chore: replace /tmp with /var
Signed-off-by: Jason Cameron <[email protected]>
* Update docs/docs/CHANGELOG.md
Co-authored-by: Xe Iaso <[email protected]>
Signed-off-by: Jason Cameron <[email protected]>
* Update docs/docs/admin/configuration/open-graph.mdx
Co-authored-by: Xe Iaso <[email protected]>
Signed-off-by: Jason Cameron <[email protected]>
* chore: add fediverse to default prefixes (#og-tags)
Signed-off-by: Jason Cameron <[email protected]>
* feat(og-tags): Remove og-query-distinct flag
This commit removes the `og-query-distinct` flag and
associated logic. URLs with different query parameters
will now always be treated as the same cache key for Open
Graph tags. This simplifies the caching logic and
improves performance.
Additionally, the http client used for fetching OG tags
is now a member of the OGTagCache struct, rather than a
global variable. This improves testability and allows
for more flexible configuration in the future.
Signed-off-by: Jason Cameron <[email protected]>
* Update docs/docs/admin/configuration/open-graph.mdx
Co-authored-by: Xe Iaso <[email protected]>
Signed-off-by: Jason Cameron <[email protected]>
* docs: remove og tags references
Signed-off-by: Jason Cameron <[email protected]>
* refactor: rename url > u to not overlap package name
Signed-off-by: Jason Cameron <[email protected]>
* Update internal/ogtags/cache.go
Co-authored-by: Xe Iaso <[email protected]>
Signed-off-by: Jason Cameron <[email protected]>
* Update internal/ogtags/cache.go
Co-authored-by: Xe Iaso <[email protected]>
Signed-off-by: Jason Cameron <[email protected]>
* fix(tests): Don't use network when network access is disabled
Signed-off-by: Jason Cameron <[email protected]>
* Fix: Handle nil URL in GetOGTags (og-tags)
Signed-off-by: Jason Cameron <[email protected]>
* chore: sort installation docs alphabetically
Signed-off-by: Jason Cameron <[email protected]>
* fix(tests): validate that no duplicate requests are made
Signed-off-by: Jason Cameron <[email protected]>
* style(tests): remove unused ok var
Signed-off-by: Jason Cameron <[email protected]>
* docs: convert to table fmt
Signed-off-by: Jason Cameron <[email protected]>
* feat(og-tags): Enhance OG tag fetching and caching
Adds additional approved OG tags (`keywords`, `author`), improves
Signed-off-by: Jason Cameron <[email protected]>
* chore: update generated templ's after format
Signed-off-by: Jason Cameron <[email protected]>
* fix(tests): update integration_test.go to reflect the new behavior of fetchHTMLDocument
Signed-off-by: Jason Cameron <[email protected]>
* Revert "data/botPolicies: allow iMessage scraper by default (#178)"
This reverts commit 21a9d777
Signed-off-by: Jason Cameron <[email protected]>
* Fix: Simplify ogTags access in cache test.
Didn't know this was possible! wow!
Signed-off-by: Jason Cameron <[email protected]>
* Fix: Handle request timeouts when fetching OG tags (#og-tags)
Cache a nil result for half the TTL to avoid repeatedly
requesting a timed-out URL.
Signed-off-by: Jason Cameron <[email protected]>
* Fix: make OG tags passthrough option function.
Signed-off-by: Jason Cameron <[email protected]>
* Fix: Handle timeouts and non-200 responses when fetching OG tags (og-tags)
- Cache empty results for timeouts and non-200 status codes
to avoid spamming the server.
- Use a non-nil empty map to represent empty results in the
cache, as nil would be a cache miss.
Signed-off-by: Jason Cameron <[email protected]>
* feat(og-tags): switch to http.MaxBytesReader
Signed-off-by: Jason Cameron <[email protected]>
* chore(og-tags): add noindex, nofollow meta tag and update error line numbers
Signed-off-by: Jason Cameron <[email protected]>
---------
Signed-off-by: Jason Cameron <[email protected]>
Signed-off-by: Jason Cameron <[email protected]>
Co-authored-by: Xe Iaso <[email protected]>
* make a half-baked tarball
Closes#217
Signed-off-by: Xe Iaso <[email protected]>
* make two tarballs: one with just the vendor, and one with vendor and npm
Signed-off-by: Xe Iaso <[email protected]>
---------
Signed-off-by: Xe Iaso <[email protected]>
* add a Makefile
Based on advice from IRC, a makefile helps downstream packagers
understand how to build the software.
Signed-off-by: Xe Iaso <[email protected]>
* Apply review suggestions
Signed-off-by: Xe Iaso <[email protected]>
---------
Signed-off-by: Xe Iaso <[email protected]>
* cmd/anubis actually check the result with the correct difficulty
* chore: changelog
* test(cmd/anubis): make test check for difficulty
* lib: add regression test for CVE-2025-24369
Signed-off-by: Xe Iaso <[email protected]>
* bump VERSION and CHANGELOG
Tracks #181
Signed-off-by: Xe Iaso <[email protected]>
---------
Signed-off-by: Xe Iaso <[email protected]>
Co-authored-by: Xe Iaso <[email protected]>
* web/js: update page to allow users to read the "Why am I seeing this?", complete with a button to send them through after challenge completed, and a 30s timeout that does the same.
* .gitignore: added .DS_store.
* docs/docs/CHANGELOG: added to the Unreleased section as requested in code quality guidelines
* web: pushing index_templ.go alongside this update.
* package.json: added postcss to dependencies list.
* package-lock: added postcss to dependencies
* Revert "package-lock: added postcss to dependencies"
This reverts commit bf02e7ba56.
* Revert "package.json: added postcss to dependencies list."
This reverts commit 1a38c63049.
* web/js: OG comments are important
---------
Signed-off-by: Xe Iaso <[email protected]>
Co-authored-by: Xe Iaso <[email protected]>
- Fixed a typo in the challenge page title, removing
an unnecessary backslash.
- Updated the index page title to "Making sure
you're not a bot!".
Signed-off-by: Jason Cameron <[email protected]>
* fix: Correctly format listener address (https://github.com/TecharoHQ/anubis/issues/93)
Handle addresses that include a hostname, not just ports. If
the address starts with a colon, assume it's just a port and
prefix it with "http://localhost". Otherwise, prefix the
entire address with "http://". This ensures that the listener
URL is correctly formatted regardless of whether it includes
a hostname or just a port.
Signed-off-by: Jason Cameron <[email protected]>
* chore(docs): add changelog entry
Signed-off-by: Jason Cameron <[email protected]>
---------
Signed-off-by: Jason Cameron <[email protected]>
* web/js: add project license in the JavaScript used by Anubis
This will allow LibreJS users to pass the captcha without problems
without having to whitelist anubis manually.
* Update docs/docs/CHANGELOG.md
Co-authored-by: Xe Iaso <[email protected]>
Signed-off-by: Fijxu <[email protected]>
---------
Signed-off-by: Fijxu <[email protected]>
Co-authored-by: Xe Iaso <[email protected]>
* cmd/anubis: add a debug option for benchmarking hashrate
Having the ability to benchmark different proof-of-work implementations
is useful for extending Anubis. This adds a flag `--debug-benchmark-js`
(and its associated environment variable `DEBUG_BENCHMARK_JS`) for
serving a tool to do so.
Internally, a there is a new policy action, "DEBUG_BENCHMARK", which
serves the benchmarking tool instead of a challenge. The flag then
replaces all bot rules with a special rule matching every request
to that action. The benchmark page makes heavy use of inline styles,
because currently all global styles are shared across all pages. This
could be fixed, but I wanted to avoid major changes to the templates.
* web/js: add signal for aborting an active proof-of-work algorithm
Both proof-of-work algorithms now take an optional `AbortSignal`, which
immediately terminates all workers and returns `false` if aborted before
the challenge is complete.
* web/js: add algorithm comparison to the benchmark page
"Compare:" is added to the benchmark page for testing the relative
performance between two algorithms. Since benchmark runs generally have
high variance, it may take a while for the averages to converge on a
stable difference.
---------
Signed-off-by: Xe Iaso <[email protected]>
Co-authored-by: Xe Iaso <[email protected]>
* Add periodic cleanup job for DecayMap
see https://github.com/TecharoHQ/anubis/issues/8
* Refactor: Improve DecayMap cleanup tests and add Len method
- Refactored DecayMap cleanup tests to use the new Len method
for more precise assertions.
- Added a Len method to DecayMap to retrieve the number of
entries.
- Simplified conditional checks in Get method.
* chore(changelog): add entry
* fix(tests): Use Impl.expire for decaymap cleanup
Signed-off-by: Jason Cameron <[email protected]>
---------
Signed-off-by: Jason Cameron <[email protected]>
Using TrimRight will remove all characters from `*dockerRepo` from right
to left that match a character contained on `"/"+filepath.Base(*dockerRepo)`
(the cutset) until it doesn't matches anymore.
So for example, if `dockerRepo` is `example.com/fijxu/anubis`, and
`"/"+filepath.Base(*dockerRepo)` is `/anubis`, it will remove
`u/anubis` and not just `/anubis` from `dockerRepo` because `u` is a character inside the
cutoff.
Xe Iaso