Otherwise this can get unbounded, which can cause problems with lesser
HTTP proxies such as Apache.
Ref: AWOO-002
Signed-off-by: Xe Iaso <me@xeiaso.net>
pprof[1] is the Go standard library profiling toolkit. It is invaluable
for diagnosing how Go programs perform in the wild. However it also is
able to expose secret data set with command line flags. This is not
ideal and should be mitigated by correctly configured firewall rules. We
don't live in a world where people correctly configure firewall rules,
so we have to fix things for people. Welcome to 2026.
[1]: https://pkg.go.dev/runtime/pprof
Ref: AWOO-001
Signed-off-by: Xe Iaso <me@xeiaso.net>
Currently the honeypotting feature has no limits or delays anywhere and
uses that to feed an internal greylist of IP networks. This can cause
issues such as in #1613 where Claude's crawler seemed to pick up on it
and egress data at over one megabit per second until the administrator
noticed and blocked the address range.
This takes a different approach by inspiration of how the classic #xkcd
IRC bot Robot9000 works. The first time a given IPv4 /24 or IPv6 /48
visits a honepot page, Anubis sleeps for 1 millisecond. The second it
sleeps for two milliseconds. The third is four milliseconds and so on.
The goal of this is to make the scraping inherently self-limiting such
that the scrapers go off in their own corner where they won't really
hurt anyone.
Let's see if this works out according to keikaku.
Ref: https://github.com/TecharoHQ/anubis/issues/1613
Signed-off-by: Xe Iaso <me@xeiaso.net>
* fix(policy): correctly wire subrequest mode through CEL/path checkers
Previously Anubis only checked for the X-Original-Url when using
subrequest mode. This header is used by the example nginx config to pass
the request path through from the original client request to Anubis in
order to do path-based filtering.
According to facts and circumstances, Traefik hardcodes its own
headers[1]:
```text
httpdebug-1 | GET /.within.website/x/cmd/anubis/api/check
httpdebug-1 | X-Forwarded-Method: GET
httpdebug-1 | X-Forwarded-Proto: http
httpdebug-1 | X-Forwarded-Server: b9a5d299c929
httpdebug-1 | X-Forwarded-Port: 8080
httpdebug-1 | X-Forwarded-Uri: /
httpdebug-1 | X-Real-Ip: 172.18.0.1
httpdebug-1 | Accept-Encoding: gzip
httpdebug-1 | User-Agent: curl/8.20.0
httpdebug-1 | Accept: */*
httpdebug-1 | X-Forwarded-For: 172.18.0.1
httpdebug-1 | X-Forwarded-Host: localhost:8080
```
As a result, this means that path-based filtering did not work.
This commit fixes this issue by amending how path based checking logic
works:
* For CEL based checks, this pipes through the `subrequestMode` flag from
main and alters the behaviour if either `X-Original-Url` or
`X-Forwarded-Url` are found. These values are currently hardcoded for
convenience but probably need to be made configurable in the policy
file at a future date.
* For path-based checks, this uses the existing `subrequestMode` flag
from main and adds `X-Forwarded-Url` to the list of headers it checks.
A smoke test was added to make sure that traefik in this mode continues
to work. Thank you https://github.com/flifloo for filing a detailed
issue with the relevant configuration fragments. Those configuration
fragments formed the core of this smoke test.
[1]: https://doc.traefik.io/traefik/v3.4/middlewares/http/forwardauth/
Closes: https://github.com/TecharoHQ/anubis/issues/1628
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-Authored-By: flifloo <flifloo@gmail.com>
* chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: flifloo <flifloo@gmail.com>
* fix: patch GHSA-6wcg-mqvh-fcvg
PR https://github.com/TecharoHQ/anubis/pull/1015 added the ability for
reverse proxies using Anubis in subrequest auth mode to look at the path
of a request as there are many rules in the wild that rely on checking
the path. This is how access to things like robots.txt or anything in the
.well-known directory is unaffected by Anubis.
However this logic was also enabled for non-subrequest deployments of Anubis,
meaning that a specially crafted request could include a /.well-known/
path in it and then get around Anubis with little effort.
This fix gates the logic behind a new plumbed variable named subrequestMode
that only fires when Anubis is running in subrequest auth mode. This
properly contains that workaround so that the logic does not fire in
most deployments.
Signed-off-by: Xe Iaso <me@xeiaso.net>
* chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
Using the User-Agent as a filtering vector for the honeypot maze was a
decent idea, however in practice it can become a DoS vector by a
malicious client adding a lot of points to Google Chrome's User-Agent
string. In practice it also seems that the worst offenders use vanilla
Google Chrome User-Agent strings as well, meaning that this backfires
horribly.
Gotta crack a few eggs to make omlettes.
Closes: #1580
Signed-off-by: Xe Iaso <me@xeiaso.net>
* Resolve#1193
Address documentation and error message issues around REDIRECT_DOMAINS and required keywords in bot specifications.
* Add CHANGELOG entry
* fix: enable CEL iterators
Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>
* test: add unit tests for CELChecker map iteration
Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>
* fix: implement map iterators for HTTPHeaders and URLValues to resolve CEL internal errors
Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>
* fix: replace checker.NewMapIterator with newMapIterator for HTTPHeaders and URLValues
Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>
---------
Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>
PR #1451 added `CONTRIBUTING.md`, but the commit message guidelines
there conflict with the ones in `developer/code-quality.md`. Since
`CONTRIBUTING.md` is newer, presumably the guidelines there are what's
expected from new commits. But after removing that section from
`code-quality.md`, there's not much content left, so this commit just
deletes the file entirely.
Signed-off-by: Max Chernoff <git@maxchernoff.ca>
* refactor(http): split long line in respondWithStatus
Signed-off-by: Max Chernoff <git@maxchernoff.ca>
* feat(http): set `Cache-Control: no-store` on error responses
Since #132, Anubis has set `Cache-Control: no-store` on challenge
responses. However, this does not apply to deny responses, meaning that
if Anubis is configured to block certain user agents and is behind a
caching reverse proxy, this error page will be cached and served to all
subsequent requests, even those with an allowed user agent. This commit
configures the error page responder to also set the `Cache-Control`
header, meaning that deny and challenge responses will now both have the
same behaviour.
Signed-off-by: Max Chernoff <git@maxchernoff.ca>
* chore(spelling): add new words to allowlist
Signed-off-by: Max Chernoff <git@maxchernoff.ca>
* chore(actions): bump Go version to fix govulncheck errors
Signed-off-by: Max Chernoff <git@maxchernoff.ca>
---------
Signed-off-by: Max Chernoff <git@maxchernoff.ca>
Signed-off-by: Xe Iaso <xe.iaso@techaro.lol>
Co-authored-by: Xe Iaso <xe.iaso@techaro.lol>
The default Anubis systemd configuration is very restrictive in
order to prevent any possible compromise of Anubis to be useful
by threat actors. As such, it assumes all logs will be pushed to
the system journal. Some administrators do not want Anubis' logs
to be pushed to the system journal and want Anubis to log to a
file instead.
This change documents how to set up ReadWritePaths in the Anubis
systemd configuration such that Anubis can lot to a file as
administrators expect.
Closes: #1468
Signed-off-by: Xe Iaso <me@xeiaso.net>
This makes it clear that when generating a kubernetes secret to pull the bot stopper image that:
- no email is required
- a user is required but the actual value of the username is not checked
- the GH token needs to be pasted in
Signed-off-by: Thomas Arrow <tarrow@users.noreply.github.com>
* fix(web): include base prefix in generated URLs
Forgot to add the base prefix to these URLs. Committed a fix for this
and added a test to ensure this does not repeat. Oops!
Closes: #1402
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: update CHANGELOG
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
* docs: split nginx configuration files to their own directory
Signed-off-by: Xe Iaso <me@xeiaso.net>
* test: add nginx config smoke test based on the config in the docs
Signed-off-by: Xe Iaso <me@xeiaso.net>
---------
Signed-off-by: Xe Iaso <me@xeiaso.net>
TIL docker clients don't include the Accept header all the time. I would
have thought they did that. Oops.
Closes: #1346
Signed-off-by: Xe Iaso <me@xeiaso.net>
Xe Iaso