Compare commits

...

43 Commits

Author SHA1 Message Date
Xe Iaso 652cef7ffe fix(docs/deploy): harden public docs deployment and pin images
Add a pod-level security context to the nginx container in the public
docs deployment (non-root uid 101, dropped capabilities, read-only
root filesystem, RuntimeDefault seccomp) and rebind it to unprivileged
port 8080 so it does not need CAP_NET_BIND_SERVICE. The nginx PID and
proxy temp paths move under a tmpfs-backed emptyDir so the read-only
root filesystem does not break startup.

Replace the mutable :main tags on both containers with immutable
sha256 digests and switch imagePullPolicy to IfNotPresent so each
rollout references an auditable artifact instead of whatever happens
to be tagged :main when the pod starts. The docs-deploy workflow now
overlays the freshly built docs digest via `kustomize edit set image`
so the manifest stays accurate without a manual edit on each push to
main. The docs Dockerfile pins its node and nginx-micro base images
to specific versions for the same reason.

Ref: AWOO-011, AWOO-012
Assisted-by: Claude Opus 4.7 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-18 22:41:12 -04:00
Xe Iaso 97d15cd803 fix(expressions): validate randInt bounds before rand.IntN
Non-positive or platform-overflowing arguments to the CEL randInt
helper used to reach rand.IntN unchecked, surfacing a CEL evaluator
error during request processing when policies passed
attacker-influenced values (e.g. contentLength). Reject non-positive
bounds and detect int narrowing explicitly, returning a typed CEL
error in both cases.

Ref: AWOO-010
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-18 21:27:28 -04:00
Xe Iaso 120a730a66 fix(lib): mend case where domainless redirects could allow cross-domain
Ref: AWOO-009
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-18 21:22:45 -04:00
Xe Iaso 386e92eb97 fix(expressions): mend possible nil pointer deref edge case
If Anubis just started up, load averages may not be set in memory. This
can cause a nil pointer dereference which could fail requests with weird
errors until the async thread sets the load averages.

Ref: AWOO-005
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-18 21:12:38 -04:00
Xe Iaso e3f500cb56 fix(policy): mend an edge case with subrequest auth and query strings
This fixes an unlikely edge case where using subrequest auth and query
strings with path based filtering can cause reality to differ from
administrator intent. This effectively strips the query string from
subrequest auth checks. This deficiency should be fixed in the future.

Ref: AWOO-004
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-18 21:08:38 -04:00
Xe Iaso 75aa251406 fix(honeypot/naive): cap r9k delay to one second
Otherwise this can get unbounded, which can cause problems with lesser
HTTP proxies such as Apache.

Ref: AWOO-002
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-18 21:00:36 -04:00
Xe Iaso 324c2f4fed fix(metrics): don't expose pprof by default
pprof[1] is the Go standard library profiling toolkit. It is invaluable
for diagnosing how Go programs perform in the wild. However it also is
able to expose secret data set with command line flags. This is not
ideal and should be mitigated by correctly configured firewall rules. We
don't live in a world where people correctly configure firewall rules,
so we have to fix things for people. Welcome to 2026.

[1]: https://pkg.go.dev/runtime/pprof

Ref: AWOO-001
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-18 20:54:28 -04:00
Xe Iaso b57508afcd fix(honeypot/naive): apply robot9001 style delays (#1632)
Currently the honeypotting feature has no limits or delays anywhere and
uses that to feed an internal greylist of IP networks. This can cause
issues such as in #1613 where Claude's crawler seemed to pick up on it
and egress data at over one megabit per second until the administrator
noticed and blocked the address range.

This takes a different approach by inspiration of how the classic #xkcd
IRC bot Robot9000 works. The first time a given IPv4 /24 or IPv6 /48
visits a honepot page, Anubis sleeps for 1 millisecond. The second it
sleeps for two milliseconds. The third is four milliseconds and so on.
The goal of this is to make the scraping inherently self-limiting such
that the scrapers go off in their own corner where they won't really
hurt anyone.

Let's see if this works out according to keikaku.

Ref: https://github.com/TecharoHQ/anubis/issues/1613

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-15 17:56:37 -04:00
Xe Iaso 276b537776 fix(policy): correctly wire subrequest mode through CEL/path checkers (#1630)
* fix(policy): correctly wire subrequest mode through CEL/path checkers

Previously Anubis only checked for the X-Original-Url when using
subrequest mode. This header is used by the example nginx config to pass
the request path through from the original client request to Anubis in
order to do path-based filtering.

According to facts and circumstances, Traefik hardcodes its own
headers[1]:

```text
httpdebug-1  | GET /.within.website/x/cmd/anubis/api/check
httpdebug-1  | X-Forwarded-Method: GET
httpdebug-1  | X-Forwarded-Proto: http
httpdebug-1  | X-Forwarded-Server: b9a5d299c929
httpdebug-1  | X-Forwarded-Port: 8080
httpdebug-1  | X-Forwarded-Uri: /
httpdebug-1  | X-Real-Ip: 172.18.0.1
httpdebug-1  | Accept-Encoding: gzip
httpdebug-1  | User-Agent: curl/8.20.0
httpdebug-1  | Accept: */*
httpdebug-1  | X-Forwarded-For: 172.18.0.1
httpdebug-1  | X-Forwarded-Host: localhost:8080
```

As a result, this means that path-based filtering did not work.

This commit fixes this issue by amending how path based checking logic
works:

* For CEL based checks, this pipes through the `subrequestMode` flag from
  main and alters the behaviour if either `X-Original-Url` or
  `X-Forwarded-Url` are found. These values are currently hardcoded for
  convenience but probably need to be made configurable in the policy
  file at a future date.
* For path-based checks, this uses the existing `subrequestMode` flag
  from main and adds `X-Forwarded-Url` to the list of headers it checks.

A smoke test was added to make sure that traefik in this mode continues
to work. Thank you https://github.com/flifloo for filing a detailed
issue with the relevant configuration fragments. Those configuration
fragments formed the core of this smoke test.

[1]: https://doc.traefik.io/traefik/v3.4/middlewares/http/forwardauth/

Closes: https://github.com/TecharoHQ/anubis/issues/1628
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-Authored-By: flifloo <flifloo@gmail.com>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: flifloo <flifloo@gmail.com>
2026-05-14 21:37:02 -04:00
dependabot[bot] 9f479f578a build(deps): bump github.com/go-jose/go-jose/v3 from 3.0.4 to 3.0.5 (#1629)
Signed-off-by: dependabot[bot] <support@github.com>
2026-05-14 14:07:24 -04:00
dependabot[bot] c184028d42 build(deps-dev): bump the npm group across 1 directory with 6 updates (#1621)
Bumps the npm group with 6 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [@commitlint/cli](https://github.com/conventional-changelog/commitlint/tree/HEAD/@commitlint/cli) | `20.5.0` | `20.5.3` |
| [@commitlint/config-conventional](https://github.com/conventional-changelog/commitlint/tree/HEAD/@commitlint/config-conventional) | `20.5.0` | `20.5.3` |
| [baseline-browser-mapping](https://github.com/web-platform-dx/baseline-browser-mapping) | `2.10.15` | `2.10.27` |
| [cssnano](https://github.com/cssnano/cssnano) | `7.1.4` | `7.1.8` |
| [cssnano-preset-advanced](https://github.com/cssnano/cssnano) | `7.0.12` | `7.0.16` |
| [prettier](https://github.com/prettier/prettier) | `3.8.1` | `3.8.3` |



Updates `@commitlint/cli` from 20.5.0 to 20.5.3
- [Release notes](https://github.com/conventional-changelog/commitlint/releases)
- [Changelog](https://github.com/conventional-changelog/commitlint/blob/master/@commitlint/cli/CHANGELOG.md)
- [Commits](https://github.com/conventional-changelog/commitlint/commits/v20.5.3/@commitlint/cli)

Updates `@commitlint/config-conventional` from 20.5.0 to 20.5.3
- [Release notes](https://github.com/conventional-changelog/commitlint/releases)
- [Changelog](https://github.com/conventional-changelog/commitlint/blob/master/@commitlint/config-conventional/CHANGELOG.md)
- [Commits](https://github.com/conventional-changelog/commitlint/commits/v20.5.3/@commitlint/config-conventional)

Updates `baseline-browser-mapping` from 2.10.15 to 2.10.27
- [Release notes](https://github.com/web-platform-dx/baseline-browser-mapping/releases)
- [Commits](https://github.com/web-platform-dx/baseline-browser-mapping/compare/v2.10.15...v2.10.27)

Updates `cssnano` from 7.1.4 to 7.1.8
- [Release notes](https://github.com/cssnano/cssnano/releases)
- [Commits](https://github.com/cssnano/cssnano/compare/cssnano@7.1.4...cssnano@7.1.8)

Updates `cssnano-preset-advanced` from 7.0.12 to 7.0.16
- [Release notes](https://github.com/cssnano/cssnano/releases)
- [Commits](https://github.com/cssnano/cssnano/compare/cssnano-preset-advanced@7.0.12...cssnano-preset-advanced@7.0.16)

Updates `prettier` from 3.8.1 to 3.8.3
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.8.1...3.8.3)

---
updated-dependencies:
- dependency-name: "@commitlint/cli"
  dependency-version: 20.5.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
- dependency-name: "@commitlint/config-conventional"
  dependency-version: 20.5.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
- dependency-name: baseline-browser-mapping
  dependency-version: 2.10.27
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
- dependency-name: cssnano
  dependency-version: 7.1.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
- dependency-name: cssnano-preset-advanced
  dependency-version: 7.0.16
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
- dependency-name: prettier
  dependency-version: 3.8.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Jason Cameron <git@jasoncameron.dev>
2026-05-12 16:32:01 -04:00
Xe Iaso 0491f1fac2 fix: patch GHSA-6wcg-mqvh-fcvg (#1616)
* fix: patch GHSA-6wcg-mqvh-fcvg

PR https://github.com/TecharoHQ/anubis/pull/1015 added the ability for
reverse proxies using Anubis in subrequest auth mode to look at the path
of a request as there are many rules in the wild that rely on checking
the path. This is how access to things like robots.txt or anything in the
.well-known directory is unaffected by Anubis.

However this logic was also enabled for non-subrequest deployments of Anubis,
meaning that a specially crafted request could include a /.well-known/
path in it and then get around Anubis with little effort.

This fix gates the logic behind a new plumbed variable named subrequestMode
that only fires when Anubis is running in subrequest auth mode. This
properly contains that workaround so that the logic does not fire in
most deployments.

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-05-08 19:17:25 -04:00
Timon de Groot d3a00da448 feat: Log weight when issuing challenge (#1611)
This can come in handy when analyzing the logs

Signed-off-by: Timon de Groot <tdegroot96@gmail.com>
2026-05-05 16:57:45 +00:00
lillian-b 7e037b65e8 feat: add ASN data from Thoth to logs/metrics (#1608)
Assisted-by: Claude Sonnet 4.6 via Claude Code

Signed-off-by: Lillian Berry <lillian@star-ark.net>
Co-authored-by: Lillian Berry <lillian@star-ark.net>
2026-05-02 11:53:00 -04:00
Xe Iaso ebf9a30878 fix(metrics): bind to the right network/bindhost (#1606)
Whoops!

Closes: #1605

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-04-30 18:18:01 -04:00
Lenny f8605bcd3c fix: Thoth geoip compare (#1564)
Co-authored-by: Jason Cameron <git@jasoncameron.dev>
2026-04-24 14:37:19 +00:00
Xe Iaso 1d700a0370 fix(honeypot): remove DoS vector (#1581)
Using the User-Agent as a filtering vector for the honeypot maze was a
decent idea, however in practice it can become a DoS vector by a
malicious client adding a lot of points to Google Chrome's User-Agent
string. In practice it also seems that the worst offenders use vanilla
Google Chrome User-Agent strings as well, meaning that this backfires
horribly.

Gotta crack a few eggs to make omlettes.

Closes: #1580

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-04-23 09:08:34 -04:00
Xe Iaso 681c2cc2ed feat(metrics): basic auth support (#1579)
* feat(internal): add basic auth HTTP middleware

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(config): add HTTP basic auth for metrics

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(metrics): wire up basic auth

Signed-off-by: Xe Iaso <me@xeiaso.net>

* doc: document HTTP basic auth for metrics server

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs(admin/policies): give people a python command

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-04-23 00:17:09 -04:00
Xe Iaso 8f8ae76d56 feat(metrics): enable TLS/mTLS serving support (#1576)
* feat(config): add metrics TLS configuration

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(metrics): add naive TLS serving for metrics

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(metrics): import keypairreloader from a private project

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(metrics): properly surface errors with the metrics server

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(config): add CA certificate config value

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(metrics): enable mTLS support

Signed-off-by: Xe Iaso <me@xeiaso.net>

* doc(default-config): document how to set up TLS and mTLS

Signed-off-by: Xe Iaso <me@xeiaso.net>

* doc: document metrics TLS and mTLS

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-04-22 19:55:09 -04:00
Benjamin Bouvier f21706eb12 feat(data): add Meta's web indexer used for AI purposes (#1573)
This indexer is documented in
https://developers.facebook.com/docs/sharing/webmasters/web-crawlers. I
saw it parsing the entirety of my Forgejo instance, so I suggest to
widely block it.

Signed-off-by: Benjamin Bouvier <benjamin@bouvier.cc>
2026-04-21 16:56:23 -04:00
Xe Iaso d5ccf9c670 feat: move metrics server config to the policy file (#1572)
* feat(config): add metrics bind config to policy file with flag hack

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(internal): move SetupListener from main

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(main): use internal.SetupListener

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(config): add metrics socket mode

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat: move metrics server to a dedicated package

Signed-off-by: Xe Iaso <me@xeiaso.net>

* doc: add metrics server configuration docs

Signed-off-by: Xe Iaso <me@xeiaso.net>

* doc(default-config): add vague references to metrics server

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-04-21 15:36:11 -04:00
Xe Iaso 3a6e368179 Revert "ci(ssh): enable local machines again"
This reverts commit 98afcf8c64.
2026-04-18 11:30:32 -04:00
Xe Iaso 98afcf8c64 ci(ssh): enable local machines again
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-04-18 11:22:04 -04:00
dependabot[bot] 982394ca91 build(deps): bump the github-actions group across 1 directory with 7 updates (#1560)
Co-authored-by: Jason Cameron <git@jasoncameron.dev>
2026-04-14 03:35:30 +00:00
dependabot[bot] 66b7b27aef build(deps): bump the npm group across 1 directory with 5 updates (#1559)
Co-authored-by: Jason Cameron <git@jasoncameron.dev>
2026-04-13 23:31:39 -04:00
Xe Iaso cc07be8a9e chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-04-10 21:40:03 -04:00
Xe Iaso 04f8b6b148 chore(sponsors): add qwertiko and wenet
Closes: #1553
Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-04-10 21:36:40 -04:00
Xe Iaso dbd64e0f4f docs(faq): document minimum versions of browsers Anubis supports (#1540)
* docs(faq): document minimum versions of browsers Anubis supports

Closes: #1534
Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: update spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-03-24 15:59:12 +00:00
dependabot[bot] 3acf8ee387 build(deps): bump the npm group with 5 updates (#1531)
Bumps the npm group with 5 updates:

| Package | From | To |
| --- | --- | --- |
| [preact](https://github.com/preactjs/preact) | `10.28.4` | `10.29.0` |
| [@commitlint/cli](https://github.com/conventional-changelog/commitlint/tree/HEAD/@commitlint/cli) | `20.4.3` | `20.5.0` |
| [@commitlint/config-conventional](https://github.com/conventional-changelog/commitlint/tree/HEAD/@commitlint/config-conventional) | `20.4.3` | `20.5.0` |
| [baseline-browser-mapping](https://github.com/web-platform-dx/baseline-browser-mapping) | `2.10.0` | `2.10.8` |
| [esbuild](https://github.com/evanw/esbuild) | `0.27.3` | `0.27.4` |


Updates `preact` from 10.28.4 to 10.29.0
- [Release notes](https://github.com/preactjs/preact/releases)
- [Commits](https://github.com/preactjs/preact/compare/10.28.4...10.29.0)

Updates `@commitlint/cli` from 20.4.3 to 20.5.0
- [Release notes](https://github.com/conventional-changelog/commitlint/releases)
- [Changelog](https://github.com/conventional-changelog/commitlint/blob/master/@commitlint/cli/CHANGELOG.md)
- [Commits](https://github.com/conventional-changelog/commitlint/commits/v20.5.0/@commitlint/cli)

Updates `@commitlint/config-conventional` from 20.4.3 to 20.5.0
- [Release notes](https://github.com/conventional-changelog/commitlint/releases)
- [Changelog](https://github.com/conventional-changelog/commitlint/blob/master/@commitlint/config-conventional/CHANGELOG.md)
- [Commits](https://github.com/conventional-changelog/commitlint/commits/v20.5.0/@commitlint/config-conventional)

Updates `baseline-browser-mapping` from 2.10.0 to 2.10.8
- [Release notes](https://github.com/web-platform-dx/baseline-browser-mapping/releases)
- [Commits](https://github.com/web-platform-dx/baseline-browser-mapping/compare/v2.10.0...v2.10.8)

Updates `esbuild` from 0.27.3 to 0.27.4
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.27.3...v0.27.4)

---
updated-dependencies:
- dependency-name: preact
  dependency-version: 10.29.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: npm
- dependency-name: "@commitlint/cli"
  dependency-version: 20.5.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm
- dependency-name: "@commitlint/config-conventional"
  dependency-version: 20.5.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: npm
- dependency-name: baseline-browser-mapping
  dependency-version: 2.10.8
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
- dependency-name: esbuild
  dependency-version: 0.27.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-24 15:54:04 +00:00
Xe Iaso 57260a562a chore(sponsors): add embeDD GmbH (#1539)
* chore(sponsors): add embeDD GmbH

Closes: #1535
Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: update spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-03-24 15:51:32 +00:00
dependabot[bot] e42a328843 build(deps): bump actions-hub/kubectl in the github-actions group (#1532)
Bumps the github-actions group with 1 update: [actions-hub/kubectl](https://github.com/actions-hub/kubectl).


Updates `actions-hub/kubectl` from 1.35.2 to 1.35.3
- [Release notes](https://github.com/actions-hub/kubectl/releases)
- [Commits](https://github.com/actions-hub/kubectl/compare/5ada4e2c02eacc03978c2437e95c8b0f979a9619...934aaa4354bbbc3d2176ae8d7cae92d515032dff)

---
updated-dependencies:
- dependency-name: actions-hub/kubectl
  dependency-version: 1.35.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-23 15:33:43 +00:00
Michael c4b26e5a75 l10n(de): revise and update German translations (#1457)
* l10n(de): revise and update German translations

Improve strings for clarity and naturalness, update Wikipedia link to German version.

Signed-off-by: Michael <87752300+michi-onl@users.noreply.github.com>

* test(localization): fix tests

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Michael <87752300+michi-onl@users.noreply.github.com>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2026-03-21 20:02:49 +00:00
Duy 1342539a41 chore: update vi.json (#1459)
* Update vi.json

Signed-off-by: Duy <104808921+honhatduy@users.noreply.github.com>

* test(localization): fix tests

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Duy <104808921+honhatduy@users.noreply.github.com>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2026-03-21 20:01:21 +00:00
ramromcode bd6f654e1f feat(i18n): add Bulgarian language translation (#1394)
* feat(localization): add bulgarian language translation

* fix(locales/bg): make tests pass

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <xe.iaso@techaro.lol>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <xe.iaso@techaro.lol>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2026-03-21 19:56:27 +00:00
Xe Iaso d2c3a326af feat(anubis): enable pprof endpoints on the metrics server (#1530)
Closes: #1366

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-03-21 19:43:47 +00:00
dwhitemv25 c2ed62f51d fix: Address issued raised in #1193 (#1519)
* Resolve #1193

Address documentation and error message issues around REDIRECT_DOMAINS and required keywords in bot specifications.

* Add CHANGELOG entry
2026-03-21 19:32:25 +00:00
Jason Cameron 11c4adc6b4 fix: add cel iterator (#1465)
* fix: enable CEL iterators

Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>

* test: add unit tests for CELChecker map iteration

Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>

* fix: implement map iterators for HTTPHeaders and URLValues to resolve CEL internal errors

Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>

* fix: replace checker.NewMapIterator with newMapIterator for HTTPHeaders and URLValues

Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>

---------

Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>
2026-03-21 19:30:05 +00:00
BALLOON | FU-SEN edbfd180b8 locales/ja: Change the position of the バージョン (version) (#1527)
When displayed in Japanese, the `バージョン` (version) is in the middle, but the version number is at the end, so it is displayed strangely. Improve this.

**"version_info":**
```
このウェブサイトはAnubisバージョンで動作しています
```
to
```
このウェブサイトはAnubisで動作しています バージョン
```

Signed-off-by: BALLOON | FU-SEN <5434159+fu-sen@users.noreply.github.com>
2026-03-21 06:36:40 +00:00
Xe Iaso efde4f0dc7 docs(faq): document that disabling JIT makes Anubis slow (#1526)
* docs(faq): document that disabling JIT makes Anubis slow

Closes: #1520
Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: fix spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Xe Iaso <xe.iaso@techaro.lol>
2026-03-20 22:16:50 +00:00
Marielle Volz 24857f430f feat(data): add Citoid to good bots list (#1524)
* Add Wikimedia Foundation citoid services file

Wikimedia Foundation runs a service called citoid which retrieves citation metadata from urls in order to create formatted citations. 

This file contains the ip ranges allocated to the WMF (https://wikitech.wikimedia.org/wiki/IP_and_AS_allocations) from which the services make requests, as well as regex for the User-Agents from both services used to generate citations (citoid, and Zotero's translation-server which citoid makes requests to as well in order to generate the metadata).

Signed-off-by: Marielle Volz <marielle.volz@gmail.com>

* Add Wikimedia Citoid crawler to allowed list

Signed-off-by: Marielle Volz <marielle.volz@gmail.com>

* chore: update spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Marielle Volz <marielle.volz@gmail.com>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2026-03-20 11:13:26 +00:00
Simon Rozman e0ece7d333 feat(docs): Update HAProxy Advanced Variant documentation (#1521)
Added note on HAProxy's responsibility to handle Git HTTP and bot
traffic whitelisting.

Signed-off-by: Simon Rozman <simon@rozman.si>
2026-03-19 11:03:14 +00:00
fhoekstra 3eab1d873d (docs): Add instructions on using Anubis with envoy-gateway (#1460)
Signed-off-by: fhoekstra <32362869+fhoekstra@users.noreply.github.com>
2026-03-18 18:03:29 +00:00
Jason Cameron c7b31d0ca9 fix: nil ptr deref (#1467)
Signed-off-by: Jason Cameron <jason.cameron@stanwith.me>
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
2026-03-18 18:02:57 +00:00
132 changed files with 3525 additions and 821 deletions
+10
View File
@@ -31,3 +31,13 @@ Stargate
FFXIV
uvensys
de
resourced
envoyproxy
unipromos
Samsung
wenet
qwertiko
setuplistener
mba
xfu
xou
+11 -4
View File
@@ -47,6 +47,7 @@ cachediptoasn
Caddyfile
caninetools
Cardyb
CAs
celchecker
celphase
cerr
@@ -80,6 +81,7 @@ databento
dayjob
dco
DDOS
ddwrt
Debian
debrpm
decaymap
@@ -102,6 +104,7 @@ duckduckbot
eerror
ellenjoe
emacs
embe
enbyware
etld
everyones
@@ -117,9 +120,9 @@ fahedouch
fastcgi
FCr
fcrdns
fcvg
fediverse
ffprobe
FFXIV
fhdr
financials
finfos
@@ -202,8 +205,10 @@ kagi
kagibot
Keyfunc
keypair
keypairreloader
KHTML
kinda
kpr
KUBECONFIG
lcj
ldflags
@@ -221,7 +226,6 @@ LLU
loadbalancer
lol
lominsa
maintainership
malware
mcr
memes
@@ -229,17 +233,18 @@ metarefresh
metrix
mimi
Minfilia
minica
mistralai
mnt
Mojeek
mojeekbot
mozilla
mqvh
myclient
mymaster
mypass
myuser
nbf
Necron
nepeat
netsurf
nginx
@@ -313,7 +318,9 @@ screenshots
searchbot
searx
sebest
seccomp
secretplans
selfsigned
Semrush
Seo
setsebool
@@ -334,7 +341,6 @@ spyderbot
srcip
srv
stackoverflow
Stargate
startprecmd
stoppostcmd
storetest
@@ -384,6 +390,7 @@ vnd
VPS
Vultr
WAIFU
wcg
weblate
webmaster
webpage
+1 -1
View File
@@ -25,7 +25,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "24.11.0"
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "1.25.7"
+1 -1
View File
@@ -29,7 +29,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "24.11.0"
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "stable"
+2 -2
View File
@@ -39,14 +39,14 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "24.11.0"
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "stable"
- uses: ko-build/setup-ko@d006021bd0c28d1ce33a07e7943d48b079944c8d # v0.9
- name: Log into registry
uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2 # v4.0.0
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
+12 -3
View File
@@ -25,7 +25,7 @@ jobs:
uses: docker/setup-buildx-action@4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd # v4.0.0
- name: Log into registry
uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2 # v4.0.0
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: techarohq
@@ -52,15 +52,24 @@ jobs:
platforms: linux/amd64
push: true
- name: Pin docs image to built digest
working-directory: docs/manifest
env:
DIGEST: ${{ steps.build.outputs.digest }}
run: |
KUSTOMIZE_VERSION=v5.4.3
curl -fsSL "https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize/${KUSTOMIZE_VERSION}/kustomize_${KUSTOMIZE_VERSION}_linux_amd64.tar.gz" | tar -xz
./kustomize edit set image "ghcr.io/techarohq/anubis/docs=ghcr.io/techarohq/anubis/docs@${DIGEST}"
- name: Apply k8s manifests to limsa lominsa
uses: actions-hub/kubectl@5ada4e2c02eacc03978c2437e95c8b0f979a9619 # v1.35.2
uses: actions-hub/kubectl@934aaa4354bbbc3d2176ae8d7cae92d515032dff # v1.35.3
env:
KUBE_CONFIG: ${{ secrets.LIMSA_LOMINSA_KUBECONFIG }}
with:
args: apply -k docs/manifest
- name: Apply k8s manifests to limsa lominsa
uses: actions-hub/kubectl@5ada4e2c02eacc03978c2437e95c8b0f979a9619 # v1.35.2
uses: actions-hub/kubectl@934aaa4354bbbc3d2176ae8d7cae92d515032dff # v1.35.3
env:
KUBE_CONFIG: ${{ secrets.LIMSA_LOMINSA_KUBECONFIG }}
with:
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
with:
persist-credentials: false
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "stable"
+2 -2
View File
@@ -27,12 +27,12 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "24.11.0"
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "stable"
- name: Cache playwright binaries
uses: actions/cache@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
uses: actions/cache@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5.0.4
id: playwright-cache
with:
path: |
+1 -1
View File
@@ -28,7 +28,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "24.11.0"
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "stable"
@@ -29,7 +29,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "24.11.0"
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "stable"
+2 -1
View File
@@ -27,6 +27,7 @@ jobs:
- palemoon/amd64
#- palemoon/i386
- robots_txt
- traefik
runs-on: ubuntu-latest
steps:
- name: Checkout code
@@ -37,7 +38,7 @@ jobs:
- uses: actions/setup-node@53b83947a5a98c8d113130e565377fae1a50d02f # v6.3.0
with:
node-version: "24.11.0"
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "stable"
+1 -1
View File
@@ -89,7 +89,7 @@ jobs:
steps:
- name: check-spelling
id: spelling
uses: check-spelling/check-spelling@c635c2f3f714eec2fcf27b643a1919b9a811ef2e # v0.0.25
uses: check-spelling/check-spelling@cfb6f7e75bbfc89c71eaa30366d0c166f1bd9c8c # v0.0.26
with:
suppress_push_for_open_pull_request: ${{ github.actor != 'dependabot[bot]' && 1 }}
checkout: true
+1 -1
View File
@@ -24,7 +24,7 @@ jobs:
fetch-depth: 0
persist-credentials: false
- name: Log into registry
uses: docker/login-action@b45d80f862d83dbcd57f89517bcf500b2ab88fb2 # v4.0.0
uses: docker/login-action@4907a6ddec9925e35a0a9e82d7399ccc52663121 # v4.1.0
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
+2 -2
View File
@@ -30,13 +30,13 @@ jobs:
persist-credentials: false
- name: Install CI target SSH key
uses: shimataro/ssh-key-action@6b84f2e793b32fa0b03a379cadadec75cc539391 # v2.8.0
uses: shimataro/ssh-key-action@87a8f067114a8ce263df83e9ed5c849953548bc3 # v2.8.1
with:
key: ${{ secrets.CI_SSH_KEY }}
name: id_rsa
known_hosts: ${{ secrets.CI_SSH_KNOWN_HOSTS }}
- uses: actions/setup-go@4b73464bb391d4059bd26b0524d20df3927bd417 # v6.3.0
- uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
with:
go-version: "stable"
+2 -2
View File
@@ -21,7 +21,7 @@ jobs:
persist-credentials: false
- name: Install the latest version of uv
uses: astral-sh/setup-uv@eac588ad8def6316056a12d4907a9d4d84ff7a3b # v7.3.0
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57 # v8.0.0
- name: Run zizmor 🌈
run: uvx zizmor --format sarif . > results.sarif
@@ -29,7 +29,7 @@ jobs:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@5d4e8d1aca955e8d8589aabd499c5cae939e33c7 # v4.31.9
uses: github/codeql-action/upload-sarif@c10b8064de6f491fea524254123dbe5e09572f13 # v4.35.1
with:
sarif_file: results.sarif
category: zizmor
+9
View File
@@ -73,6 +73,15 @@ Anubis is brought to you by sponsors and donors like:
<a href="https://www.anexia.com/">
<img src="./docs/static/img/sponsors/anexia-cloudsolutions-logo.webp" alt="ANEXIA Cloud Solutions" height="64">
</a>
<a href="https://dd-wrt.com/">
<img src="./docs/static/img/sponsors/ddwrt-logo.webp" alt="embeDD GmbH" height="64">
</a>
<a href="https://www.qwertiko.de?utm_campaign=github&utm_medium=referral&utm_content=anubis">
<img src="./docs/static/img/sponsors/qwertiko-logo.webp" alt="Qwertiko" height="64">
</a>
<a href="https://wenet.pl/?utm_campaign=github&utm_medium=referral&utm_content=anubis">
<img src="./docs/static/img/sponsors/wenet-logo.webp" alt="Wenet" height="64">
</a>
## Overview
+28 -126
View File
@@ -21,7 +21,6 @@ import (
"os"
"os/signal"
"path/filepath"
"strconv"
"strings"
"sync"
"syscall"
@@ -32,12 +31,12 @@ import (
"github.com/TecharoHQ/anubis/internal"
libanubis "github.com/TecharoHQ/anubis/lib"
"github.com/TecharoHQ/anubis/lib/config"
"github.com/TecharoHQ/anubis/lib/metrics"
botPolicy "github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/lib/thoth"
"github.com/TecharoHQ/anubis/web"
"github.com/facebookgo/flagenv"
_ "github.com/joho/godotenv/autoload"
"github.com/prometheus/client_golang/prometheus/promhttp"
healthv1 "google.golang.org/grpc/health/grpc_health_v1"
)
@@ -118,33 +117,6 @@ func doHealthCheck() error {
return nil
}
// parseBindNetFromAddr determine bind network and address based on the given network and address.
func parseBindNetFromAddr(address string) (string, string) {
defaultScheme := "http://"
if !strings.Contains(address, "://") {
if strings.HasPrefix(address, ":") {
address = defaultScheme + "localhost" + address
} else {
address = defaultScheme + address
}
}
bindUri, err := url.Parse(address)
if err != nil {
log.Fatal(fmt.Errorf("failed to parse bind URL: %w", err))
}
switch bindUri.Scheme {
case "unix":
return "unix", bindUri.Path
case "tcp", "http", "https":
return "tcp", bindUri.Host
default:
log.Fatal(fmt.Errorf("unsupported network scheme %s in address %s", bindUri.Scheme, address))
}
return "", address
}
func parseSameSite(s string) http.SameSite {
switch strings.ToLower(s) {
case "none":
@@ -161,53 +133,6 @@ func parseSameSite(s string) http.SameSite {
return http.SameSiteDefaultMode
}
func setupListener(network string, address string) (net.Listener, string) {
formattedAddress := ""
if network == "" {
// keep compatibility
network, address = parseBindNetFromAddr(address)
}
switch network {
case "unix":
formattedAddress = "unix:" + address
case "tcp":
if strings.HasPrefix(address, ":") { // assume it's just a port e.g. :4259
formattedAddress = "http://localhost" + address
} else {
formattedAddress = "http://" + address
}
default:
formattedAddress = fmt.Sprintf(`(%s) %s`, network, address)
}
listener, err := net.Listen(network, address)
if err != nil {
log.Fatal(fmt.Errorf("failed to bind to %s: %w", formattedAddress, err))
}
// additional permission handling for unix sockets
if network == "unix" {
mode, err := strconv.ParseUint(*socketMode, 8, 0)
if err != nil {
listener.Close()
log.Fatal(fmt.Errorf("could not parse socket mode %s: %w", *socketMode, err))
}
err = os.Chmod(address, os.FileMode(mode))
if err != nil {
err := listener.Close()
if err != nil {
log.Printf("failed to close listener: %v", err)
}
log.Fatal(fmt.Errorf("could not change socket mode: %w", err))
}
}
return listener, formattedAddress
}
func makeReverseProxy(target string, targetSNI string, targetHost string, insecureSkipVerify bool, targetDisableKeepAlive bool) (http.Handler, error) {
targetUri, err := url.Parse(target)
if err != nil {
@@ -303,11 +228,6 @@ func main() {
wg := new(sync.WaitGroup)
if *metricsBind != "" {
wg.Add(1)
go metricsServer(ctx, *lg.With("subsystem", "metrics"), wg.Done)
}
var rp http.Handler
// when using anubis via Systemd and environment variables, then it is not possible to set targe to an empty string but only to space
if strings.TrimSpace(*target) != "" {
@@ -339,7 +259,7 @@ func main() {
}
lg.Info("loading policy file", "fname", *policyFname)
policy, err := libanubis.LoadPoliciesOrDefault(ctx, *policyFname, *challengeDifficulty, *slogLevel)
policy, err := libanubis.LoadPoliciesOrDefault(ctx, *policyFname, *challengeDifficulty, *slogLevel, strings.TrimSpace(*target) == "")
if err != nil {
log.Fatalf("can't parse policy file: %v", err)
}
@@ -347,6 +267,26 @@ func main() {
lg.Debug("swapped to new logger")
slog.SetDefault(lg)
if *metricsBind != "" || policy.Metrics != nil {
wg.Add(1)
ms := &metrics.Server{
Config: policy.Metrics,
Log: lg,
}
if policy.Metrics == nil {
lg.Debug("migrating flags to metrics config", "bind", *metricsBind, "network", *metricsBindNetwork, "socket-mode", *socketMode)
ms.Config = &config.Metrics{
Bind: *metricsBind,
Network: *metricsBindNetwork,
SocketMode: *socketMode,
}
}
go ms.Run(ctx, wg.Done)
}
// Warn if persistent storage is used without a configured signing key
if policy.Store.IsPersistent() {
if *hs512Secret == "" && *ed25519PrivateKeyHex == "" && *ed25519PrivateKeyHexFile == "" {
@@ -427,7 +367,7 @@ func main() {
redirectDomainsList = append(redirectDomainsList, strings.TrimSpace(domain))
}
} else {
lg.Warn("REDIRECT_DOMAINS is not set, Anubis will only redirect to the same domain a request is coming from, see https://anubis.techaro.lol/docs/admin/configuration/redirect-domains")
lg.Warn("REDIRECT_DOMAINS is not set, Anubis will redirect to any domain, see https://anubis.techaro.lol/docs/admin/configuration/redirect-domains")
}
anubis.CookieName = *cookiePrefix + "-auth"
@@ -483,7 +423,11 @@ func main() {
h = internal.JA4H(h)
srv := http.Server{Handler: h, ErrorLog: internal.GetFilteredHTTPLogger()}
listener, listenerUrl := setupListener(*bindNetwork, *bind)
listener, listenerUrl, err := internal.SetupListener(*bindNetwork, *bind, *socketMode)
if err != nil {
log.Fatalf("SetupListener(%q, %q, %q): %v", *bindNetwork, *bind, *socketMode, err)
}
lg.Info(
"listening",
"url", listenerUrl,
@@ -518,48 +462,6 @@ func main() {
wg.Wait()
}
func metricsServer(ctx context.Context, lg slog.Logger, done func()) {
defer done()
mux := http.NewServeMux()
mux.Handle("/metrics", promhttp.Handler())
mux.HandleFunc("/healthz", func(w http.ResponseWriter, r *http.Request) {
st, ok := internal.GetHealth("anubis")
if !ok {
slog.Error("health service anubis does not exist, file a bug")
}
switch st {
case healthv1.HealthCheckResponse_NOT_SERVING:
http.Error(w, "NOT OK", http.StatusInternalServerError)
return
case healthv1.HealthCheckResponse_SERVING:
fmt.Fprintln(w, "OK")
return
default:
http.Error(w, "UNKNOWN", http.StatusFailedDependency)
return
}
})
srv := http.Server{Handler: mux, ErrorLog: internal.GetFilteredHTTPLogger()}
listener, metricsUrl := setupListener(*metricsBindNetwork, *metricsBind)
lg.Debug("listening for metrics", "url", metricsUrl)
go func() {
<-ctx.Done()
c, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := srv.Shutdown(c); err != nil {
log.Printf("cannot shut down: %v", err)
}
}()
if err := srv.Serve(listener); !errors.Is(err, http.ErrServerClosed) {
log.Fatal(err)
}
}
func extractEmbedFS(fsys embed.FS, root string, destDir string) error {
return fs.WalkDir(fsys, root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
+30
View File
@@ -166,6 +166,36 @@ status_codes:
CHALLENGE: 200
DENY: 200
# # Configuration for the metrics server. See the docs for more information:
# #
# # https://anubis.techaro.lol/docs/admin/policies#metrics-server
# #
# # This is commented out by default so that command line flags take precedence.
# metrics:
# bind: ":9090"
# network: "tcp"
#
# # To protect your metrics server with basic auth, set credentials below:
# #
# # https://anubis.techaro.lol/docs/admin/policies#http-basic-authentication
# basicAuth:
# username: ""
# password: ""
#
# # To serve metrics over TLS, set the path to the right TLS certificate and key
# # here. When the files change on disk, they will automatically be reloaded.
# #
# # https://anubis.techaro.lol/docs/admin/policies#tls
# tls:
# certificate: /path/to/tls.crt
# key: /path/to/tls.key
#
# # If you want to secure your metrics endpoint using mutual TLS (mTLS), set
# # the path to a certificate authority public certificate here.
# #
# # https://anubis.techaro.lol/docs/admin/policies#mtls
# ca: /path/to/ca.crt
# Anubis can store temporary data in one of a few backends. See the storage
# backends section of the docs for more information:
#
+1
View File
@@ -8,4 +8,5 @@
- import: (data)/crawlers/marginalia.yaml
- import: (data)/crawlers/mojeekbot.yaml
- import: (data)/crawlers/commoncrawl.yaml
- import: (data)/crawlers/wikimedia-citoid.yaml
- import: (data)/crawlers/yandexbot.yaml
+1 -1
View File
@@ -4,5 +4,5 @@
# - Claude-SearchBot: No published IP allowlist
- name: "ai-crawlers-search"
user_agent_regex: >-
OAI-SearchBot|Claude-SearchBot|PerplexityBot
OAI-SearchBot|Claude-SearchBot|PerplexityBot|meta-webindexer
action: DENY
+18
View File
@@ -0,0 +1,18 @@
# Wikimedia Foundation citation services
# https://www.mediawiki.org/wiki/Citoid
- name: wikimedia-citoid
user_agent_regex: "Citoid/WMF"
action: ALLOW
remote_addresses: [
"208.80.152.0/22",
"2620:0:860::/46",
]
- name: wikimedia-zotero-translation-server
user_agent_regex: "ZoteroTranslationServer/WMF"
action: ALLOW
remote_addresses: [
"208.80.152.0/22",
"2620:0:860::/46",
]
+2 -2
View File
@@ -1,11 +1,11 @@
FROM docker.io/library/node:lts AS build
FROM docker.io/library/node:22.22.3-alpine AS build
WORKDIR /app
COPY . .
RUN npm ci && npm run build
FROM ghcr.io/xe/nginx-micro
FROM ghcr.io/xe/nginx-micro:v1.29.0
COPY --from=build /app/build /www
COPY ./manifest/cfg/nginx/nginx.conf /conf
LABEL org.opencontainers.image.source="https://github.com/TecharoHQ/anubis"
+25 -2
View File
@@ -11,10 +11,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
<!-- This changes the project to: -->
- Patch [GHSA-6wcg-mqvh-fcvg](https://github.com/TecharoHQ/anubis/security/advisories/GHSA-6wcg-mqvh-fcvg) by containing subrequest logic to Anubis instances in subrequest mode.
- Implement robot9001 style delays on the honeypot feature so that the first hit takes 1 millisecond, the second takes 2, etc.
- Move metrics server configuration to [the policy file](./admin/policies.mdx#metrics-server).
- Expose [pprof endpoints](https://pkg.go.dev/net/http/pprof) on the metrics listener to enable profiling Anubis in production.
- fix: prevent nil pointer panic in challenge validation when threshold rules match during PassChallenge (#1463)
- Instruct reverse proxies to not cache error pages.
- Fixed mixed tab/space indentation in Caddy documentation code block
<!-- This changes the project to: -->
- Improve error messages and fix broken REDIRECT_DOMAINS link in docs ([#1193](https://github.com/TecharoHQ/anubis/issues/1193))
- Add Bulgarian locale ([#1394](https://github.com/TecharoHQ/anubis/pull/1394))
- Fixed case-sensitivity mismatch in geoipchecker.go
- Fix CEL internal errors when iterating `headers`/`query` map wrappers by implementing map iterators for `HTTPHeaders` and `URLValues` ([#1465](https://github.com/TecharoHQ/anubis/pull/1465)).
- Enable [metrics serving via TLS](./admin/policies.mdx#tls), including [mutual TLS (mTLS)](./admin/policies.mdx#mtls).
- Enable [HTTP basic auth](./admin/policies.mdx#http-basic-authentication) for the metrics server.
- Fix a bug in the dataset poisoning maze that could allow denial of service [#1580](https://github.com/TecharoHQ/anubis/issues/1580).
- Add config option to add ASN to logs/metrics.
- Log weight when issuing challenge.
- Gate pprof endpoints behind `metrics.debug` in the policy file.
- Limit naive honeypot r9k delay to one second.
- Fix an obscure case where adding query values to a subrequest match could cause an invalid rule match when using path based matching for protected resources.
- Fix an edge case where load average expression values could nil pointer dereference when Anubis just started up.
- Fix an obscure case where Anubis in subrequest mode could allow redirects to invalid domains with strange instructions.
- Fix `path_regex` and CEL `path` rules not matching when using Traefik `forwardAuth` middleware. Anubis now checks `X-Forwarded-Uri` (Traefik) in addition to `X-Original-URI` (nginx) when resolving the request path in subrequest mode ([#1628](https://github.com/TecharoHQ/anubis/issues/1628)).
- Validate bounds in the CEL `randInt` helper so non-positive or platform-overflowing arguments surface a typed CEL error instead of an evaluator panic.
- Harden the public docs deployment: add a pod-level security context to the nginx container (non-root uid 101, dropped capabilities, read-only root filesystem, `RuntimeDefault` seccomp) and rebind it to unprivileged port `8080`.
- Pin docs deployment images to immutable digests with `imagePullPolicy: IfNotPresent`, and have the docs-deploy workflow overlay the just-built digest via `kustomize edit set image` so each rollout references an auditable artifact instead of a floating `:main` tag. The docs `Dockerfile` now pins `node` and `nginx-micro` base images to specific versions.
## v1.25.0: Necron
+2
View File
@@ -48,6 +48,8 @@ This simply enables SSL offloading, sets some useful and required headers and ro
Due to the fact that HAProxy can decode JWT, we are able to verify the Anubis token directly in HAProxy and route the traffic to the specific backends ourselves.
Mind that rule logic to allow Git HTTP and other legit bot traffic to bypass is delegated from Anubis to HAProxy then. If required, you should implement any whitelisting in HAProxy using `acl_anubis_ignore` yourself.
In this example are three applications behind one HAProxy frontend. Only App1 and App2 are secured via Anubis; App3 is open for everyone. The path `/excluded/path` can also be accessed by anyone.
```mermaid
@@ -130,3 +130,52 @@ Then point your Ingress to the Anubis port:
# diff-add
name: anubis
```
## Envoy Gateway
If you are using envoy-gateway, the `X-Real-Ip` header is not set by default, but Anubis does require it. You can resolve this by adding the header, either on the specific `HTTPRoute` where Anubis is listening, or on the `ClientTrafficPolicy` to apply it to any number of Gateways:
HTTPRoute:
```yaml
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: app-route
spec:
hostnames: ["app.domain.tld"]
parentRefs:
- name: envoy-external
namespace: network
sectionName: https
rules:
- backendRefs:
- identifier: *app
port: anubis
filters:
- type: RequestHeaderModifier
requestHeaderModifier:
set:
- name: X-Real-Ip
value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
```
Applying to any number of Gateways:
```yaml
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
name: envoy
spec:
headers:
earlyRequestHeaders:
set:
- name: X-Real-Ip
value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
clientIPDetection:
xForwardedFor:
trustedCIDRs:
- 10.96.0.0/16 # Cluster pod CIDR
targetSelectors: # These will apply to all Gateways
- group: gateway.networking.k8s.io
kind: Gateway
```
+3 -3
View File
@@ -87,15 +87,15 @@ Anubis uses these environment variables for configuration:
| `ED25519_PRIVATE_KEY_HEX_FILE` | unset | Path to a file containing the hex-encoded ed25519 private key. Only one of this or its sister option may be set. **Required when using persistent storage backends** (like bbolt) to ensure challenges survive service restarts. When running multiple instances on the same base domain, the key must be the same across all instances. |
| `ERROR_TITLE` | unset | <EO /> If set, override the translation stack to show a custom title for error pages such as "Something went wrong!". See [Customizing messages](./botstopper.mdx#customizing-messages) for more details. |
| `JWT_RESTRICTION_HEADER` | `X-Real-IP` | If set, the JWT is only valid if the current value of this header matches the value when the JWT was created. You can use it e.g. to restrict a JWT to the source IP of the user using `X-Real-IP`. |
| `METRICS_BIND` | `:9090` | The network address that Anubis serves Prometheus metrics on. See `BIND` for more information. |
| `METRICS_BIND_NETWORK` | `tcp` | The address family that the Anubis metrics server listens on. See `BIND_NETWORK` for more information. |
| `METRICS_BIND` | `:9090` | The legacy configuration value for the network address that Anubis serves Prometheus metrics on. Please migrate this to [the policy file](./policies.mdx#metrics-server) as soon as possible. |
| `METRICS_BIND_NETWORK` | `tcp` | The legacy configuration value for the address family that Anubis serves Prometheus metrics on. Please migrate this to [the policy file](./policies.mdx#metrics-server) as soon as possible. |
| `OG_EXPIRY_TIME` | `24h` | The expiration time for the Open Graph tag cache. Prefer using [the policy file](./configuration/open-graph.mdx) to configure the Open Graph subsystem. |
| `OG_PASSTHROUGH` | `false` | If set to `true`, Anubis will enable Open Graph tag passthrough. Prefer using [the policy file](./configuration/open-graph.mdx) to configure the Open Graph subsystem. |
| `OG_CACHE_CONSIDER_HOST` | `false` | If set to `true`, Anubis will consider the host in the Open Graph tag cache key. Prefer using [the policy file](./configuration/open-graph.mdx) to configure the Open Graph subsystem. |
| `OVERLAY_FOLDER` | unset | <EO /> If set, treat the given path as an [overlay folder](./botstopper.mdx#custom-images-and-css), allowing you to customize CSS, fonts, images, and add other assets to BotStopper deployments. |
| `POLICY_FNAME` | unset | The file containing [bot policy configuration](./policies.mdx). See the bot policy documentation for more details. If unset, the default bot policy configuration is used. |
| `PUBLIC_URL` | unset | The externally accessible URL for this Anubis instance, used for constructing redirect URLs (e.g., for Traefik forwardAuth). Leave it unset when Anubis terminates traffic directly (sidecar/standalone deployments) or redirect building will fail with `redir=null`. |
| `REDIRECT_DOMAINS` | unset | Comma-separated list of domain names that Anubis should allow redirects to when passing a challenge. See [Redirect Domain Configuration](./configuration/redirect-domains) for more details. |
| `REDIRECT_DOMAINS` | unset | Comma-separated list of domain names that Anubis should allow redirects to when passing a challenge. See [Redirect Domain Configuration](./configuration/redirect-domains.mdx) for more details. |
| `SERVE_ROBOTS_TXT` | `false` | If set `true`, Anubis will serve a default `robots.txt` file that disallows all known AI scrapers by name and then additionally disallows every scraper. This is useful if facts and circumstances make it difficult to change the underlying service to serve such a `robots.txt` file. |
| `SLOG_LEVEL` | `INFO` | The log level for structured logging. Valid values are `DEBUG`, `INFO`, `WARN`, and `ERROR`. Set to `DEBUG` to see all requests, evaluations, and detailed diagnostic information. |
| `SOCKET_MODE` | `0770` | _Only used when at least one of the `*_BIND_NETWORK` variables are set to `unix`._ The socket mode (permissions) for Unix domain sockets. |
+91
View File
@@ -117,6 +117,96 @@ remote_addresses:
- 100.64.0.0/10
```
## Metrics server
Anubis includes support for [Prometheus-style metrics](https://prometheus.io/docs/introduction/overview/), allowing systems administrators to monitor Anubis' performance and effectiveness. This is a separate HTTP server with metrics, health checking, and debug routes.
Anubis' metrics server is configured with the `metrics` block in the configuration file:
```yaml
metrics:
bind: ":9090"
network: "tcp"
```
If you want to bind metrics to a Unix socket, make sure to set the network to `unix` and add a socket mode:
```yaml
metrics:
bind: "/tmp/anubis_metrics.sock"
network: unix
socketMode: "0700" # must be a string
```
### Debug routes
Anubis' metrics server supports [pprof](https://pkg.go.dev/runtime/pprof), the Go standard library tool for profiling Go applications. This is very useful for debugging how Anubis works in the wild with regards to CPU, multicore, and RAM usage. pprof is very powerful and can expose command line arguments as part of the debugging setup (inside Google, everything is done with command line flags).
Prior versions of Anubis exposed pprof endpoints on all TCP bindhosts by default. This means that machines with incorrectly configured firewalls can expose command line arguments to the public internet in the right conditions.
In order to enable pprof profiling endpoints on the Metrics server, set the `debug` flag under the `metrics` block:
```yaml
metrics:
bind: ":9090"
network: "tcp"
debug: true
```
To err on the side of caution, this defaults to disabled. If this defaults migration breaks your configuration, please let us know in a ticket.
### TLS
If you want to serve the metrics server over TLS, use the `tls` block:
```yaml
metrics:
bind: ":9090"
network: "tcp"
tls:
certificate: /path/to/tls.crt
key: /path/to/tls.key
```
The certificate and key will automatically be reloaded when the respective files change.
### mTLS
If you want to validate requests to ensure that they use a client certificate signed by a certificate authority (mutual TLS or mTLS), set the `ca` value in the `tls` block:
```yaml
metrics:
bind: ":9090"
network: "tcp"
tls:
certificate: /path/to/tls.crt
key: /path/to/tls.key
ca: /path/to/ca.crt
```
As it is not expected for certificate authority certificates to change often, the CA certificate will NOT be automatically reloaded when the respective file changes.
### HTTP basic authentication
Anubis' metrics server also supports setting HTTP basic auth as a lightweight protection against unauthorized users viewing metrics data. As the basic auth credentials are hardcoded in the configuration file, administrators SHOULD use randomly generated credentials, such as type-4 UUIDs or other high entropy strings. These credentials MUST NOT be sensitive or used to protect other high value systems.
Configure it with the `basicAuth` block under `metrics`:
```yaml
metrics:
bind: ":9090"
network: "tcp"
basicAuth:
username: azurediamond
password: hunter2
```
If you have Python installed, you can generate a high entropy password with `python -c 'import secrets; print(secrets.token_urlsafe(32))'`.
## Imprint / Impressum support
Anubis has support for showing imprint / impressum information. This is defined in the `impressum` block of your configuration. See [Imprint / Impressum configuration](./configuration/impressum.mdx) for more information.
@@ -339,6 +429,7 @@ Anubis exposes the following logging settings in the policy file:
| `level` | [log level](#log-levels) | `info` | The logging level threshold. Any logs that are at or above this threshold will be drained to the sink. Any other logs will be discarded. |
| `sink` | string | `stdio`, `file` | The sink where the logs drain to as they are being recorded in Anubis. |
| `parameters` | object | | Parameters for the given logging sink. This will vary based on the logging sink of choice. See below for more information. |
| `asn` | bool | `true`, `false` | Add ASN information to logs/metrics. (Requires a Thoth client configured) |
Anubis supports the following logging sinks:
+9
View File
@@ -87,6 +87,15 @@ Anubis is brought to you by sponsors and donors like:
height="64"
/>
</a>
<a href="https://dd-wrt.com/">
<img src="/img/sponsors/ddwrt-logo.webp" alt="embeDD GmbH" height="64" />
</a>
<a href="https://www.qwertiko.de?utm_campaign=github&utm_medium=referral&utm_content=anubis">
<img src="/img/sponsors/qwertiko-logo.webp" alt="Qwertiko" height="64" />
</a>
<a href="https://wenet.pl/?utm_campaign=github&utm_medium=referral&utm_content=anubis">
<img src="/img/sponsors/wenet-logo.webp" alt="Wenet" height="64" />
</a>
## Overview
@@ -22,3 +22,24 @@ If you use a browser extension such as [JShelter](https://jshelter.org/), you wi
## Does Anubis mine Bitcoin?
No. Anubis does not mine Bitcoin or any other cryptocurrency.
## I disabled Just-in-time compilation in my browser. Why is Anubis slow?
Anubis proof-of-work checks run an open source JavaScript program in your browser. These checks do a lot of complicated math and aim to be done quickly, so the execution speed depends on [Just-in-time (JIT) compilation](https://en.wikipedia.org/wiki/Just-in-time_compilation). JIT compiles JavaScript from the Internet into native machine code at runtime. The code produced by the JIT engine is almost as good as if it was written in a native programming language and compiled for your computer in particular. Without JIT, all JavaScript programs on every website you visit run through a slow interpreter.
This interpreter is much slower than native code because it has to translate each low level JavaScript operation into many dozens of calls to execute. This means that using the interpreter incurs a massive performance hit by its very nature; it takes longer to add numbers than if the CPU just added the numbers directly.
Some users choose to disable JIT as a hardening measure against theoretical browser exploits. This is a reasonable choice if you face targeted attacks from well-resourced adversaries (such as nation-state actors), but it comes with real performance costs.
If you've disabled JIT and find Anubis checks slow, re-enabling JIT is the fix. There is no way for Anubis to work around this on our end.
## What versions of browsers does Anubis support?
Anubis is written mainly by a single person in a basement in Canada. As such it is impossible for Anubis to support every version of every browser on the planet. As such, here's a few rules of thumb for the browsers that Anubis focuses on supporting:
- At least the two (2) most recent LTS releases of Firefox and Chrome.
- At least the version of Chromium as used by the Samsung Browser on Android.
- At least the last version of Chromium and Firefox that are known to run on Windows 7.
- At least the version of Safari that runs on the second-to-oldest iPhone model currently on the market.
We cannot give more cohesive version bounds than this. If you run into problems, please file an issue. Sometimes you may just need to upgrade hardware though.
+8 -2
View File
@@ -1,7 +1,7 @@
user nginx;
worker_processes 2;
error_log /dev/stdout warn;
pid /nginx.pid;
pid /tmp/nginx.pid;
events {
worker_connections 1024;
@@ -12,11 +12,17 @@ http {
default_type application/octet-stream;
access_log /dev/stdout;
client_body_temp_path /tmp/client_body;
proxy_temp_path /tmp/proxy;
fastcgi_temp_path /tmp/fastcgi;
uwsgi_temp_path /tmp/uwsgi;
scgi_temp_path /tmp/scgi;
sendfile on;
keepalive_timeout 65;
server {
listen 80 default_server;
listen 8080 default_server;
server_name _;
error_page 404 /404.html;
+23 -8
View File
@@ -20,10 +20,12 @@ spec:
name: nginx-cfg
- name: temporary-data
emptyDir: {}
- name: nginx-tmp
emptyDir: {}
containers:
- name: anubis-docs
image: ghcr.io/techarohq/anubis/docs:main
imagePullPolicy: Always
image: ghcr.io/techarohq/anubis/docs@sha256:f13a7c862bbcba8e19feba9f157120c6f03e23b03367ace4ca55da69dc894e12
imagePullPolicy: IfNotPresent
resources:
limits:
memory: "128Mi"
@@ -34,23 +36,36 @@ spec:
volumeMounts:
- name: nginx
mountPath: /conf
- name: nginx-tmp
mountPath: /tmp
ports:
- containerPort: 80
- containerPort: 8080
readinessProbe:
httpGet:
path: /
port: 80
port: 8080
initialDelaySeconds: 1
periodSeconds: 10
livenessProbe:
httpGet:
path: /
port: 80
port: 8080
initialDelaySeconds: 10
periodSeconds: 20
securityContext:
runAsUser: 101
runAsGroup: 101
runAsNonRoot: true
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
- name: anubis
image: ghcr.io/techarohq/anubis:main
imagePullPolicy: Always
image: ghcr.io/techarohq/anubis@sha256:533e57956ae3afd1612dab16f02dd4db937ca14fad5399208f403686e14feed5
imagePullPolicy: IfNotPresent
env:
- name: "BIND"
value: ":8081"
@@ -65,7 +80,7 @@ spec:
- name: "SERVE_ROBOTS_TXT"
value: "false"
- name: "TARGET"
value: "http://localhost:80"
value: "http://localhost:8080"
# - name: "SLOG_LEVEL"
# value: "debug"
volumeMounts:
+1 -1
View File
@@ -7,7 +7,7 @@ spec:
app: anubis-docs
ports:
- port: 80
targetPort: 80
targetPort: 8080
name: http
- port: 8081
targetPort: 8081
Binary file not shown.

After

Width:  |  Height:  |  Size: 4.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.7 KiB

+1 -1
View File
@@ -106,7 +106,7 @@ require (
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 // indirect
github.com/go-git/go-billy/v5 v5.6.2 // indirect
github.com/go-git/go-git/v5 v5.16.2 // indirect
github.com/go-jose/go-jose/v3 v3.0.4 // indirect
github.com/go-jose/go-jose/v3 v3.0.5 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
+2 -2
View File
@@ -189,8 +189,8 @@ github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399 h1:eMj
github.com/go-git/go-git-fixtures/v4 v4.3.2-0.20231010084843-55a94097c399/go.mod h1:1OCfN199q1Jm3HZlxleg+Dw/mwps2Wbk9frAWm+4FII=
github.com/go-git/go-git/v5 v5.16.2 h1:fT6ZIOjE5iEnkzKyxTHK1W4HGAsPhqEqiSAssSO77hM=
github.com/go-git/go-git/v5 v5.16.2/go.mod h1:4Ge4alE/5gPs30F2H1esi2gPd69R0C39lolkucHBOp8=
github.com/go-jose/go-jose/v3 v3.0.4 h1:Wp5HA7bLQcKnf6YYao/4kpRpVMp/yf6+pJKV8WFSaNY=
github.com/go-jose/go-jose/v3 v3.0.4/go.mod h1:5b+7YgP7ZICgJDBdfjZaIt+H/9L9T/YQrVfLAMboGkQ=
github.com/go-jose/go-jose/v3 v3.0.5 h1:BLLJWbC4nMZOfuPVxoZIxeYsn6Nl2r1fITaJ78UQlVQ=
github.com/go-jose/go-jose/v3 v3.0.5/go.mod h1:5b+7YgP7ZICgJDBdfjZaIt+H/9L9T/YQrVfLAMboGkQ=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
+52
View File
@@ -0,0 +1,52 @@
package internal
import (
"crypto/sha256"
"crypto/subtle"
"fmt"
"log/slog"
"net/http"
)
// BasicAuth wraps next in HTTP Basic authentication using the provided
// credentials. If either username or password is empty, next is returned
// unchanged and a debug log line is emitted.
//
// Credentials are compared in constant time to avoid leaking information
// through timing side channels.
func BasicAuth(realm, username, password string, next http.Handler) http.Handler {
if username == "" || password == "" {
slog.Debug("skipping middleware, basic auth credentials are empty")
return next
}
expectedUser := sha256.Sum256([]byte(username))
expectedPass := sha256.Sum256([]byte(password))
challenge := fmt.Sprintf("Basic realm=%q, charset=\"UTF-8\"", realm)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
user, pass, ok := r.BasicAuth()
if !ok {
unauthorized(w, challenge)
return
}
gotUser := sha256.Sum256([]byte(user))
gotPass := sha256.Sum256([]byte(pass))
userMatch := subtle.ConstantTimeCompare(gotUser[:], expectedUser[:])
passMatch := subtle.ConstantTimeCompare(gotPass[:], expectedPass[:])
if userMatch&passMatch != 1 {
unauthorized(w, challenge)
return
}
next.ServeHTTP(w, r)
})
}
func unauthorized(w http.ResponseWriter, challenge string) {
w.Header().Set("WWW-Authenticate", challenge)
http.Error(w, "Unauthorized", http.StatusUnauthorized)
}
+138
View File
@@ -0,0 +1,138 @@
package internal
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func okHandler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("ok"))
})
}
func TestBasicAuth(t *testing.T) {
t.Parallel()
const (
realm = "test-realm"
username = "admin"
password = "hunter2"
)
for _, tt := range []struct {
name string
setAuth bool
user string
pass string
wantStatus int
wantBody string
wantChall bool
}{
{
name: "valid credentials",
setAuth: true,
user: username,
pass: password,
wantStatus: http.StatusOK,
wantBody: "ok",
},
{
name: "missing credentials",
setAuth: false,
wantStatus: http.StatusUnauthorized,
wantChall: true,
},
{
name: "wrong username",
setAuth: true,
user: "nobody",
pass: password,
wantStatus: http.StatusUnauthorized,
wantChall: true,
},
{
name: "wrong password",
setAuth: true,
user: username,
pass: "wrong",
wantStatus: http.StatusUnauthorized,
wantChall: true,
},
{
name: "empty supplied credentials",
setAuth: true,
user: "",
pass: "",
wantStatus: http.StatusUnauthorized,
wantChall: true,
},
} {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
h := BasicAuth(realm, username, password, okHandler())
req := httptest.NewRequest(http.MethodGet, "/", nil)
if tt.setAuth {
req.SetBasicAuth(tt.user, tt.pass)
}
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != tt.wantStatus {
t.Errorf("status = %d, want %d", rec.Code, tt.wantStatus)
}
if tt.wantBody != "" && rec.Body.String() != tt.wantBody {
t.Errorf("body = %q, want %q", rec.Body.String(), tt.wantBody)
}
chall := rec.Header().Get("WWW-Authenticate")
if tt.wantChall {
if chall == "" {
t.Error("WWW-Authenticate header missing on 401")
}
if !strings.Contains(chall, realm) {
t.Errorf("WWW-Authenticate = %q, want realm %q", chall, realm)
}
} else if chall != "" {
t.Errorf("unexpected WWW-Authenticate header: %q", chall)
}
})
}
}
func TestBasicAuthPassthrough(t *testing.T) {
t.Parallel()
for _, tt := range []struct {
name string
username string
password string
}{
{name: "empty username", username: "", password: "hunter2"},
{name: "empty password", username: "admin", password: ""},
{name: "both empty", username: "", password: ""},
} {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
h := BasicAuth("realm", tt.username, tt.password, okHandler())
req := httptest.NewRequest(http.MethodGet, "/", nil)
rec := httptest.NewRecorder()
h.ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Errorf("status = %d, want %d (passthrough expected)", rec.Code, http.StatusOK)
}
if rec.Body.String() != "ok" {
t.Errorf("body = %q, want %q", rec.Body.String(), "ok")
}
})
}
}
+17 -22
View File
@@ -5,6 +5,7 @@ import (
_ "embed"
"fmt"
"log/slog"
"math"
"math/rand/v2"
"net/http"
"net/netip"
@@ -76,13 +77,6 @@ type Impl struct {
affirmation, body, title spintax.Spintax
}
func (i *Impl) incrementUA(ctx context.Context, userAgent string) int {
result, _ := i.uaWeight.Get(ctx, internal.SHA256sum(userAgent))
result++
i.uaWeight.Set(ctx, internal.SHA256sum(userAgent), result, time.Hour)
return result
}
func (i *Impl) incrementNetwork(ctx context.Context, network string) int {
result, _ := i.networkWeight.Get(ctx, internal.SHA256sum(network))
result++
@@ -90,20 +84,19 @@ func (i *Impl) incrementNetwork(ctx context.Context, network string) int {
return result
}
func (i *Impl) CheckUA() checker.Impl {
return checker.Func(func(r *http.Request) (bool, error) {
result, _ := i.uaWeight.Get(r.Context(), internal.SHA256sum(r.UserAgent()))
if result >= 25 {
return true, nil
}
return false, nil
})
}
func (i *Impl) CheckNetwork() checker.Impl {
return checker.Func(func(r *http.Request) (bool, error) {
result, _ := i.uaWeight.Get(r.Context(), internal.SHA256sum(r.UserAgent()))
realIP, _ := internal.RealIP(r)
if !realIP.IsValid() {
realIP = netip.MustParseAddr(r.Header.Get("X-Real-Ip"))
}
network, ok := internal.ClampIP(realIP)
if !ok {
return false, nil
}
result, _ := i.networkWeight.Get(r.Context(), internal.SHA256sum(network.String()))
if result >= 25 {
return true, nil
}
@@ -164,7 +157,6 @@ func (i *Impl) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
networkCount := i.incrementNetwork(r.Context(), network.String())
uaCount := i.incrementUA(r.Context(), r.UserAgent())
stage := r.PathValue("stage")
@@ -172,11 +164,14 @@ func (i *Impl) ServeHTTP(w http.ResponseWriter, r *http.Request) {
lg.Debug("found new entrance point", "id", id, "stage", stage, "userAgent", r.UserAgent(), "clampedIP", network)
} else {
switch {
case networkCount%256 == 0, uaCount%256 == 0:
lg.Warn("found possible crawler", "id", id, "network", network)
case networkCount%256 == 0:
lg.Warn("found possible crawler", "id", id, "network", network, "userAgent", r.UserAgent())
}
}
millisecondAmount := min(math.Pow(float64(networkCount), 2), 1000)
time.Sleep(time.Duration(millisecondAmount) * time.Millisecond)
spins := i.makeSpins()
affirmations := i.makeAffirmations()
title := i.makeTitle()
+92
View File
@@ -0,0 +1,92 @@
package internal
import (
"errors"
"fmt"
"net"
"net/url"
"os"
"strconv"
"strings"
)
// parseBindNetFromAddr determine bind network and address based on the given network and address.
func parseBindNetFromAddr(address string) (string, string, error) {
defaultScheme := "http://"
if !strings.Contains(address, "://") {
if strings.HasPrefix(address, ":") {
address = defaultScheme + "localhost" + address
} else {
address = defaultScheme + address
}
}
bindUri, err := url.Parse(address)
if err != nil {
return "", "", fmt.Errorf("failed to parse bind URL: %w", err)
}
switch bindUri.Scheme {
case "unix":
return "unix", bindUri.Path, nil
case "tcp", "http", "https":
return "tcp", bindUri.Host, nil
default:
return "", "", fmt.Errorf("unsupported network scheme %s in address %s", bindUri.Scheme, address)
}
}
// SetupListener sets up a network listener based on the input from configuration
// envvars. It returns a network listener and the URL to that listener or an error.
func SetupListener(network, address, socketMode string) (net.Listener, string, error) {
formattedAddress := ""
var err error
if network == "" {
// keep compatibility
network, address, err = parseBindNetFromAddr(address)
}
if err != nil {
return nil, "", fmt.Errorf("can't parse bind and network: %w", err)
}
switch network {
case "unix":
formattedAddress = "unix:" + address
case "tcp":
if strings.HasPrefix(address, ":") { // assume it's just a port e.g. :4259
formattedAddress = "http://localhost" + address
} else {
formattedAddress = "http://" + address
}
default:
formattedAddress = fmt.Sprintf(`(%s) %s`, network, address)
}
ln, err := net.Listen(network, address)
if err != nil {
return nil, "", fmt.Errorf("failed to bind to %s: %w", formattedAddress, err)
}
// additional permission handling for unix sockets
if network == "unix" {
mode, err := strconv.ParseUint(socketMode, 8, 0)
if err != nil {
ln.Close()
return nil, "", fmt.Errorf("could not parse socket mode %s: %w", socketMode, err)
}
err = os.Chmod(address, os.FileMode(mode))
if err != nil {
err := fmt.Errorf("could not change socket mode: %w", err)
clErr := ln.Close()
if clErr != nil {
return nil, "", errors.Join(err, clErr)
}
return nil, "", err
}
}
return ln, formattedAddress, nil
}
+180
View File
@@ -0,0 +1,180 @@
package internal
import (
"io/fs"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
)
func TestParseBindNetFromAddr(t *testing.T) {
for _, tt := range []struct {
name string
address string
wantErr bool
network string
bind string
}{
{
name: "simple tcp",
address: "localhost:9090",
wantErr: false,
network: "tcp",
bind: "localhost:9090",
},
{
name: "simple unix",
address: "unix:///tmp/foo.sock",
wantErr: false,
network: "unix",
bind: "/tmp/foo.sock",
},
{
name: "invalid network",
address: "foo:///tmp/bar.sock",
wantErr: true,
},
{
name: "tcp uri",
address: "tcp://[::]:9090",
wantErr: false,
network: "tcp",
bind: "[::]:9090",
},
{
name: "http uri",
address: "http://[::]:9090",
wantErr: false,
network: "tcp",
bind: "[::]:9090",
},
{
name: "https uri",
address: "https://[::]:9090",
wantErr: false,
network: "tcp",
bind: "[::]:9090",
},
} {
t.Run(tt.name, func(t *testing.T) {
network, bind, err := parseBindNetFromAddr(tt.address)
switch {
case tt.wantErr && err == nil:
t.Errorf("parseBindNetFromAddr(%q) should have errored but did not", tt.address)
case !tt.wantErr && err != nil:
t.Errorf("parseBindNetFromAddr(%q) threw an error: %v", tt.address, err)
}
if network != tt.network {
t.Errorf("parseBindNetFromAddr(%q) wanted network: %q, got: %q", tt.address, tt.network, network)
}
if bind != tt.bind {
t.Errorf("parseBindNetFromAddr(%q) wanted bind: %q, got: %q", tt.address, tt.bind, bind)
}
})
}
}
func TestSetupListener(t *testing.T) {
td := t.TempDir()
for _, tt := range []struct {
name string
network, address, socketMode string
wantErr bool
socketURLPrefix string
}{
{
name: "simple tcp",
network: "",
address: ":0",
wantErr: false,
socketURLPrefix: "http://localhost:",
},
{
name: "simple unix",
network: "",
address: "unix://" + filepath.Join(td, "a"),
socketMode: "0770",
wantErr: false,
socketURLPrefix: "unix:" + filepath.Join(td, "a"),
},
{
name: "tcp",
network: "tcp",
address: ":0",
wantErr: false,
socketURLPrefix: "http://localhost:",
},
{
name: "udp",
network: "udp",
address: ":0",
wantErr: true,
socketURLPrefix: "http://localhost:",
},
{
name: "unix socket",
network: "unix",
socketMode: "0770",
address: filepath.Join(td, "a"),
wantErr: false,
socketURLPrefix: "unix:" + filepath.Join(td, "a"),
},
{
name: "invalid socket mode",
network: "unix",
socketMode: "taco bell",
address: filepath.Join(td, "a"),
wantErr: true,
socketURLPrefix: "unix:" + filepath.Join(td, "a"),
},
{
name: "empty socket mode",
network: "unix",
socketMode: "",
address: filepath.Join(td, "a"),
wantErr: true,
socketURLPrefix: "unix:" + filepath.Join(td, "a"),
},
} {
t.Run(tt.name, func(t *testing.T) {
ln, socketURL, err := SetupListener(tt.network, tt.address, tt.socketMode)
switch {
case tt.wantErr && err == nil:
t.Errorf("SetupListener(%q, %q, %q) should have errored but did not", tt.network, tt.address, tt.socketMode)
case !tt.wantErr && err != nil:
t.Fatalf("SetupListener(%q, %q, %q) threw an error: %v", tt.network, tt.address, tt.socketMode, err)
}
if ln != nil {
defer ln.Close()
}
if !tt.wantErr && !strings.HasPrefix(socketURL, tt.socketURLPrefix) {
t.Errorf("SetupListener(%q, %q, %q) should have returned a URL with prefix %q but got: %q", tt.network, tt.address, tt.socketMode, tt.socketURLPrefix, socketURL)
}
if tt.socketMode != "" {
mode, err := strconv.ParseUint(tt.socketMode, 8, 0)
if err != nil {
return
}
sockPath := strings.TrimPrefix(socketURL, "unix:")
st, err := os.Stat(sockPath)
if err != nil {
t.Fatalf("can't os.Stat(%q): %v", sockPath, err)
}
if st.Mode().Perm() != fs.FileMode(mode) {
t.Errorf("file mode of %q should be %s but is actually %s", sockPath, strconv.FormatUint(mode, 8), strconv.FormatUint(uint64(st.Mode()), 8))
}
}
})
}
}
+1 -1
View File
@@ -595,7 +595,7 @@ func spawnAnubisWithOptions(t *testing.T, basePrefix string) string {
fmt.Fprintf(w, "<html><body><span id=anubis-test>%d</span></body></html>", time.Now().Unix())
})
policy, err := libanubis.LoadPoliciesOrDefault(t.Context(), "", anubis.DefaultDifficulty, "info")
policy, err := libanubis.LoadPoliciesOrDefault(t.Context(), "", anubis.DefaultDifficulty, "info", false)
if err != nil {
t.Fatal(err)
}
+93 -17
View File
@@ -11,6 +11,7 @@ import (
"net"
"net/http"
"net/url"
"strconv"
"strings"
"time"
@@ -32,6 +33,7 @@ import (
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/lib/policy/checker"
"github.com/TecharoHQ/anubis/lib/store"
iptoasnv1 "github.com/TecharoHQ/thoth-proto/gen/techaro/thoth/iptoasn/v1"
// challenge implementations
_ "github.com/TecharoHQ/anubis/lib/challenge/metarefresh"
@@ -39,31 +41,52 @@ import (
_ "github.com/TecharoHQ/anubis/lib/challenge/proofofwork"
)
type contextKey int
const asnContextKey contextKey = iota
type asnInfo struct {
ASN string
Description string
}
func asnFromContext(ctx context.Context) (string, string) {
if v, ok := ctx.Value(asnContextKey).(asnInfo); ok {
return v.ASN, v.Description
}
return "", ""
}
var (
challengesIssued = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_challenges_issued",
Help: "The total number of challenges issued",
}, []string{"method"})
}, []string{"method", "asn", "asn_description"})
challengesValidated = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_challenges_validated",
Help: "The total number of challenges validated",
}, []string{"method"})
}, []string{"method", "asn", "asn_description"})
droneBLHits = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_dronebl_hits",
Help: "The total number of hits from DroneBL",
}, []string{"status"})
}, []string{"status", "asn", "asn_description"})
failedValidations = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_failed_validations",
Help: "The total number of failed validations",
}, []string{"method"})
}, []string{"method", "asn", "asn_description"})
requestsProxied = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_proxied_requests_total",
Help: "Number of requests proxied through Anubis to upstream targets",
}, []string{"host"})
}, []string{"host", "asn", "asn_description"})
requestsByASN = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_requests_by_asn_total",
Help: "Number of requests by ASN",
}, []string{"asn", "asn_description"})
)
type Server struct {
@@ -78,6 +101,28 @@ type Server struct {
hs512Secret []byte
}
func (s *Server) getRequestLogger(r *http.Request) (*slog.Logger, *http.Request) {
lg := internal.GetRequestLogger(s.logger, r)
if s.policy.LogASN && s.policy.ThothClient != nil {
ctx, cancel := context.WithTimeout(r.Context(), 500*time.Millisecond)
defer cancel()
ip := r.Header.Get("X-Real-Ip")
if info, err := s.policy.ThothClient.IPToASN.Lookup(ctx, &iptoasnv1.LookupRequest{IpAddress: ip}); err == nil && info.GetAnnounced() {
asn := strconv.FormatUint(uint64(info.GetAsNumber()), 10)
lg = lg.With("asn", info.GetAsNumber(), "asn_description", info.GetDescription())
requestsByASN.WithLabelValues(asn, info.GetDescription()).Inc()
r = r.WithContext(context.WithValue(r.Context(), asnContextKey, asnInfo{
ASN: asn,
Description: info.GetDescription(),
}))
}
}
return lg, r
}
func (s *Server) getTokenKeyfunc() jwt.Keyfunc {
// return ED25519 key if HS512 is not set
if len(s.hs512Secret) == 0 {
@@ -106,6 +151,13 @@ func (s *Server) issueChallenge(ctx context.Context, r *http.Request, lg *slog.L
//return nil, errors.New("[unexpected] this codepath should be impossible, asked to issue a challenge for a non-challenge rule")
}
if rule.Challenge == nil {
rule.Challenge = &config.ChallengeRules{
Difficulty: s.policy.DefaultDifficulty,
Algorithm: config.DefaultAlgorithm,
}
}
id, err := uuid.NewV7()
if err != nil {
return nil, err
@@ -134,7 +186,7 @@ func (s *Server) issueChallenge(ctx context.Context, r *http.Request, lg *slog.L
return nil, err
}
lg.Info("new challenge issued", "challenge", id.String())
lg.Info("new challenge issued", "challenge", id.String(), "weight", cr.Weight)
return &chall, err
}
@@ -186,7 +238,7 @@ func (s *Server) maybeReverseProxyOrPage(w http.ResponseWriter, r *http.Request)
}
func (s *Server) maybeReverseProxy(w http.ResponseWriter, r *http.Request, httpStatusOnly bool) {
lg := internal.GetRequestLogger(s.logger, r)
lg, r := s.getRequestLogger(r)
if val, _ := s.store.Get(r.Context(), fmt.Sprintf("ogtags:allow:%s%s", r.Host, r.URL.String())); val != nil {
lg.Debug("serving opengraph tag asset")
@@ -211,7 +263,10 @@ func (s *Server) maybeReverseProxy(w http.ResponseWriter, r *http.Request, httpS
r.Header.Add("X-Anubis-Rule", cr.Name)
r.Header.Add("X-Anubis-Action", string(cr.Rule))
lg = lg.With("check_result", cr)
policy.Applications.WithLabelValues(cr.Name, string(cr.Rule)).Add(1)
{
asn, asnDesc := asnFromContext(r.Context())
policy.Applications.WithLabelValues(cr.Name, string(cr.Rule), asn, asnDesc).Add(1)
}
ip := r.Header.Get("X-Real-Ip")
@@ -341,7 +396,8 @@ func (s *Server) handleDNSBL(w http.ResponseWriter, r *http.Request, ip string,
lg.Error("can't look up ip in dnsbl", "err", err)
}
db.Set(r.Context(), ip, resp, 24*time.Hour)
droneBLHits.WithLabelValues(resp.String()).Inc()
asn, asnDesc := asnFromContext(r.Context())
droneBLHits.WithLabelValues(resp.String(), asn, asnDesc).Inc()
}
if resp != dnsbl.AllGood {
@@ -359,7 +415,7 @@ func (s *Server) handleDNSBL(w http.ResponseWriter, r *http.Request, ip string,
}
func (s *Server) MakeChallenge(w http.ResponseWriter, r *http.Request) {
lg := internal.GetRequestLogger(s.logger, r)
lg, r := s.getRequestLogger(r)
localizer := localization.GetLocalizer(r)
redir := r.FormValue("redir")
@@ -428,11 +484,14 @@ func (s *Server) MakeChallenge(w http.ResponseWriter, r *http.Request) {
return
}
lg.Debug("made challenge", "challenge", chall, "rules", rule.Challenge, "cr", cr)
challengesIssued.WithLabelValues("api").Inc()
{
asn, asnDesc := asnFromContext(r.Context())
challengesIssued.WithLabelValues("api", asn, asnDesc).Inc()
}
}
func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
lg := internal.GetRequestLogger(s.logger, r)
lg, r := s.getRequestLogger(r)
localizer := localization.GetLocalizer(r)
redir := r.FormValue("redir")
@@ -491,7 +550,11 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
chall, err := s.getChallenge(r)
if err != nil {
lg.Error("getChallenge failed", "err", err)
s.respondWithError(w, r, fmt.Sprintf("%s: %s", localizer.T("internal_server_error"), rule.Challenge.Algorithm), makeCode(err))
algorithm := "unknown"
if rule.Challenge != nil {
algorithm = rule.Challenge.Algorithm
}
s.respondWithError(w, r, fmt.Sprintf("%s: %s", localizer.T("internal_server_error"), algorithm), makeCode(err))
return
}
@@ -519,7 +582,8 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
}
if err := impl.Validate(r, lg, in); err != nil {
failedValidations.WithLabelValues(rule.Challenge.Algorithm).Inc()
asn, asnDesc := asnFromContext(r.Context())
failedValidations.WithLabelValues(rule.Challenge.Algorithm, asn, asnDesc).Inc()
var cerr *challenge.Error
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
lg.Debug("challenge validate call failed", "err", err)
@@ -579,7 +643,10 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
lg.Debug("can't update information about challenge", "err", err)
}
challengesValidated.WithLabelValues(rule.Challenge.Algorithm).Inc()
{
asn, asnDesc := asnFromContext(r.Context())
challengesValidated.WithLabelValues(rule.Challenge.Algorithm, asn, asnDesc).Inc()
}
lg.Debug("challenge passed, redirecting to app")
http.Redirect(w, r, redir, http.StatusFound)
}
@@ -618,7 +685,8 @@ func (s *Server) check(r *http.Request, lg *slog.Logger) (policy.CheckResult, *p
return cr("bot/"+b.Name, b.Action, weight), &b, nil
case config.RuleWeigh:
lg.Debug("adjusting weight", "name", b.Name, "delta", b.Weight.Adjust)
policy.Applications.WithLabelValues("bot/"+b.Name, "WEIGH").Add(1)
asn, asnDesc := asnFromContext(r.Context())
policy.Applications.WithLabelValues("bot/"+b.Name, "WEIGH", asn, asnDesc).Add(1)
weight += b.Weight.Adjust
}
}
@@ -638,8 +706,16 @@ func (s *Server) check(r *http.Request, lg *slog.Logger) (policy.CheckResult, *p
}
if matches {
challRules := t.Challenge
if challRules == nil {
// Non-CHALLENGE thresholds (ALLOW/DENY) don't have challenge config.
// Use an empty struct so hydrateChallengeRule can fill from stored
// challenge data during validation, rather than baking in defaults
// that could mismatch the difficulty the client actually solved for.
challRules = &config.ChallengeRules{}
}
return cr("threshold/"+t.Name, t.Action, weight), &policy.Bot{
Challenge: t.Challenge,
Challenge: challRules,
Rules: &checker.List{},
}, nil
}
+2 -2
View File
@@ -58,7 +58,7 @@ func loadPolicies(t *testing.T, fname string, difficulty int) *policy.ParsedConf
t.Logf("loading policy file: %s", fname)
anubisPolicy, err := LoadPoliciesOrDefault(ctx, fname, difficulty, "info")
anubisPolicy, err := LoadPoliciesOrDefault(ctx, fname, difficulty, "info", false)
if err != nil {
t.Fatal(err)
}
@@ -250,7 +250,7 @@ func TestLoadPolicies(t *testing.T) {
}
defer fin.Close()
if _, err := policy.ParseConfig(t.Context(), fin, fname, 4, "info"); err != nil {
if _, err := policy.ParseConfig(t.Context(), fin, fname, 4, "info", false); err != nil {
t.Fatal(err)
}
})
+1
View File
@@ -10,6 +10,7 @@ var (
ErrFailed = errors.New("challenge: user failed challenge")
ErrMissingField = errors.New("challenge: missing field")
ErrInvalidFormat = errors.New("challenge: field has invalid format")
ErrInvalidInput = errors.New("challenge: input is nil or missing required fields")
)
func NewError(verb, publicReason string, privateReason error) *Error {
+33
View File
@@ -1,6 +1,7 @@
package challenge
import (
"fmt"
"log/slog"
"net/http"
"sort"
@@ -50,12 +51,44 @@ type IssueInput struct {
Store store.Interface
}
func (in *IssueInput) Valid() error {
if in == nil {
return fmt.Errorf("%w: IssueInput is nil", ErrInvalidInput)
}
if in.Rule == nil {
return fmt.Errorf("%w: Rule is nil", ErrInvalidInput)
}
if in.Rule.Challenge == nil {
return fmt.Errorf("%w: Rule.Challenge is nil", ErrInvalidInput)
}
if in.Challenge == nil {
return fmt.Errorf("%w: Challenge is nil", ErrInvalidInput)
}
return nil
}
type ValidateInput struct {
Rule *policy.Bot
Challenge *Challenge
Store store.Interface
}
func (in *ValidateInput) Valid() error {
if in == nil {
return fmt.Errorf("%w: ValidateInput is nil", ErrInvalidInput)
}
if in.Rule == nil {
return fmt.Errorf("%w: Rule is nil", ErrInvalidInput)
}
if in.Rule.Challenge == nil {
return fmt.Errorf("%w: Rule.Challenge is nil", ErrInvalidInput)
}
if in.Challenge == nil {
return fmt.Errorf("%w: Challenge is nil", ErrInvalidInput)
}
return nil
}
type Impl interface {
// Setup registers any additional routes with the Impl for assets or API routes.
Setup(mux *http.ServeMux)
+8
View File
@@ -24,6 +24,10 @@ type Impl struct{}
func (i *Impl) Setup(mux *http.ServeMux) {}
func (i *Impl) Issue(w http.ResponseWriter, r *http.Request, lg *slog.Logger, in *challenge.IssueInput) (templ.Component, error) {
if err := in.Valid(); err != nil {
return nil, err
}
u, err := r.URL.Parse(anubis.BasePrefix + "/.within.website/x/cmd/anubis/api/pass-challenge")
if err != nil {
return nil, fmt.Errorf("can't render page: %w", err)
@@ -49,6 +53,10 @@ func (i *Impl) Issue(w http.ResponseWriter, r *http.Request, lg *slog.Logger, in
}
func (i *Impl) Validate(r *http.Request, lg *slog.Logger, in *challenge.ValidateInput) error {
if err := in.Valid(); err != nil {
return challenge.NewError("validate", "invalid input", err)
}
wantTime := in.Challenge.IssuedAt.Add(time.Duration(in.Rule.Challenge.Difficulty) * 800 * time.Millisecond)
if time.Now().Before(wantTime) {
+8
View File
@@ -39,6 +39,10 @@ type impl struct{}
func (i *impl) Setup(mux *http.ServeMux) {}
func (i *impl) Issue(w http.ResponseWriter, r *http.Request, lg *slog.Logger, in *challenge.IssueInput) (templ.Component, error) {
if err := in.Valid(); err != nil {
return nil, err
}
u, err := r.URL.Parse(anubis.BasePrefix + "/.within.website/x/cmd/anubis/api/pass-challenge")
if err != nil {
return nil, fmt.Errorf("can't render page: %w", err)
@@ -57,6 +61,10 @@ func (i *impl) Issue(w http.ResponseWriter, r *http.Request, lg *slog.Logger, in
}
func (i *impl) Validate(r *http.Request, lg *slog.Logger, in *challenge.ValidateInput) error {
if err := in.Valid(); err != nil {
return challenge.NewError("validate", "invalid input", err)
}
wantTime := in.Challenge.IssuedAt.Add(time.Duration(in.Rule.Challenge.Difficulty) * 80 * time.Millisecond)
if time.Now().Before(wantTime) {
+4
View File
@@ -33,6 +33,10 @@ func (i *Impl) Issue(w http.ResponseWriter, r *http.Request, lg *slog.Logger, in
}
func (i *Impl) Validate(r *http.Request, lg *slog.Logger, in *chall.ValidateInput) error {
if err := in.Valid(); err != nil {
return chall.NewError("validate", "invalid input", err)
}
rule := in.Rule
challenge := in.Challenge.RandomData
@@ -30,6 +30,62 @@ func mkRequest(t *testing.T, values map[string]string) *http.Request {
return req
}
// TestValidateNilRuleChallenge reproduces the panic from
// https://github.com/TecharoHQ/anubis/issues/1463
//
// When a threshold rule matches during PassChallenge, check() can return
// a policy.Bot with Challenge == nil. After hydrateChallengeRule fails to
// run (or the error path hits before it), Validate dereferences
// rule.Challenge.Difficulty and panics.
func TestValidateNilRuleChallenge(t *testing.T) {
i := &Impl{Algorithm: "fast"}
lg := slog.With()
// This is the exact response for SHA256("hunter" + "0") with 0 leading zeros required.
const challengeStr = "hunter"
const response = "2652bdba8fb4d2ab39ef28d8534d7694c557a4ae146c1e9237bd8d950280500e"
req := mkRequest(t, map[string]string{
"nonce": "0",
"elapsedTime": "69",
"response": response,
})
for _, tc := range []struct {
name string
input *challenge.ValidateInput
}{
{
name: "nil-rule-challenge",
input: &challenge.ValidateInput{
Rule: &policy.Bot{},
Challenge: &challenge.Challenge{RandomData: challengeStr},
},
},
{
name: "nil-rule",
input: &challenge.ValidateInput{
Challenge: &challenge.Challenge{RandomData: challengeStr},
},
},
{
name: "nil-challenge",
input: &challenge.ValidateInput{Rule: &policy.Bot{Challenge: &config.ChallengeRules{Algorithm: "fast"}}},
},
{
name: "nil-input",
input: nil,
},
} {
t.Run(tc.name, func(t *testing.T) {
err := i.Validate(req, lg, tc.input)
if !errors.Is(err, challenge.ErrInvalidInput) {
t.Fatalf("expected ErrInvalidInput, got: %v", err)
}
})
}
}
func TestBasic(t *testing.T) {
i := &Impl{Algorithm: "fast"}
bot := &policy.Bot{
+2 -10
View File
@@ -55,7 +55,7 @@ type Options struct {
DifficultyInJWT bool
}
func LoadPoliciesOrDefault(ctx context.Context, fname string, defaultDifficulty int, logLevel string) (*policy.ParsedConfig, error) {
func LoadPoliciesOrDefault(ctx context.Context, fname string, defaultDifficulty int, logLevel string, subrequestMode bool) (*policy.ParsedConfig, error) {
var fin io.ReadCloser
var err error
@@ -79,7 +79,7 @@ func LoadPoliciesOrDefault(ctx context.Context, fname string, defaultDifficulty
}
}(fin)
anubisPolicy, err := policy.ParseConfig(ctx, fin, fname, defaultDifficulty, logLevel)
anubisPolicy, err := policy.ParseConfig(ctx, fin, fname, defaultDifficulty, logLevel, subrequestMode)
if err != nil {
return nil, fmt.Errorf("can't parse policy file %s: %w", fname, err)
}
@@ -190,14 +190,6 @@ func New(opts Options) (*Server, error) {
},
Name: "honeypot/network",
},
policy.Bot{
Rules: mazeGen.CheckUA(),
Action: config.RuleWeigh,
Weight: &config.Weight{
Adjust: 30,
},
Name: "honeypot/user-agent",
},
)
} else {
result.logger.Error("can't init honeypot subsystem", "err", err)
+10 -1
View File
@@ -19,7 +19,7 @@ import (
var (
ErrNoBotRulesDefined = errors.New("config: must define at least one (1) bot rule")
ErrBotMustHaveName = errors.New("config.Bot: must set name")
ErrBotMustHaveUserAgentOrPath = errors.New("config.Bot: must set either user_agent_regex, path_regex, headers_regex, or remote_addresses")
ErrBotMustHaveUserAgentOrPath = errors.New("config.Bot: must set one of user_agent_regex, path_regex, headers_regex, remote_addresses, expression, or Thoth keyword")
ErrBotMustHaveUserAgentOrPathNotBoth = errors.New("config.Bot: must set either user_agent_regex, path_regex, and not both")
ErrUnknownAction = errors.New("config.Bot: unknown action")
ErrInvalidUserAgentRegex = errors.New("config.Bot: invalid user agent regex")
@@ -334,6 +334,7 @@ type fileConfig struct {
DNSBL bool `json:"dnsbl"`
DNSTTL DnsTTL `json:"dns_ttl"`
Logging *Logging `json:"logging"`
Metrics *Metrics `json:"metrics,omitempty"`
}
func (c *fileConfig) Valid() error {
@@ -375,6 +376,12 @@ func (c *fileConfig) Valid() error {
}
}
if c.Metrics != nil {
if err := c.Metrics.Valid(); err != nil {
errs = append(errs, err)
}
}
if len(errs) != 0 {
return fmt.Errorf("config is not valid:\n%w", errors.Join(errs...))
}
@@ -417,6 +424,7 @@ func Load(fin io.Reader, fname string) (*Config, error) {
StatusCodes: c.StatusCodes,
Store: c.Store,
Logging: c.Logging,
Metrics: c.Metrics,
}
if c.OpenGraph.TimeToLive != "" {
@@ -508,6 +516,7 @@ type Config struct {
Logging *Logging
DNSBL bool
DNSTTL DnsTTL
Metrics *Metrics
}
func (c Config) Valid() error {
+1
View File
@@ -17,6 +17,7 @@ type Logging struct {
Sink string `json:"sink"` // Logging sink, either "stdio" or "file"
Level *slog.Level `json:"level"` // Log level, if set supersedes the level in flags
Parameters *LoggingFileConfig `json:"parameters"` // Logging parameters, to be dynamic in the future
LogASN bool `json:"asn" yaml:"asn"`
}
const (
+167
View File
@@ -0,0 +1,167 @@
package config
import (
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"os"
"strconv"
)
var (
ErrInvalidMetricsConfig = errors.New("config: invalid metrics configuration")
ErrInvalidMetricsTLSConfig = errors.New("config: invalid metrics TLS configuration")
ErrInvalidMetricsBasicAuthConfig = errors.New("config: invalid metrics basic auth configuration")
ErrNoMetricsBind = errors.New("config.Metrics: must define bind")
ErrNoMetricsNetwork = errors.New("config.Metrics: must define network")
ErrNoMetricsSocketMode = errors.New("config.Metrics: must define socket mode when using unix sockets")
ErrInvalidMetricsSocketMode = errors.New("config.Metrics: invalid unix socket mode")
ErrInvalidMetricsNetwork = errors.New("config.Metrics: invalid metrics network")
ErrNoMetricsTLSCertificate = errors.New("config.Metrics.TLS: must define certificate file")
ErrNoMetricsTLSKey = errors.New("config.Metrics.TLS: must define key file")
ErrInvalidMetricsTLSKeypair = errors.New("config.Metrics.TLS: keypair is invalid")
ErrInvalidMetricsCACertificate = errors.New("config.Metrics.TLS: invalid CA certificate")
ErrCantReadFile = errors.New("config: can't read required file")
ErrNoMetricsBasicAuthUsername = errors.New("config.Metrics.BasicAuth: must define username")
ErrNoMetricsBasicAuthPassword = errors.New("config.Metrics.BasicAuth: must define password")
)
type Metrics struct {
Bind string `json:"bind" yaml:"bind"`
Network string `json:"network" yaml:"network"`
SocketMode string `json:"socketMode" yaml:"socketMode"`
TLS *MetricsTLS `json:"tls" yaml:"tls"`
Debug bool `json:"debug" yaml:"debug"`
BasicAuth *MetricsBasicAuth `json:"basicAuth" yaml:"basicAuth"`
}
func (m *Metrics) Valid() error {
var errs []error
if m.Bind == "" {
errs = append(errs, ErrNoMetricsBind)
}
if m.Network == "" {
errs = append(errs, ErrNoMetricsNetwork)
}
switch m.Network {
case "tcp", "tcp4", "tcp6": // https://pkg.go.dev/net#Listen
case "unix":
if m.SocketMode == "" {
errs = append(errs, ErrNoMetricsSocketMode)
}
if _, err := strconv.ParseUint(m.SocketMode, 8, 0); err != nil {
errs = append(errs, fmt.Errorf("%w: %w", ErrInvalidMetricsSocketMode, err))
}
default:
errs = append(errs, ErrInvalidMetricsNetwork)
}
if m.TLS != nil {
if err := m.TLS.Valid(); err != nil {
errs = append(errs, err)
}
}
if m.BasicAuth != nil {
if err := m.BasicAuth.Valid(); err != nil {
errs = append(errs, err)
}
}
if len(errs) != 0 {
return errors.Join(ErrInvalidMetricsConfig, errors.Join(errs...))
}
return nil
}
type MetricsTLS struct {
Certificate string `json:"certificate" yaml:"certificate"`
Key string `json:"key" yaml:"key"`
CA string `json:"ca" yaml:"ca"`
}
func (mt *MetricsTLS) Valid() error {
var errs []error
if mt.Certificate == "" {
errs = append(errs, ErrNoMetricsTLSCertificate)
}
if err := canReadFile(mt.Certificate); err != nil {
errs = append(errs, fmt.Errorf("%w %s: %w", ErrCantReadFile, mt.Certificate, err))
}
if mt.Key == "" {
errs = append(errs, ErrNoMetricsTLSKey)
}
if err := canReadFile(mt.Key); err != nil {
errs = append(errs, fmt.Errorf("%w %s: %w", ErrCantReadFile, mt.Key, err))
}
if _, err := tls.LoadX509KeyPair(mt.Certificate, mt.Key); err != nil {
errs = append(errs, fmt.Errorf("%w: %w", ErrInvalidMetricsTLSKeypair, err))
}
if mt.CA != "" {
caCert, err := os.ReadFile(mt.CA)
if err != nil {
errs = append(errs, fmt.Errorf("%w %s: %w", ErrCantReadFile, mt.CA, err))
}
certPool := x509.NewCertPool()
if !certPool.AppendCertsFromPEM(caCert) {
errs = append(errs, fmt.Errorf("%w %s", ErrInvalidMetricsCACertificate, mt.CA))
}
}
if len(errs) != 0 {
return errors.Join(ErrInvalidMetricsTLSConfig, errors.Join(errs...))
}
return nil
}
func canReadFile(fname string) error {
fin, err := os.Open(fname)
if err != nil {
return err
}
defer fin.Close()
data := make([]byte, 64)
if _, err := fin.Read(data); err != nil {
return fmt.Errorf("can't read %s: %w", fname, err)
}
return nil
}
type MetricsBasicAuth struct {
Username string `json:"username" yaml:"username"`
Password string `json:"password" yaml:"password"`
}
func (mba *MetricsBasicAuth) Valid() error {
var errs []error
if mba.Username == "" {
errs = append(errs, ErrNoMetricsBasicAuthUsername)
}
if mba.Password == "" {
errs = append(errs, ErrNoMetricsBasicAuthPassword)
}
if len(errs) != 0 {
return errors.Join(ErrInvalidMetricsBasicAuthConfig, errors.Join(errs...))
}
return nil
}
+242
View File
@@ -0,0 +1,242 @@
package config
import (
"errors"
"testing"
)
func TestMetricsValid(t *testing.T) {
for _, tt := range []struct {
name string
input *Metrics
err error
}{
{
name: "basic TCP",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
},
},
{
name: "basic TCP4",
input: &Metrics{
Bind: ":9090",
Network: "tcp4",
},
},
{
name: "basic TCP6",
input: &Metrics{
Bind: ":9090",
Network: "tcp6",
},
},
{
name: "basic unix",
input: &Metrics{
Bind: "/tmp/anubis-metrics.sock",
Network: "unix",
SocketMode: "0770",
},
},
{
name: "no bind",
input: &Metrics{},
err: ErrNoMetricsBind,
},
{
name: "no network",
input: &Metrics{},
err: ErrNoMetricsNetwork,
},
{
name: "no unix socket mode",
input: &Metrics{
Bind: "/tmp/anubis-metrics.sock",
Network: "unix",
},
err: ErrNoMetricsSocketMode,
},
{
name: "invalid unix socket mode",
input: &Metrics{
Bind: "/tmp/anubis-metrics.sock",
Network: "unix",
SocketMode: "taco bell",
},
err: ErrInvalidMetricsSocketMode,
},
{
name: "invalid network",
input: &Metrics{
Bind: ":9090",
Network: "taco",
},
err: ErrInvalidMetricsNetwork,
},
{
name: "invalid TLS config",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
TLS: &MetricsTLS{},
},
err: ErrInvalidMetricsTLSConfig,
},
{
name: "selfsigned TLS cert",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
TLS: &MetricsTLS{
Certificate: "./testdata/tls/selfsigned.crt",
Key: "./testdata/tls/selfsigned.key",
},
},
},
{
name: "wrong path to selfsigned TLS cert",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
TLS: &MetricsTLS{
Certificate: "./testdata/tls2/selfsigned.crt",
Key: "./testdata/tls2/selfsigned.key",
},
},
err: ErrCantReadFile,
},
{
name: "unparseable TLS cert",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
TLS: &MetricsTLS{
Certificate: "./testdata/tls/invalid.crt",
Key: "./testdata/tls/invalid.key",
},
},
err: ErrInvalidMetricsTLSKeypair,
},
{
name: "mTLS with CA",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
TLS: &MetricsTLS{
Certificate: "./testdata/tls/selfsigned.crt",
Key: "./testdata/tls/selfsigned.key",
CA: "./testdata/tls/minica.pem",
},
},
},
{
name: "mTLS with nonexistent CA",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
TLS: &MetricsTLS{
Certificate: "./testdata/tls/selfsigned.crt",
Key: "./testdata/tls/selfsigned.key",
CA: "./testdata/tls/nonexistent.crt",
},
},
err: ErrCantReadFile,
},
{
name: "mTLS with invalid CA",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
TLS: &MetricsTLS{
Certificate: "./testdata/tls/selfsigned.crt",
Key: "./testdata/tls/selfsigned.key",
CA: "./testdata/tls/invalid.crt",
},
},
err: ErrInvalidMetricsCACertificate,
},
{
name: "basic auth credentials set",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
BasicAuth: &MetricsBasicAuth{
Username: "admin",
Password: "hunter2",
},
},
},
{
name: "invalid basic auth config",
input: &Metrics{
Bind: ":9090",
Network: "tcp",
BasicAuth: &MetricsBasicAuth{},
},
err: ErrInvalidMetricsBasicAuthConfig,
},
} {
t.Run(tt.name, func(t *testing.T) {
if err := tt.input.Valid(); !errors.Is(err, tt.err) {
t.Logf("wanted error: %v", tt.err)
t.Logf("got error: %v", err)
t.Error("validation failed")
}
})
}
}
func TestMetricsBasicAuthValid(t *testing.T) {
for _, tt := range []struct {
name string
input *MetricsBasicAuth
err error
}{
{
name: "both set",
input: &MetricsBasicAuth{
Username: "admin",
Password: "hunter2",
},
},
{
name: "empty username and password",
input: &MetricsBasicAuth{},
err: ErrInvalidMetricsBasicAuthConfig,
},
{
name: "missing username",
input: &MetricsBasicAuth{
Password: "hunter2",
},
err: ErrNoMetricsBasicAuthUsername,
},
{
name: "missing password",
input: &MetricsBasicAuth{
Username: "admin",
},
err: ErrNoMetricsBasicAuthPassword,
},
{
name: "missing both surfaces wrapper error",
input: &MetricsBasicAuth{},
err: ErrNoMetricsBasicAuthUsername,
},
{
name: "missing both surfaces password error",
input: &MetricsBasicAuth{},
err: ErrNoMetricsBasicAuthPassword,
},
} {
t.Run(tt.name, func(t *testing.T) {
if err := tt.input.Valid(); !errors.Is(err, tt.err) {
t.Logf("wanted error: %v", tt.err)
t.Logf("got error: %v", err)
t.Error("validation failed")
}
})
}
}
+3
View File
@@ -0,0 +1,3 @@
metrics:
bind: ":9090"
network: taco
+5 -1
View File
@@ -5,5 +5,9 @@
"remote_addresses": ["0.0.0.0/0", "::/0"],
"action": "ALLOW"
}
]
],
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -4,3 +4,7 @@ bots:
- "0.0.0.0/0"
- "::/0"
action: ALLOW
metrics:
bind: ":9090"
network: "tcp"
+5 -1
View File
@@ -8,5 +8,9 @@
"action": "DENY"
}
],
"dnsbl": false
"dnsbl": false,
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -3,3 +3,7 @@ bots:
headers_regex:
CF-Worker: .*
action: DENY
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -4,3 +4,7 @@ bots:
asns:
match:
- 13335 # Cloudflare
metrics:
bind: ":9090"
network: "tcp"
+5 -1
View File
@@ -5,5 +5,9 @@
"user_agent_regex": "Mozilla",
"action": "CHALLENGE"
}
]
],
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -2,3 +2,7 @@ bots:
- name: generic-browser
user_agent_regex: Mozilla
action: CHALLENGE
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -6,3 +6,7 @@ bots:
- name: "test"
user_agent_regex: ".*"
action: "DENY"
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -6,3 +6,7 @@ bots:
- '"Accept" in headers'
- headers["Accept"].contains("text/html")
- randInt(1) == 0
metrics:
bind: ":9090"
network: "tcp"
+5 -1
View File
@@ -6,5 +6,9 @@
"action": "DENY"
}
],
"dnsbl": false
"dnsbl": false,
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -2,3 +2,7 @@ bots:
- name: everything
user_agent_regex: .*
action: DENY
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -4,3 +4,7 @@ bots:
geoip:
countries:
- US
metrics:
bind: ":9090"
network: "tcp"
+5 -1
View File
@@ -10,5 +10,9 @@
]
}
}
]
],
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -6,3 +6,7 @@ bots:
- userAgent.startsWith("git/") || userAgent.contains("libgit")
- >
"Git-Protocol" in headers && headers["Git-Protocol"] == "version=2"
metrics:
bind: ":9090"
network: "tcp"
+5 -1
View File
@@ -3,5 +3,9 @@
{
"import": "./testdata/hack-test.json"
}
]
],
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -1,2 +1,6 @@
bots:
- import: ./testdata/hack-test.yaml
metrics:
bind: ":9090"
network: "tcp"
+5 -1
View File
@@ -3,5 +3,9 @@
{
"import": "(data)/common/keep-internet-working.yaml"
}
]
],
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
@@ -1,2 +1,6 @@
bots:
- import: (data)/common/keep-internet-working.yaml
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -8,3 +8,7 @@ impressum:
page:
title: Test
body: <p>This is a test</p>
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -13,3 +13,7 @@ logs:
oldFileTimeFormat: 2006-01-02T15-04-05 # RFC 3339-ish
compress: true
useLocalTime: false # timezone for rotated files is UTC
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -5,3 +5,7 @@ bots:
logging:
sink: "stdio"
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -6,3 +6,7 @@ bots:
adjust: 5
thresholds: []
metrics:
bind: ":9090"
network: "tcp"
+5 -1
View File
@@ -75,5 +75,9 @@
"user_agent_regex": "Mozilla",
"action": "CHALLENGE"
}
]
],
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -10,3 +10,7 @@ openGraph:
default:
"og:title": "Xe's magic land of fun"
"og:description": "We're no strangers to love, you know the rules and so do I"
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -4,3 +4,7 @@ bots:
user_agent_regex: Mozilla
weight:
adjust: 5
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -9,5 +9,9 @@
"status_codes": {
"CHALLENGE": 200,
"DENY": 200
},
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -6,3 +6,7 @@ bots:
status_codes:
CHALLENGE: 200
DENY: 200
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -9,5 +9,9 @@
"status_codes": {
"CHALLENGE": 403,
"DENY": 403
},
"metrics": {
"bind": ":9090",
"network": "tcp"
}
}
+4
View File
@@ -6,3 +6,7 @@ bots:
status_codes:
CHALLENGE: 403
DENY: 403
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -33,3 +33,7 @@ thresholds:
challenge:
algorithm: fast
difficulty: 4
metrics:
bind: ":9090"
network: "tcp"
+4
View File
@@ -2,3 +2,7 @@ bots:
- name: weight
action: WEIGH
user_agent_regex: Mozilla
metrics:
bind: ":9090"
network: "tcp"
+12
View File
@@ -0,0 +1,12 @@
-----BEGIN CERTIFICATE-----
MIIB1zCCAVygAwIBAgIIYO0SAFtXlVgwCgYIKoZIzj0EAwMwIDEeMBwGA1UEAxMV
bWluaWNhIHJvb3QgY2EgNDE2MmMwMB4XDTI2MDQyMjIzMjUwMVoXDTI4MDUyMjIz
MjUwMVowEjEQMA4GA1UEAxMHMS4xLjEuMTB2MBAGByqGSM49AgEGBSuBBAAiA2IA
BLsuA2LKGbEBuSA4LTm1KaKc7/QCkUOsipXR4+D5/3sWBZiAH7iWUgHwpx5YZf2q
kZn6oRda+ks0JLTQ6VhteQedmb7l86bMeDMR8p4Lg2b38l/xEr7S25UfUDKudXrO
AqNxMG8wDgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF
BQcDAjAMBgNVHRMBAf8EAjAAMB8GA1UdIwQYMBaAFE/7VDxF2+cUs9bu0pJM3xoC
L1TSMA8GA1UdEQQIMAaHBAEBAQEwCgYIKoZIzj0EAwMDaQAwZgIxAPLXds9MMH4K
F5FxTf9i0PKPsLQARsABVTgwB94hMR70rqW8Pwbjl7ZGNaYlaeRHUwIxAPMQ8zoF
nim+YS1xLqQek/LXuJto8jxcfkQQBsboVzcTa5uaNRhNd5YwrpomGl3lKA==
-----END CERTIFICATE-----
+6
View File
@@ -0,0 +1,6 @@
-----BEGIN PRIVATE KEY-----
MIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDBN8QsHxxHGJpStu8K7
D/FmaBBNo6c514KGFSIfqGFuREF5aOL3gN/W11yk2OIibdWhZANiAAS7LgNiyhmx
AbkgOC05tSminO/0ApFDrIqV0ePg+f97FgWYgB+4llIB8KceWGX9qpGZ+qEXWvpL
NCS00OlYbXkHnZm+5fOmzHgzEfKeC4Nm9/Jf8RK+0tuVH1AyrnV6zgI=
-----END PRIVATE KEY-----
View File
View File
+6
View File
@@ -0,0 +1,6 @@
-----BEGIN PRIVATE KEY-----
MIG2AgEAMBAGByqGSM49AgEGBSuBBAAiBIGeMIGbAgEBBDDr9QQo7ZaTgUL6d73G
2BG7+YRTFJHAZa0FogRglfc+jYttL1J4/xTig3RmHoqSgrehZANiAASDhijM9Xe0
G9Vam6AJMeKC6aWDNSLwrxNVmPxemsY/yJ1urBgnxRd9GFH6YW1ki/B8rS+Xl1UX
NnhBrukLaXvgAQQq782/5IUYGsvK5jw8+dSscYVMCQJwGfmQuaNeczQ=
-----END PRIVATE KEY-----
+13
View File
@@ -0,0 +1,13 @@
-----BEGIN CERTIFICATE-----
MIIB+zCCAYKgAwIBAgIIQWLAtv4ijQ0wCgYIKoZIzj0EAwMwIDEeMBwGA1UEAxMV
bWluaWNhIHJvb3QgY2EgNDE2MmMwMCAXDTI2MDQyMjIzMjUwMVoYDzIxMjYwNDIy
MjMyNTAxWjAgMR4wHAYDVQQDExVtaW5pY2Egcm9vdCBjYSA0MTYyYzAwdjAQBgcq
hkjOPQIBBgUrgQQAIgNiAASDhijM9Xe0G9Vam6AJMeKC6aWDNSLwrxNVmPxemsY/
yJ1urBgnxRd9GFH6YW1ki/B8rS+Xl1UXNnhBrukLaXvgAQQq782/5IUYGsvK5jw8
+dSscYVMCQJwGfmQuaNeczSjgYYwgYMwDgYDVR0PAQH/BAQDAgKEMB0GA1UdJQQW
MBQGCCsGAQUFBwMBBggrBgEFBQcDAjASBgNVHRMBAf8ECDAGAQH/AgEAMB0GA1Ud
DgQWBBRP+1Q8RdvnFLPW7tKSTN8aAi9U0jAfBgNVHSMEGDAWgBRP+1Q8RdvnFLPW
7tKSTN8aAi9U0jAKBggqhkjOPQQDAwNnADBkAjBfY7vb7cuLTjg7uoe+kl07FMYT
BGMSnWdhN3yXqMUS3A6XZxD/LntXT6V7yFOlAJYCMH7w8/ATYaTqbk2jBRyQt9/x
ajN+kZ6ZK+fKttqE8CD62mbHg09xoNxRq+K2I3PVyQ==
-----END CERTIFICATE-----
+11
View File
@@ -0,0 +1,11 @@
-----BEGIN CERTIFICATE-----
MIIBnzCCAVGgAwIBAgIUK39B3Ft+kU5o81IuISs79O4u1ncwBQYDK2VwMEUxCzAJ
BgNVBAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5l
dCBXaWRnaXRzIFB0eSBMdGQwHhcNMjYwNDIyMTQyNjE4WhcNMjYwNTIyMTQyNjE4
WjBFMQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwY
SW50ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMCowBQYDK2VwAyEAfgpAUpp8MIOOdQpH
fxaw3R7mFKQRMR6Kmxzk1Xn/2VujUzBRMB0GA1UdDgQWBBSmkBmzo0RiZ2iocMR8
uIIpz9cZyTAfBgNVHSMEGDAWgBSmkBmzo0RiZ2iocMR8uIIpz9cZyTAPBgNVHRMB
Af8EBTADAQH/MAUGAytlcANBAG37XXZrVUUzGyy3T9qsPIzvJQAGpGhdjJ7bt9O6
sBhzrliTONPrudYuyUggWsHgFb0JlN2xs4/2HhKU+PY7AAQ=
-----END CERTIFICATE-----
+3
View File
@@ -0,0 +1,3 @@
-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIL0HxjjfVlg6zQPB9/zTLq0IBzfp8gEoifEYzQZYIj+T
-----END PRIVATE KEY-----
+4 -4
View File
@@ -12,7 +12,7 @@ import (
)
func TestInvalidChallengeMethod(t *testing.T) {
if _, err := LoadPoliciesOrDefault(t.Context(), "testdata/invalid-challenge-method.yaml", 4, "info"); !errors.Is(err, policy.ErrChallengeRuleHasWrongAlgorithm) {
if _, err := LoadPoliciesOrDefault(t.Context(), "testdata/invalid-challenge-method.yaml", 4, "info", false); !errors.Is(err, policy.ErrChallengeRuleHasWrongAlgorithm) {
t.Fatalf("wanted error %v but got %v", policy.ErrChallengeRuleHasWrongAlgorithm, err)
}
}
@@ -25,7 +25,7 @@ func TestBadConfigs(t *testing.T) {
for _, st := range finfos {
t.Run(st.Name(), func(t *testing.T) {
if _, err := LoadPoliciesOrDefault(t.Context(), filepath.Join("config", "testdata", "bad", st.Name()), anubis.DefaultDifficulty, "info"); err == nil {
if _, err := LoadPoliciesOrDefault(t.Context(), filepath.Join("config", "testdata", "bad", st.Name()), anubis.DefaultDifficulty, "info", false); err == nil {
t.Fatal(err)
} else {
t.Log(err)
@@ -44,13 +44,13 @@ func TestGoodConfigs(t *testing.T) {
t.Run(st.Name(), func(t *testing.T) {
t.Run("with-thoth", func(t *testing.T) {
ctx := thothmock.WithMockThoth(t)
if _, err := LoadPoliciesOrDefault(ctx, filepath.Join("config", "testdata", "good", st.Name()), anubis.DefaultDifficulty, "info"); err != nil {
if _, err := LoadPoliciesOrDefault(ctx, filepath.Join("config", "testdata", "good", st.Name()), anubis.DefaultDifficulty, "info", false); err != nil {
t.Fatal(err)
}
})
t.Run("without-thoth", func(t *testing.T) {
if _, err := LoadPoliciesOrDefault(t.Context(), filepath.Join("config", "testdata", "good", st.Name()), anubis.DefaultDifficulty, "info"); err != nil {
if _, err := LoadPoliciesOrDefault(t.Context(), filepath.Join("config", "testdata", "good", st.Name()), anubis.DefaultDifficulty, "info", false); err != nil {
t.Fatal(err)
}
})
+30 -17
View File
@@ -207,7 +207,7 @@ func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, cr policy.C
return
}
lg := internal.GetRequestLogger(s.logger, r)
lg, r := s.getRequestLogger(r)
if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") && randomChance(64) {
lg.Error("client was given a challenge but does not in fact support gzip compression")
@@ -215,12 +215,19 @@ func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, cr policy.C
return
}
challengesIssued.WithLabelValues("embedded").Add(1)
{
asn, asnDesc := asnFromContext(r.Context())
challengesIssued.WithLabelValues("embedded", asn, asnDesc).Add(1)
}
chall, err := s.issueChallenge(r.Context(), r, lg, cr, rule)
if err != nil {
lg.Error("can't get challenge", "err", err)
algorithm := "unknown"
if rule.Challenge != nil {
algorithm = rule.Challenge.Algorithm
}
s.ClearCookie(w, CookieOpts{Name: anubis.TestCookieName, Host: r.Host})
s.respondWithError(w, r, fmt.Sprintf("%s: %s", localizer.T("internal_server_error"), rule.Challenge.Algorithm), makeCode(err))
s.respondWithError(w, r, fmt.Sprintf("%s: %s", localizer.T("internal_server_error"), algorithm), makeCode(err))
return
}
@@ -245,9 +252,13 @@ func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, cr policy.C
impl, ok := challenge.Get(chall.Method)
if !ok {
lg.Error("check failed", "err", "can't get algorithm", "algorithm", rule.Challenge.Algorithm)
algorithm := "unknown"
if rule.Challenge != nil {
algorithm = rule.Challenge.Algorithm
}
lg.Error("check failed", "err", "can't get algorithm", "algorithm", algorithm)
s.ClearCookie(w, CookieOpts{Name: anubis.TestCookieName, Host: r.Host})
s.respondWithError(w, r, fmt.Sprintf("%s: %s", localizer.T("internal_server_error"), rule.Challenge.Algorithm), makeCode(err))
s.respondWithError(w, r, fmt.Sprintf("%s: %s", localizer.T("internal_server_error"), algorithm), makeCode(err))
return
}
@@ -298,14 +309,14 @@ func (s *Server) constructRedirectURL(r *http.Request) (string, error) {
case "http", "https":
// allowed
default:
lg := internal.GetRequestLogger(s.logger, r)
lg, _ := s.getRequestLogger(r)
lg.Warn("invalid protocol in X-Forwarded-Proto", "proto", proto)
return "", errors.New(localizer.T("invalid_redirect"))
}
// Check if host is allowed in RedirectDomains (supports '*' via glob)
if len(s.opts.RedirectDomains) > 0 && !matchRedirectDomain(s.opts.RedirectDomains, host) {
lg := internal.GetRequestLogger(s.logger, r)
lg, _ := s.getRequestLogger(r)
lg.Debug("domain not allowed", "domain", host)
return "", errors.New(localizer.T("redirect_domain_not_allowed"))
}
@@ -392,14 +403,15 @@ func (s *Server) ServeHTTPNext(w http.ResponseWriter, r *http.Request) {
localizer := localization.GetLocalizer(r)
redir := r.FormValue("redir")
urlParsed, err := url.ParseRequestURI(redir)
urlParsed, err := url.Parse(redir)
if err != nil {
// if ParseRequestURI fails, try as relative URL
urlParsed, err = r.URL.Parse(redir)
if err != nil {
s.respondWithStatus(w, r, localizer.T("redirect_not_parseable"), makeCode(err), http.StatusBadRequest)
return
}
s.respondWithStatus(w, r, localizer.T("redirect_not_parseable"), makeCode(err), http.StatusBadRequest)
return
}
if urlParsed.Opaque != "" || (urlParsed.Scheme == "" && strings.HasPrefix(redir, "//")) {
s.respondWithStatus(w, r, localizer.T("invalid_redirect"), "", http.StatusBadRequest)
return
}
// validate URL scheme to prevent javascript:, data:, file:, tel:, etc.
@@ -407,7 +419,7 @@ func (s *Server) ServeHTTPNext(w http.ResponseWriter, r *http.Request) {
case "", "http", "https":
// allowed: empty scheme means relative URL
default:
lg := internal.GetRequestLogger(s.logger, r)
lg, _ := s.getRequestLogger(r)
lg.Warn("XSS attempt blocked, invalid redirect scheme", "scheme", urlParsed.Scheme, "redir", redir)
s.respondWithStatus(w, r, localizer.T("invalid_redirect"), "", http.StatusBadRequest)
return
@@ -419,7 +431,7 @@ func (s *Server) ServeHTTPNext(w http.ResponseWriter, r *http.Request) {
hostMismatch := r.URL.Host != "" && urlParsed.Host != "" && urlParsed.Host != r.URL.Host
if hostNotAllowed || hostMismatch {
lg := internal.GetRequestLogger(s.logger, r)
lg, _ := s.getRequestLogger(r)
lg.Debug("domain not allowed", "domain", urlParsed.Host)
s.respondWithStatus(w, r, localizer.T("redirect_domain_not_allowed"), makeCode(err), http.StatusBadRequest)
return
@@ -434,7 +446,8 @@ func (s *Server) ServeHTTPNext(w http.ResponseWriter, r *http.Request) {
web.Base(localizer.T("you_are_not_a_bot"), web.StaticHappy(localizer), s.policy.Impressum, localizer),
).ServeHTTP(w, r)
} else {
requestsProxied.WithLabelValues(r.Host).Inc()
asn, asnDesc := asnFromContext(r.Context())
requestsProxied.WithLabelValues(r.Host, asn, asnDesc).Inc()
r = s.stripBasePrefixFromRequest(r)
s.next.ServeHTTP(w, r)
}
+14
View File
@@ -223,3 +223,17 @@ func TestNoCacheOnError(t *testing.T) {
})
}
}
func TestRejectsHostlessRedirect(t *testing.T) {
pol := loadPolicies(t, "testdata/useragent.yaml", 0)
srv := spawnAnubis(t, Options{Policy: pol, RedirectDomains: []string{"allowed.example"}})
req := httptest.NewRequest(http.MethodGet, "https://anubis.example/.within.website/?redir=%2f%2fevil.example%2fphish", nil)
rr := httptest.NewRecorder()
srv.ServeHTTPNext(rr, req)
if rr.Code != http.StatusBadRequest {
t.Fatalf("expected hostless redirect to be rejected, got HTTP %d body %q", rr.Code, rr.Body.String())
}
if got := rr.Header().Get("Location"); got != "" {
t.Fatalf("expected no Location header on rejected redirect, got %q", got)
}
}
+66
View File
@@ -0,0 +1,66 @@
{
"loading": "Зареждане...",
"why_am_i_seeing": "Защо виждам това?",
"protected_by": "Защитено от",
"protected_from": "От",
"made_with": "Направено с ❤️ в 🇨🇦",
"mascot_design": "Дизайн на талисмана от",
"ai_companies_explanation": "Виждате това, защото администраторът на този уебсайт е kонфигурирал Anubis, за да защити сървъра от агресивното събиране на данни от компании, занимаващи се с изкуствен интелект. Това може и причинява прекъсвания на уебсайтовете, което прави техните ресурси недостъпни за всички.",
"anubis_compromise": "Anubis е компромис. Anubis използва схема за ддоказателство-за-работа по подобие на Hashcash, предложена схема за доказателство-за-работа за намаляване на спама в имейлите. Идеята е, че при индивидуални мащаби допълнителното натоварване е пренебрежимо, но при масов ниво на събиране на данни то се натрупва и прави събирането на данни много по-скъпо.",
"hack_purpose": "В крайна сметка, това е временно решение, за да се отдели повече време за идентифициране и разпознаване на безглави браузъри (например чрез това как те рендират шрифтовете), така че страницата за доказателство-за-работа да не се налага да се показва на потребители, които е по-вероятно да са легитимни.",
"simplified_explanation": "Това е мярка срещу ботове и злонамерени заявки, подобна на CAPTCHA. Вместо да трябва да правите нещо сами, браузърът ви получава задача за изчисление, която трябва да реши, за да се увери, че е валиден клиент. Тази концепция се нарича схема доказателство-за-работа. Задачата се изчислява за няколко секунди и ви се дава достъп до уебсайта. Благодаря ви за разбирането и търпението.",
"jshelter_note": "Моля, имайте предвид, че Anubis изисква използването на модерни функции на JavaScript, сред които и като JShelter ще деактивират. Моля, деактивирайте JShelter или други подобни добавки за този домейн.",
"version_info": "Този уебсайт използва версия на Anubis",
"try_again": "Опитайте отново",
"go_home": "Отидете на началната страница",
"contact_webmaster": "или ако смятате, че не трябва да бъдете блокирани, моля свържете се с уебмастъра на",
"connection_security": "Моля, изчакайте, докато се уверим в сигурността на връзката ви",
"javascript_required": "За съжаление, трябва да включите JavaScript, за да минете през това предизвикателство. Това е необходимо, защото компаниите за изкуствен интелект промениха социалния договор около начина на хостинг на уебсайтове. Решение без JavaScript е в процес на разработка.",
"benchmark_requires_js": "За да използвате инструмента за тестване, е необходимо да включите JavaScript.",
"difficulty": "Трудност:",
"algorithm": "Алгоритъм:",
"compare": "Сравни:",
"time": "Време",
"iters": "Итерации",
"time_a": "Време А",
"iters_a": "Итерации А",
"time_b": "Време Б",
"iters_b": "Итерации Б",
"static_check_endpoint": "Това е просто краен пункт за проверка, който да използва обратният ви прокси.",
"authorization_required": "Изисква се авторизация",
"cookies_disabled": "Браузърът ви е настроен да деактивира бисквитките. Anubis изисква бисквитки за законния интерес да се увери, че сте валиден клиент. Моля, включете бисквитките за този домейн",
"access_denied": "Достъпът е отказан: код на грешка",
"dronebl_entry": "DroneBL докладва запис",
"see_dronebl_lookup": "вижте",
"internal_server_error": "Вътрешна сървърна грешка: администраторът е грешно конфигурирал Anubis. Моля, свържете се с администратора и ги помолете да проверят логовете около",
"invalid_redirect": "Невалидно пренасочване",
"redirect_not_parseable": "URL адресът за пренасочване не може да бъде разпознат",
"redirect_domain_not_allowed": "Домейнът за пренасочване не е позволен",
"missing_required_forwarded_headers": "Липсват необходимите X-Forwarded-* заглавни части",
"failed_to_sign_jwt": "неуспешно подписване на JWT",
"invalid_invocation": "Невалидно извикване на MakeChallenge",
"client_error_browser": "Крешка в клиента: Моля, уверете се, че браузърът ви е актуализиран и опитайте отново по-късно.",
"oh_noes": "О, не!",
"benchmarking_anubis": "Тестване на Anubis!",
"you_are_not_a_bot": "Ти не си бот!",
"making_sure_not_bot": "Уверяваме се, че не си бот!",
"celphase": "CELPHASE",
"js_web_crypto_error": "Браузърът ви няма функциониращ web.crypto елемент. Гледате ли това през сигурен контекст?",
"js_web_workers_error": "Браузърът ви не поддържа уеб работници (Anubis използва това, за да избегне замръзване на браузъра ви). Имате ли инсталирана добавка като JShelter?",
"js_cookies_error": "Браузърът ви не съхранява бисквитки. Anubis използва бисквитки, за да определи които клиенти са преминали задачите, като съхранява подписан токен в бисквитка. Моля, включете съхраняването на бисквитки за този домейн. Имената на бисквитките, съхранени от Anubis, могат да се променят без предварително уведомление. Имената и стойностите на бисквитките не са част от публичния API.",
"js_context_not_secure": "Вашият контекст не е сигурен!",
"js_context_not_secure_msg": "Опитайте да се свържете чрез HTTPS или уведомете администратора да kонфигурира HTTPS. За повече информация вижте MDN.",
"js_calculating": "Изчисляване...",
"js_missing_feature": "Липсваща функция",
"js_challenge_error": "Грешка при задачата!",
"js_challenge_error_msg": "Неуспешно разрешаване на алгоритъма за проверка. Може би искате да презаредите страницата.",
"js_calculating_difficulty": "Изчисляване... Трудност:",
"js_speed": "Скорост:",
"js_verification_longer": "Проверката отнема повече време от очакваното. Моля, не презареждайте страницата.",
"js_success": "Успех!",
"js_done_took": "Готово! Отне",
"js_iterations": "итерации",
"js_finished_reading": "Приключих с четенето, продължете →",
"js_calculation_error": "Грешка при изчислението!",
"js_calculation_error_msg": "Неуспешно изчисление на задачата:"
}

Some files were not shown because too many files have changed in this diff Show More