Compare commits

...

53 Commits

Author SHA1 Message Date
Xe Iaso
50e48df993 fix(Dockerfile): add HEALTHCHECK
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-13 15:24:42 -04:00
Xe Iaso
5e38c7d730 feat: build with docker buildx bake
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-10 09:00:57 -04:00
Xe Iaso
c638653172 feat(lib): implement request weight (#621)
* feat(lib): implement request weight

Replaces #608

This is a big one and will be what makes Anubis a generic web
application firewall. This introduces the WEIGH option, allowing
administrators to have facets of request metadata add or remove
"weight", or the level of suspicion. This really makes Anubis weigh
the soul of requests.

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib): maintain legacy challenge behavior

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib): make weight have dedicated checkers for the hashes

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(data): convert some rules over to weight points

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: document request weight

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(CHANGELOG): spelling error

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: fix links to challenge information

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs(policies): fix formatting

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(config): make default weight adjustment 5

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-09 15:25:04 -04:00
Fierelier
0fe46b48cf Make progress bar styling more compatible (UXP, etc) (#636)
* Make progress bar styling more compatible (UXP, etc)

* Add 'Make progress bar styling more compatible (UXP, etc)'

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Fierelier <fier@airmail.cc>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-06-09 12:19:38 -04:00
David Chandek-Stark
d6e5561768 Adds ability to toggle off stripping of private addrs from XFF (#619)
* Adds ability to toggle off stripping of private addrs from XFF

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: refactor to flow better

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-06-09 13:33:19 +00:00
dependabot[bot]
6594ae0eef build(deps): bump github/codeql-action in the github-actions group (#635)
Bumps the github-actions group with 1 update: [github/codeql-action](https://github.com/github/codeql-action).


Updates `github/codeql-action` from 3.28.18 to 3.28.19
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](ff0a06e83c...fca7ace96b)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-version: 3.28.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-09 08:56:41 -04:00
Lothar Serra Mari
ad09f82c3c docs(admin/environments): Prefer IPv6 over IPv4 for apache2 listener directive (#628)
Signed-off-by: Lothar Serra Mari <mail@serra.me>
2025-06-09 08:56:30 -04:00
Xe Iaso
372b797f64 chore: go generate
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-08 20:52:22 -04:00
dependabot[bot]
6eaf0e13a2 build(deps): bump the gomod group with 2 updates (#634)
Bumps the gomod group with 2 updates: [github.com/a-h/templ](https://github.com/a-h/templ) and [golang.org/x/net](https://github.com/golang/net).


Updates `github.com/a-h/templ` from 0.3.887 to 0.3.898
- [Release notes](https://github.com/a-h/templ/releases)
- [Changelog](https://github.com/a-h/templ/blob/main/.goreleaser.yaml)
- [Commits](https://github.com/a-h/templ/compare/v0.3.887...v0.3.898)

Updates `golang.org/x/net` from 0.40.0 to 0.41.0
- [Commits](https://github.com/golang/net/compare/v0.40.0...v0.41.0)

---
updated-dependencies:
- dependency-name: github.com/a-h/templ
  dependency-version: 0.3.898
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: gomod
- dependency-name: golang.org/x/net
  dependency-version: 0.41.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: gomod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-08 20:44:25 -04:00
Dryusdan
281b6c5c00 Bump ai.robots.txt to v1.34 (#632) 2025-06-08 14:54:47 -04:00
Jason Cameron
9539668049 style: Some minor fixes (#548)
* chore(deps): update dependencies in go.mod and go.sum

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor: rename variables for clarity in anubis.go and main.go

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* fix(checker): handle error when inserting IP range in ranger

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* fix(tests): simplify boolean checks in header and URL value tests

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor(api): remove unused /test-error endpoint and restrict /make-challenge to development

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* build(deps): update golang-set to v2.8.0 in go.sum

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* Update metadata

check-spelling run (pull_request) for json/stuff

Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
on-behalf-of: @check-spelling <check-spelling-bot@check-spelling.dev>

---------

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
2025-06-07 18:21:22 +00:00
Xe Iaso
8eff57fcb6 chore(docs/manifest): try no-js challenge to see how it impacts false positive rate
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-06 21:40:28 -04:00
Xe Iaso
4ac59c3a79 feat(lib/challenge): HTTP meta refresh challenge method (#623)
* feat(lib/challenge): HTTP meta refresh challenge method

Closes #95

This challenge method enables users that don't (or won't) support
JavaScript to pass Anubis challenges. It works by using HTML meta
refresh directives to ensure that the client is a browser.

This is OFF by default. In order to enable it, an administrator MUST
choose to make the default challenge method `metarefresh`.

TODO(Xe):

- [ ] Documentation on this challenge method
- [ ] Amend wording around Anubis being a proof of work proxy in the docs
- [ ] Add configuration file syntax for the default challenge method and settings
- [ ] Test with early customers

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib/challenge/metarefresh): use this value of err

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: add metarefresh challenge info, Web AI Firewall Utility

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-06 21:18:55 -04:00
Lothar Serra Mari
bee1c22b96 docs(known-instances): add wiki.dolphin-emu.org to known instances (#626)
Signed-off-by: Lothar Serra Mari <mail@serra.me>
2025-06-06 13:35:24 -04:00
Xe Iaso
5a7499ea3b fix(lib/challenge): allow challenges to register HTTP routes (#620)
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-06 00:26:23 +00:00
Jan Pieter Waagmeester
5f3861ab37 docs: Adjust the name of the cookie to the current "techaro.lol-anubis-auth" (#615)
* docs: Adjust the name of the cookie to the current "techaro.lol-anubis-auth"

Name definition:

76fa3e01a5/anubis.go (L12-L14)

The name changed in 6c0ff3f4d5


Signed-off-by: Jan Pieter Waagmeester <jieter@jieter.nl>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Jan Pieter Waagmeester <jieter@jieter.nl>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-06-05 20:59:16 +00:00
foosinn
9f1d791991 docs(subrequest-auth): document required policy changes (#613)
* docs(subrequest-auth): document required policy changes

Signed-off-by: foosinn <foosinn@f2o.io>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: foosinn <foosinn@f2o.io>
Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-06-05 16:53:18 -04:00
Markus Sommer
76fa3e01a5 docs(known-instances): add Alliance of Hessian Libraries (#611)
Signed-off-by: Markus Sommer <markus@splork.de>
2025-06-04 02:03:57 +00:00
Xe Iaso
f2db43ad4b feat: implement challenge registry (#607)
* feat: implement challenge method registry

This paves the way for implementing a no-js check method (#95) by making
the challenge providers more generic.

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib/challenge): rename proof-of-work package to proofofwork

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib): make validated challenges a CounterVec

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib): annotate jwts with challenge method

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(lib/challenge/proofofwork): implement tests

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(lib): add smoke tests for known good and known bad config files

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: update CHANGELOG

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib): use challenge.Impl#Issue when issuing challenges

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-04 02:01:58 +00:00
Xe Iaso
ba4412c907 chore(sponsors): add Raptor Computing Systems
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-03 17:49:28 -04:00
Xe Iaso
f184cd81e7 docs(faq): anubis does not mine bitcoin (#609)
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-03 07:14:41 -04:00
Xe Iaso
59bfced8bf docs(admin/environments): update suggested HTTP headers
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-03 06:57:37 -04:00
Xe Iaso
780a935cb8 chore(sponsors): add wildbase
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-03 06:18:40 -04:00
Xe Iaso
f4bc1df797 chore(sponsors): add Uberspace
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-02 09:42:13 -04:00
dependabot[bot]
b496c90e86 build(deps): bump github.com/a-h/templ in the gomod group (#601)
Bumps the gomod group with 1 update: [github.com/a-h/templ](https://github.com/a-h/templ).


Updates `github.com/a-h/templ` from 0.3.865 to 0.3.887
- [Release notes](https://github.com/a-h/templ/releases)
- [Changelog](https://github.com/a-h/templ/blob/main/.goreleaser.yaml)
- [Commits](https://github.com/a-h/templ/compare/v0.3.865...v0.3.887)

---
updated-dependencies:
- dependency-name: github.com/a-h/templ
  dependency-version: 0.3.887
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: gomod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-01 23:39:42 -04:00
dependabot[bot]
ec73bcbaf1 build(deps): bump docker/build-push-action in the github-actions group (#602)
Bumps the github-actions group with 1 update: [docker/build-push-action](https://github.com/docker/build-push-action).


Updates `docker/build-push-action` from 6.17.0 to 6.18.0
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](1dc7386353...263435318d)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: 6.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-01 23:39:05 -04:00
dependabot[bot]
8d19eed200 build(deps-dev): bump esbuild from 0.25.4 to 0.25.5 in the npm group (#600)
Bumps the npm group with 1 update: [esbuild](https://github.com/evanw/esbuild).


Updates `esbuild` from 0.25.4 to 0.25.5
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.25.4...v0.25.5)

---
updated-dependencies:
- dependency-name: esbuild
  dependency-version: 0.25.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: npm
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-01 23:38:45 -04:00
Xe Iaso
ec733e93a5 v1.19.1
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-01 17:17:24 -04:00
Xe Iaso
51c384eefd fix(data/bots): bring back ai-robots-txt.yaml
Closes #599

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-01 17:15:00 -04:00
Xe Iaso
44d5ec0b6e chore: release version v1.19.0
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-01 16:35:03 -04:00
Xe Iaso
3bc9040a96 chore: bump yeet to v0.6.0
Gives us many nice things like:

* Windows support for yeet (modulo TecharoHQ/yeet#29)
* Removes the dependency on /bin/sh or /bin/bash thanks to
  mvdan.cc/sh/v3
* Checksum-compliant reproducible builds by default

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-01 16:33:08 -04:00
Corry Haines
de7dbfe6d6 Split up AI filtering files (#592)
* Split up AI filtering files

Create aggressive/moderate/permissive policies to allow administrators to choose their AI/LLM stance.

Aggressive policy matches existing default in Anubis.

Removes `Google-Extended` flag from `ai-robots-txt.yaml` as it doesn't exist in requests.

Rename `ai-robots-txt.yaml` to `ai-catchall.yaml` as the file is no longer a copy of the source repo/file.

* chore: spelling

* chore: fix embeds

* chore: fix data includes

* chore: fix file name typo

* chore: Ignore READMEs in configs

* chore(lib/policy/config): go tool goimports -w

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-06-01 20:21:18 +00:00
minihoot
77e0bbbce9 add my site to known-instances.md (#595)
Signed-off-by: minihoot <70825884+minihoot@users.noreply.github.com>
2025-06-01 15:28:16 +00:00
Zohiu
b4b5d2f82e docs(known-instances): add catgirl.click (#597)
Signed-off-by: Zohiu <102434603+Zohiu@users.noreply.github.com>
2025-06-01 14:55:02 +00:00
Aleksei
988fff77f1 docs(known-instances): add openwrt.org (#594)
Signed-off-by: Aleksei <Aloki@users.noreply.github.com>
2025-06-01 13:13:52 +00:00
Corry Haines
0d9ebebff6 Opt-in policies for OpenAI and MistralAI bots (#590)
* Define OpenAI bot ALLOW policies

Allows OpenAI bots to be allowlisted at the choice of the Anubis administrator. None are enabled by default.

* Define MistralAI bot ALLOW policy

* chore: spelling
2025-05-31 16:48:57 -04:00
Jason
ba00cdacd2 docs(known-instances): Add Gitea to the known instances list (#591)
Signed-off-by: Jason <57271945+jesentz@users.noreply.github.com>
2025-05-31 14:20:39 -04:00
Corry Haines
68a71c6a99 Add Applebot definition (#589)
* Add Applebot definition

Adds Apple's search indexing bot, and allowlists it by default.

Allowlisted by default because it is equivalent to Googlebot/Bingbot. Remove Applebot from `ai-robots-txt.yaml` for the same reasons.

Remove `Applebot-Extended` from `ai-robots-txt.yaml` as it has no effect.

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-05-31 10:18:32 -04:00
Xe Iaso
fbbab5a035 feat(lib): annotate cookies with what rule was passed (#576)
* feat(lib): annotate cookies with what rule was passed

Anubis JWTs now contain a policyRule claim with the cryptographic hash
of the rule that it passed. This is intended to help with a future move
away from proof of work being the default.

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(lib): fix cookie storage logic

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-05-30 14:42:02 -04:00
Jason Cameron
28ab29389c style(bench): small cleanup (#546)
* fix(bench): await benchmark loop and adjust outline styles in templates

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* refactor: remove unused showContinueBar function and clean up video error handling

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* style: format code for consistency and readability using prettier

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

---------

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
2025-05-30 17:57:56 +00:00
Xe Iaso
497005ce3e fix(lib): only use the first five characters of Accept-Language header values (#588)
For some reason, Google Chrome will randomly send a "full"
Accept-Language header, and other times it will send a "partial"
Accept-Language header. This makes the challenge construction
inconsistent.

This commit fixes this issue by only considering up to the first five
characters of the Accept-Language header when making a challenge string.

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-05-30 13:15:03 -04:00
Xe Iaso
669eb4ba4b fix(web): show Anubis version number on challenge pages (#587)
Closes #565

The page already had the version number embedded into it, but that was
not printed to the page. This prints the version number set at compile
time to the page.

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-05-30 16:23:27 +00:00
Kian Kasad
6c4e739b0b feat(lib): Add anubis_proxied_requests_total metric (#570)
Adds a new counter metric which counts the number of requests proxied to
the upstream target, labeled with the host name of the request.
2025-05-30 12:21:04 -04:00
Xe Iaso
c8635357dc feat(yeetfile): build GOARCH=ppc64le packages (#583)
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-05-30 00:35:56 -04:00
Xe Iaso
0ed905fd4e fix(internal/test): skip integration tests if SKIP_INTEGRATION is set (#586)
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-05-30 00:49:53 +00:00
Xe Iaso
cd8a7eb2e2 feat(data): add x-firefox-ai default challenge rule (#580)
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-05-28 21:08:39 +00:00
Xe Iaso
22c47f40d1 feat(expressions): add randInt function to allow making rules nondeterministic (#578)
This seems counter-intuitive at first glance, but let me cook.

One of the problems with Anubis is that the rule matching is super
deterministic. This means that attackers can figure out what patterns
they are hitting and change things to bypass them.

The randInt function lets you have rulesets behave nondeterministically.
This is a very easy way to hang yourself, but can be great to
psychologically mess with scraper operators. Consider this rule:

```yaml
- name: deny-lightpanda-sometimes
  action: DENY
  expression:
    all:
      - userAgent.matches("LightPanda")
      - randInt(16) >= 4
```

It would match about 75% of the time.

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-05-28 16:36:27 -04:00
Xe Iaso
669671bd46 fix(internal): register mime type for .mjs files (#577)
Closes #575

I'm gonna be totally honest, I'm not sure if this is needed. However,
measure twice, cut once.

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-05-28 13:52:48 +00:00
dependabot[bot]
6c247cdec8 build(deps): bump k8s.io/apimachinery in the gomod group (#524)
Bumps the gomod group with 1 update: [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery).


Updates `k8s.io/apimachinery` from 0.33.0 to 0.33.1
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.33.0...v0.33.1)

---
updated-dependencies:
- dependency-name: k8s.io/apimachinery
  dependency-version: 0.33.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: gomod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-28 09:41:25 -04:00
Kian Kasad
eeae28f459 feat(cli): Add --version flag (#572) 2025-05-28 04:16:44 +00:00
jordigh
9ba10262e3 add Weblate to known-instances.md (#571)
Signed-off-by: jordigh <jordigh@octave.org>
2025-05-28 00:03:04 +00:00
dependabot[bot]
a28a3d155a build(deps): bump astral-sh/setup-uv in the github-actions group (#558)
Bumps the github-actions group with 1 update: [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv).


Updates `astral-sh/setup-uv` from 6.0.1 to 6.1.0
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](6b9c6063ab...f0ec1fc3b3)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 6.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-27 11:12:31 -04:00
Anna
086f43e3ca Create Anubis OpenRC init.d script (#561)
Signed-off-by: Anna @CyberTailor <cyber@sysrq.in>
2025-05-27 01:58:59 +00:00
112 changed files with 2344 additions and 730 deletions

25
.dockerignore Normal file
View File

@@ -0,0 +1,25 @@
.env
*.deb
*.rpm
# Additional package locks
pnpm-lock.yaml
yarn.lock
# Go binaries and test artifacts
main
*.test
node_modules
# MacOS
.DS_store
# Intellij
.idea
# how does this get here
doc/VERSION
web/static/js/*
!web/static/js/.gitignore

View File

@@ -6,38 +6,46 @@ amazonbot
anthro
anubis
anubistest
Applebot
archlinux
badregexes
bdba
berr
bingbot
Bitcoin
bitcoin
blogging
Bluesky
blueskybot
boi
botnet
BPort
Brightbot
broked
Bytespider
cachebuster
Caddyfile
caninetools
Cardyb
celchecker
CELPHASE
cerr
certresolver
CGNAT
cgr
chainguard
chall
challengemozilla
checkpath
checkresult
chen
chibi
cidranger
ckie
cloudflare
confd
containerbuild
coreutils
Cotoyogi
CRDs
crt
daemonizing
@@ -46,6 +54,7 @@ Debian
debrpm
decaymap
decompiling
Diffbot
discordapp
discordbot
distros
@@ -56,17 +65,22 @@ dracula
dronebl
droneblresponse
duckduckbot
eerror
ellenjoe
enbyware
everyones
evilbot
evilsite
expressionorlist
externalagent
externalfetcher
extldflags
facebookgo
Factset
fastcgi
fediverse
finfos
Firecrawl
flagenv
Fordola
forgejo
@@ -81,22 +95,30 @@ goodbot
googlebot
govulncheck
GPG
GPT
gptbot
grw
Hashcash
hashrate
headermap
healthcheck
hebis
hec
hmc
hostable
htmx
httpdebug
hypertext
iaskspider
iat
ifm
Imagesift
imgproxy
inp
iss
isset
ivh
Jenomis
JGit
journalctl
jshelter
@@ -108,8 +130,10 @@ keypair
KHTML
kinda
KUBECONFIG
lcj
ldflags
letsencrypt
Lexentale
lgbt
licend
licstart
@@ -125,19 +149,30 @@ maintainership
malware
mcr
memes
metarefresh
metrix
mimi
minica
mistralai
Mojeek
mojeekbot
mozilla
nbf
netsurf
nginx
nobots
NONINFRINGEMENT
nosleep
OCOB
ogtags
omgili
omgilibot
onionservice
openai
openrc
pag
palemoon
Pangu
parseable
passthrough
Patreon
@@ -153,6 +188,7 @@ prebaked
privkey
promauto
promhttp
proofofwork
pwcmd
pwuser
qualys
@@ -167,21 +203,27 @@ reputational
reqmeta
risc
ruleset
runlevels
RUnlock
sas
sasl
Scumm
searchbot
searx
sebest
secretplans
selfsigned
Semrush
setsebool
shellcheck
Sidetrade
sitemap
sls
sni
Sourceware
Spambot
sparkline
spyderbot
srv
stackoverflow
startprecmd
@@ -189,6 +231,7 @@ stoppostcmd
subgrid
subr
subrequest
SVCNAME
tagline
tarballs
techaro
@@ -196,12 +239,17 @@ techarohq
templ
templruntime
testarea
Tik
Timpibot
torproject
traefik
uberspace
unixhttpd
unmarshal
uvx
UXP
Varis
Velen
vendored
vhosts
videotest
@@ -211,8 +259,12 @@ webmaster
webpage
websecure
websites
Webzio
wildbase
wordpress
Workaround
workdir
wpbot
xcaddy
Xeact
xeiaso

View File

@@ -39,7 +39,7 @@ jobs:
- name: Build and push
id: build
uses: docker/build-push-action@1dc73863535b631f98b2378be8619f83b136f4a0 # v6.17.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ./docs
cache-to: type=gha

View File

@@ -28,7 +28,7 @@ jobs:
- name: Build and push
id: build
uses: docker/build-push-action@1dc73863535b631f98b2378be8619f83b136f4a0 # v6.17.0
uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0
with:
context: ./docs
cache-to: type=gha

View File

@@ -21,7 +21,7 @@ jobs:
persist-credentials: false
- name: Install the latest version of uv
uses: astral-sh/setup-uv@6b9c6063abd6010835644d4c2e1bef4cf5cd0fca # v6.0.1
uses: astral-sh/setup-uv@f0ec1fc3b38f5e7cd731bb6ce540c5af426746bb # v6.1.0
- name: Run zizmor 🌈
run: uvx zizmor --format sarif . > results.sarif
@@ -29,7 +29,7 @@ jobs:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@ff0a06e83cb2de871e5a09832bc6a81e7276941f # v3.28.18
uses: github/codeql-action/upload-sarif@fca7ace96b7d713c7035871441bd52efbe39e27e # v3.28.19
with:
sarif_file: results.sarif
category: zizmor

30
Dockerfile Normal file
View File

@@ -0,0 +1,30 @@
ARG ALPINE_VERSION=edge
FROM --platform=${BUILDPLATFORM} alpine:${ALPINE_VERSION} AS build
ARG TARGETOS
ARG TARGETARCH
ARG COMPONENT=anubis
ARG VERSION=devel-docker
RUN apk -U add go nodejs git build-base git npm bash zstd brotli gzip
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN --mount=type=cache,target=/root/.cache npm ci && npm run assets
RUN --mount=type=cache,target=/root/.cache GOOS=${TARGETOS} GOARCH=${TARGETARCH} CGO_ENABLED=0 GOARM=7 go build -gcflags "all=-N -l" -o /app/bin/${COMPONENT} -ldflags "-s -w -extldflags -static -X github.com/TecharoHQ/anubis.Version=${VERSION}" ./cmd/${COMPONENT}
FROM alpine:${ALPINE_VERSION} AS run
WORKDIR /app
RUN apk -U add ca-certificates mailcap
COPY --from=build /app/bin/anubis /app/bin/anubis
CMD ["/app/bin/anubis"]
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 CMD [ "/app/bin/anubis", "--healthcheck" ]
LABEL org.opencontainers.image.source="https://github.com/TecharoHQ/anubis"

View File

@@ -14,14 +14,36 @@
Anubis is brought to you by sponsors and donors like:
[![Distrust](./docs/static/img/sponsors/distrust-logo.webp)](https://distrust.co?utm_campaign=github&utm_medium=referral&utm_content=anubis)
[![Terminal Trove](./docs/static/img/sponsors/terminal-trove.webp)](https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=anubis&utm_source=abgh)
[![canine.tools](./docs/static/img/sponsors/caninetools-logo.webp)](https://canine.tools?utm_campaign=github&utm_medium=referral&utm_content=anubis)
[![Weblate](./docs/static/img/sponsors/weblate-logo.webp)](https://weblate.org/?utm_campaign=github&utm_medium=referral&utm_content=anubis)
### Diamond Tier
<a href="https://www.raptorcs.com/content/base/products.html">
<img src="./docs/static/img/sponsors/raptor-computing-logo.webp" alt="Raptor Computing Systems" height=64 />
</a>
### Gold Tier
<a href="https://distrust.co?utm_campaign=github&utm_medium=referral&utm_content=anubis">
<img src="./docs/static/img/sponsors/distrust-logo.webp" alt="Distrust" height="64">
</a>
<a href="https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=anubis&utm_source=abgh">
<img src="./docs/static/img/sponsors/terminal-trove.webp" alt="Terminal Trove" height="64">
</a>
<a href="https://canine.tools?utm_campaign=github&utm_medium=referral&utm_content=anubis">
<img src="./docs/static/img/sponsors/caninetools-logo.webp" alt="canine.tools" height="64">
</a>
<a href="https://weblate.org/">
<img src="./docs/static/img/sponsors/weblate-logo.webp" alt="Weblate" height="64">
</a>
<a href="https://uberspace.de/">
<img src="./docs/static/img/sponsors/uberspace-logo.webp" alt="Uberspace" height="64">
</a>
<a href="https://wildbase.xyz/">
<img src="./docs/static/img/sponsors/wildbase-logo.webp" alt="Wildbase" height="64">
</a>
## Overview
Anubis [weighs the soul of your connection](https://en.wikipedia.org/wiki/Weighing_of_souls) using a proof-of-work challenge in order to protect upstream resources from scraper bots.
Anubis is a Web AI Firewall Utility that [weighs the soul of your connection](https://en.wikipedia.org/wiki/Weighing_of_souls) using one or more challenges in order to protect upstream resources from scraper bots.
This program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies. Anubis is as lightweight as possible to ensure that everyone can afford to protect the communities closest to them.

View File

@@ -1 +1 @@
1.19.0-pre1
1.19.1

View File

@@ -67,6 +67,8 @@ var (
ogCacheConsiderHost = flag.Bool("og-cache-consider-host", false, "enable or disable the use of the host in the Open Graph tag cache")
extractResources = flag.String("extract-resources", "", "if set, extract the static resources to the specified folder")
webmasterEmail = flag.String("webmaster-email", "", "if set, displays webmaster's email on the reject page for appeals")
versionFlag = flag.Bool("version", false, "print Anubis version")
xffStripPrivate = flag.Bool("xff-strip-private", true, "if set, strip private addresses from X-Forwarded-For")
)
func keyFromHex(value string) (ed25519.PrivateKey, error) {
@@ -202,6 +204,11 @@ func main() {
flagenv.Parse()
flag.Parse()
if *versionFlag {
fmt.Println("Anubis", anubis.Version)
return
}
internal.InitSlog(*slogLevel)
if *extractResources != "" {
@@ -330,7 +337,7 @@ func main() {
h = s
h = internal.RemoteXRealIP(*useRemoteAddress, *bindNetwork, h)
h = internal.XForwardedForToXRealIP(h)
h = internal.XForwardedForUpdate(h)
h = internal.XForwardedForUpdate(*xffStripPrivate, h)
srv := http.Server{Handler: h, ErrorLog: internal.GetFilteredHTTPLogger()}
listener, listenerUrl := setupListener(*bindNetwork, *bind)
@@ -414,11 +421,11 @@ func extractEmbedFS(fsys embed.FS, root string, destDir string) error {
return os.MkdirAll(destPath, 0o700)
}
data, err := fs.ReadFile(fsys, path)
embeddedData, err := fs.ReadFile(fsys, path)
if err != nil {
return err
}
return os.WriteFile(destPath, data, 0o644)
return os.WriteFile(destPath, embeddedData, 0o644)
})
}

View File

@@ -4,7 +4,7 @@
"import": "(data)/bots/_deny-pathological.yaml"
},
{
"import": "(data)/bots/ai-robots-txt.yaml"
"import": "(data)/meta/ai-block-aggressive.yaml"
},
{
"import": "(data)/crawlers/_allow-good.yaml"

View File

@@ -11,44 +11,53 @@
## /usr/share/docs/anubis/data or in the tarball you extracted Anubis from.
bots:
# Pathological bots to deny
- # This correlates to data/bots/deny-pathological.yaml in the source tree
# https://github.com/TecharoHQ/anubis/blob/main/data/bots/deny-pathological.yaml
import: (data)/bots/_deny-pathological.yaml
- import: (data)/bots/aggressive-brazilian-scrapers.yaml
# Pathological bots to deny
- # This correlates to data/bots/deny-pathological.yaml in the source tree
# https://github.com/TecharoHQ/anubis/blob/main/data/bots/deny-pathological.yaml
import: (data)/bots/_deny-pathological.yaml
- import: (data)/bots/aggressive-brazilian-scrapers.yaml
# Enforce https://github.com/ai-robots-txt/ai.robots.txt
- import: (data)/bots/ai-robots-txt.yaml
# Aggressively block AI/LLM related bots/agents by default
- import: (data)/meta/ai-block-aggressive.yaml
# Search engine crawlers to allow, defaults to:
# - Google (so they don't try to bypass Anubis)
# - Bing
# - DuckDuckGo
# - Qwant
# - The Internet Archive
# - Kagi
# - Marginalia
# - Mojeek
- import: (data)/crawlers/_allow-good.yaml
# Consider replacing the aggressive AI policy with more selective policies:
# - import: (data)/meta/ai-block-moderate.yaml
# - import: (data)/meta/ai-block-permissive.yaml
# Allow common "keeping the internet working" routes (well-known, favicon, robots.txt)
- import: (data)/common/keep-internet-working.yaml
# Search engine crawlers to allow, defaults to:
# - Google (so they don't try to bypass Anubis)
# - Apple
# - Bing
# - DuckDuckGo
# - Qwant
# - The Internet Archive
# - Kagi
# - Marginalia
# - Mojeek
- import: (data)/crawlers/_allow-good.yaml
# Challenge Firefox AI previews
- import: (data)/clients/x-firefox-ai.yaml
# # Punish any bot with "bot" in the user-agent string
# # This is known to have a high false-positive rate, use at your own risk
# - name: generic-bot-catchall
# user_agent_regex: (?i:bot|crawler)
# action: CHALLENGE
# challenge:
# difficulty: 16 # impossible
# report_as: 4 # lie to the operator
# algorithm: slow # intentionally waste CPU cycles and time
# Allow common "keeping the internet working" routes (well-known, favicon, robots.txt)
- import: (data)/common/keep-internet-working.yaml
# Generic catchall rule
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: CHALLENGE
# # Punish any bot with "bot" in the user-agent string
# # This is known to have a high false-positive rate, use at your own risk
# - name: generic-bot-catchall
# user_agent_regex: (?i:bot|crawler)
# action: CHALLENGE
# challenge:
# difficulty: 16 # impossible
# report_as: 4 # lie to the operator
# algorithm: slow # intentionally waste CPU cycles and time
# Generic catchall rule
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: WEIGH
weight:
adjust: 10
dnsbl: false
@@ -58,4 +67,4 @@ dnsbl: false
# will stop sending requests once they get it.
status_codes:
CHALLENGE: 200
DENY: 200
DENY: 200

View File

@@ -1,28 +1,26 @@
- name: deny-aggressive-brazilian-scrapers
action: DENY
action: WEIGH
weight:
adjust: 20
expression:
any:
# Internet Explorer should be out of support
- userAgent.contains("MSIE")
# Trident is the Internet Explorer browser engine
- userAgent.contains("Trident")
# Opera is a fork of chrome now
- userAgent.contains("Presto")
# Windows CE is discontinued
- userAgent.contains("Windows CE")
# Windows 95 is discontinued
- userAgent.contains("Windows 95")
# Windows 98 is discontinued
- userAgent.contains("Windows 98")
# Windows 9.x is discontinued
- userAgent.contains("Win 9x")
# Amazon does not have an Alexa Toolbar.
- userAgent.contains("Alexa Toolbar")
- name: challenge-aggressive-brazilian-scrapers
action: CHALLENGE
expression:
any:
# This is not released, even Windows 11 calls itself Windows 10
- userAgent.contains("Windows NT 11.0")
# iPods are not in common use
- userAgent.contains("iPod")
# Internet Explorer should be out of support
- userAgent.contains("MSIE")
# Trident is the Internet Explorer browser engine
- userAgent.contains("Trident")
# Opera is a fork of chrome now
- userAgent.contains("Presto")
# Windows CE is discontinued
- userAgent.contains("Windows CE")
# Windows 95 is discontinued
- userAgent.contains("Windows 95")
# Windows 98 is discontinued
- userAgent.contains("Windows 98")
# Windows 9.x is discontinued
- userAgent.contains("Win 9x")
# Amazon does not have an Alexa Toolbar.
- userAgent.contains("Alexa Toolbar")
# This is not released, even Windows 11 calls itself Windows 10
- userAgent.contains("Windows NT 11.0")
# iPods are not in common use
- userAgent.contains("iPod")

View File

@@ -0,0 +1,11 @@
# Extensive list of AI-affiliated agents based on https://github.com/ai-robots-txt/ai.robots.txt
# Add new/undocumented agents here. Where documentation exists, consider moving to dedicated policy files.
# Notes on various agents:
# - Amazonbot: Well documented, but they refuse to state which agent collects training data.
# - anthropic-ai/Claude-Web: Undocumented by Anthropic. Possibly deprecated or hallucinations?
# - Perplexity*: Well documented, but they refuse to state which agent collects training data.
# Warning: May contain user agents that _must_ be blocked in robots.txt, or the opt-out will have no effect.
- name: "ai-catchall"
user_agent_regex: >-
AI2Bot|Ai2Bot-Dolma|aiHitBot|Amazonbot|anthropic-ai|Brightbot 1.0|Bytespider|CCBot|Claude-Web|cohere-ai|cohere-training-data-crawler|Cotoyogi|Crawlspace|Diffbot|DuckAssistBot|FacebookBot|Factset_spyderbot|FirecrawlAgent|FriendlyCrawler|Google-CloudVertexBot|GoogleOther|GoogleOther-Image|GoogleOther-Video|iaskspider/2.0|ICC-Crawler|ImagesiftBot|img2dataset|imgproxy|ISSCyberRiskCrawler|Kangaroo Bot|meta-externalagent|Meta-ExternalAgent|meta-externalfetcher|Meta-ExternalFetcher|NovaAct|omgili|omgilibot|Operator|PanguBot|Perplexity-User|PerplexityBot|PetalBot|QualifiedBot|Scrapy|SemrushBot-OCOB|SemrushBot-SWA|Sidetrade indexer bot|TikTokSpider|Timpibot|VelenPublicWebCrawler|Webzio-Extended|wpbot|YouBot
action: DENY

View File

@@ -1,4 +1,6 @@
# Warning: Contains user agents that _must_ be blocked in robots.txt, or the opt-out will have no effect.
# Note: Blocks human-directed/non-training user agents
- name: "ai-robots-txt"
user_agent_regex: >-
AI2Bot|Ai2Bot-Dolma|aiHitBot|Amazonbot|anthropic-ai|Applebot|Applebot-Extended|Brightbot 1.0|Bytespider|CCBot|ChatGPT-User|Claude-SearchBot|Claude-User|Claude-Web|ClaudeBot|cohere-ai|cohere-training-data-crawler|Cotoyogi|Crawlspace|Diffbot|DuckAssistBot|FacebookBot|Factset_spyderbot|FirecrawlAgent|FriendlyCrawler|Google-CloudVertexBot|Google-Extended|GoogleOther|GoogleOther-Image|GoogleOther-Video|GPTBot|iaskspider/2.0|ICC-Crawler|ImagesiftBot|img2dataset|imgproxy|ISSCyberRiskCrawler|Kangaroo Bot|meta-externalagent|Meta-ExternalAgent|meta-externalfetcher|Meta-ExternalFetcher|MistralAI-User/1.0|NovaAct|OAI-SearchBot|omgili|omgilibot|Operator|PanguBot|Perplexity-User|PerplexityBot|PetalBot|QualifiedBot|Scrapy|SemrushBot-OCOB|SemrushBot-SWA|Sidetrade indexer bot|TikTokSpider|Timpibot|VelenPublicWebCrawler|Webzio-Extended|wpbot|YouBot
AI2Bot|Ai2Bot-Dolma|aiHitBot|Amazonbot|Andibot|anthropic-ai|Applebot|Applebot-Extended|bedrockbot|Brightbot 1.0|Bytespider|CCBot|ChatGPT-User|Claude-SearchBot|Claude-User|Claude-Web|ClaudeBot|cohere-ai|cohere-training-data-crawler|Cotoyogi|Crawlspace|Diffbot|DuckAssistBot|FacebookBot|Factset_spyderbot|FirecrawlAgent|FriendlyCrawler|Google-CloudVertexBot|Google-Extended|GoogleOther|GoogleOther-Image|GoogleOther-Video|GPTBot|iaskspider/2.0|ICC-Crawler|ImagesiftBot|img2dataset|ISSCyberRiskCrawler|Kangaroo Bot|meta-externalagent|Meta-ExternalAgent|meta-externalfetcher|Meta-ExternalFetcher|MistralAI-User/1.0|NovaAct|OAI-SearchBot|omgili|omgilibot|Operator|PanguBot|Panscient|panscient.com|Perplexity-User|PerplexityBot|PetalBot|PhindBot|QualifiedBot|QuillBot|quillbot.com|SBIntuitionsBot|Scrapy|SemrushBot-OCOB|SemrushBot-SWA|Sidetrade indexer bot|TikTokSpider|Timpibot|VelenPublicWebCrawler|Webzio-Extended|wpbot|YandexAdditional|YandexAdditionalBot|YouBot
action: DENY

View File

@@ -1,4 +1,6 @@
- name: cloudflare-workers
headers_regex:
CF-Worker: .*
action: DENY
action: WEIGH
weight:
adjust: 15

8
data/clients/ai.yaml Normal file
View File

@@ -0,0 +1,8 @@
# User agents that act on behalf of humans in AI tools, e.g. searching the web.
# Each entry should have a positive/ALLOW entry created as well, with further documentation.
# Exceptions:
# - Claude-User: No published IP allowlist
- name: "ai-clients"
user_agent_regex: >-
ChatGPT-User|Claude-User|MistralAI-User
action: DENY

View File

@@ -0,0 +1,10 @@
# Acts on behalf of user requests
# https://docs.mistral.ai/robots/
- name: mistral-mistralai-user
user_agent_regex: MistralAI-User/.+; \+https\://docs\.mistral\.ai/robots
action: ALLOW
# https://mistral.ai/mistralai-user-ips.json
remote_addresses: [
"20.240.160.161/32",
"20.240.160.1/32",
]

View File

@@ -0,0 +1,93 @@
# Acts on behalf of user requests
# https://platform.openai.com/docs/bots/overview-of-openai-crawlers
- name: openai-chatgpt-user
user_agent_regex: ChatGPT-User/.+; \+https\://openai\.com/bot
action: ALLOW
# https://openai.com/chatgpt-user.json
# curl 'https://openai.com/chatgpt-user.json' | jq '.prefixes.[].ipv4Prefix' | sed 's/$/,/'
remote_addresses: [
"13.65.138.112/28",
"23.98.179.16/28",
"13.65.138.96/28",
"172.183.222.128/28",
"20.102.212.144/28",
"40.116.73.208/28",
"172.183.143.224/28",
"52.190.190.16/28",
"13.83.237.176/28",
"51.8.155.64/28",
"74.249.86.176/28",
"51.8.155.48/28",
"20.55.229.144/28",
"135.237.131.208/28",
"135.237.133.48/28",
"51.8.155.112/28",
"135.237.133.112/28",
"52.159.249.96/28",
"52.190.137.16/28",
"52.255.111.112/28",
"40.84.181.32/28",
"172.178.141.112/28",
"52.190.142.64/28",
"172.178.140.144/28",
"52.190.137.144/28",
"172.178.141.128/28",
"57.154.187.32/28",
"4.196.118.112/28",
"20.193.50.32/28",
"20.215.188.192/28",
"20.215.214.16/28",
"4.197.22.112/28",
"4.197.115.112/28",
"172.213.21.16/28",
"172.213.11.144/28",
"172.213.12.112/28",
"172.213.21.144/28",
"20.90.7.144/28",
"57.154.175.0/28",
"57.154.174.112/28",
"52.236.94.144/28",
"137.135.191.176/28",
"23.98.186.192/28",
"23.98.186.96/28",
"23.98.186.176/28",
"23.98.186.64/28",
"68.221.67.192/28",
"68.221.67.160/28",
"13.83.167.128/28",
"20.228.106.176/28",
"52.159.227.32/28",
"68.220.57.64/28",
"172.213.21.112/28",
"68.221.67.224/28",
"68.221.75.16/28",
"20.97.189.96/28",
"52.252.113.240/28",
"52.230.163.32/28",
"172.212.159.64/28",
"52.255.111.80/28",
"52.255.111.0/28",
"4.151.241.240/28",
"52.255.111.32/28",
"52.255.111.48/28",
"52.255.111.16/28",
"52.230.164.176/28",
"52.176.139.176/28",
"52.173.234.16/28",
"4.151.71.176/28",
"4.151.119.48/28",
"52.255.109.112/28",
"52.255.109.80/28",
"20.161.75.208/28",
"68.154.28.96/28",
"52.255.109.128/28",
"52.225.75.208/28",
"52.190.139.48/28",
"68.221.67.240/28",
"52.156.77.144/28",
"52.148.129.32/28",
"40.84.221.208/28",
"104.210.139.224/28",
"40.84.221.224/28",
"104.210.139.192/28",
]

View File

@@ -0,0 +1,2 @@
- import: (data)/clients/small-internet-browsers/netsurf.yaml
- import: (data)/clients/small-internet-browsers/palemoon.yaml

View File

@@ -0,0 +1,5 @@
- name: "reduce-weight-netsurf"
user_agent_regex: "NetSurf"
action: WEIGH
weight:
adjust: -5

View File

@@ -0,0 +1,5 @@
- name: "reduce-weight-palemoon"
user_agent_regex: "PaleMoon"
action: WEIGH
weight:
adjust: -5

View File

@@ -0,0 +1,6 @@
# https://connect.mozilla.org/t5/firefox-labs/try-out-link-previews-in-firefox-labs-138-and-share-your/td-p/92012
- name: x-firefox-ai
action: WEIGH
expression: '"X-Firefox-Ai" in headers'
weight:
adjust: 5

View File

@@ -1,15 +1,15 @@
- name: ipv4-rfc-1918
action: ALLOW
remote_addresses:
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
- 100.64.0.0/10
- 10.0.0.0/8
- 172.16.0.0/12
- 192.168.0.0/16
- 100.64.0.0/10
- name: ipv6-ula
action: ALLOW
remote_addresses:
- fc00::/7
- fc00::/7
- name: ipv6-link-local
action: ALLOW
remote_addresses:
- fe80::/10
- fe80::/10

View File

@@ -1,4 +1,5 @@
- import: (data)/crawlers/googlebot.yaml
- import: (data)/crawlers/applebot.yaml
- import: (data)/crawlers/bingbot.yaml
- import: (data)/crawlers/duckduckbot.yaml
- import: (data)/crawlers/qwantbot.yaml

View File

@@ -0,0 +1,8 @@
# User agents that index exclusively for search in for AI systems.
# Each entry should have a positive/ALLOW entry created as well, with further documentation.
# Exceptions:
# - Claude-SearchBot: No published IP allowlist
- name: "ai-crawlers-search"
user_agent_regex: >-
OAI-SearchBot|Claude-SearchBot
action: DENY

View File

@@ -0,0 +1,8 @@
# User agents that crawl for training AI/LLM systems
# Each entry should have a positive/ALLOW entry created as well, with further documentation.
# Exceptions:
# - ClaudeBot: No published IP allowlist
- name: "ai-crawlers-training"
user_agent_regex: >-
GPTBot|ClaudeBot
action: DENY

View File

@@ -0,0 +1,20 @@
# Indexing for search and Siri
# https://support.apple.com/en-us/119829
- name: applebot
user_agent_regex: Applebot
action: ALLOW
# https://search.developer.apple.com/applebot.json
remote_addresses: [
"17.241.208.160/27",
"17.241.193.160/27",
"17.241.200.160/27",
"17.22.237.0/24",
"17.22.245.0/24",
"17.22.253.0/24",
"17.241.75.0/24",
"17.241.219.0/24",
"17.241.227.0/24",
"17.246.15.0/24",
"17.246.19.0/24",
"17.246.23.0/24",
]

View File

@@ -0,0 +1,16 @@
# Collects AI training data
# https://platform.openai.com/docs/bots/overview-of-openai-crawlers
- name: openai-gptbot
user_agent_regex: GPTBot/1\.1; \+https\://openai\.com/gptbot
action: ALLOW
# https://openai.com/gptbot.json
remote_addresses: [
"52.230.152.0/24",
"20.171.206.0/24",
"20.171.207.0/24",
"4.227.36.0/25",
"20.125.66.80/28",
"172.182.204.0/24",
"172.182.214.0/24",
"172.182.215.0/24",
]

View File

@@ -0,0 +1,13 @@
# Indexing for search, does not collect training data
# https://platform.openai.com/docs/bots/overview-of-openai-crawlers
- name: openai-searchbot
user_agent_regex: OAI-SearchBot/1\.0; \+https\://openai\.com/searchbot
action: ALLOW
# https://openai.com/searchbot.json
remote_addresses: [
"20.42.10.176/28",
"172.203.190.128/28",
"104.210.140.128/28",
"51.8.102.0/24",
"135.234.64.0/24"
]

View File

@@ -3,6 +3,6 @@ package data
import "embed"
var (
//go:embed botPolicies.yaml botPolicies.json all:apps all:bots all:clients all:common all:crawlers
//go:embed botPolicies.yaml botPolicies.json all:apps all:bots all:clients all:common all:crawlers all:meta
BotPolicies embed.FS
)

5
data/meta/README.md Normal file
View File

@@ -0,0 +1,5 @@
# meta policies
Contains policies that exclusively reference policies in _multiple_ other data folders.
Akin to "stances" that the administrator can take, with reference to various topics, such as AI/LLM systems.

View File

@@ -0,0 +1,6 @@
# Blocks all AI/LLM associated user agents, regardless of purpose or human agency
# Warning: To completely block some AI/LLM training, such as with Google, you _must_ place flags in robots.txt.
- import: (data)/bots/ai-catchall.yaml
- import: (data)/clients/ai.yaml
- import: (data)/crawlers/ai-search.yaml
- import: (data)/crawlers/ai-training.yaml

View File

@@ -0,0 +1,7 @@
# Blocks all AI/LLM bots used for training or unknown/undocumented purposes.
# Permits user agents with explicitly documented non-training use, and published IP allowlists.
- import: (data)/bots/ai-catchall.yaml
- import: (data)/crawlers/ai-training.yaml
- import: (data)/crawlers/openai-searchbot.yaml
- import: (data)/clients/openai-chatgpt-user.yaml
- import: (data)/clients/mistral-mistralai-user.yaml

View File

@@ -0,0 +1,6 @@
# Permits all well documented AI/LLM user agents with published IP allowlists.
- import: (data)/bots/ai-catchall.yaml
- import: (data)/crawlers/openai-searchbot.yaml
- import: (data)/crawlers/openai-gptbot.yaml
- import: (data)/clients/openai-chatgpt-user.yaml
- import: (data)/clients/mistral-mistralai-user.yaml

27
docker-bake.hcl Normal file
View File

@@ -0,0 +1,27 @@
variable "ALPINE_VERSION" { default = "3.22" }
variable "GITHUB_SHA" { default = "devel" }
group "default" {
targets = [
"anubis",
]
}
target "anubis" {
args = {
ALPINE_VERSION = "3.22"
}
context = "."
dockerfile = "./Dockerfile"
platforms = [
"linux/amd64",
"linux/arm64",
"linux/arm/v7",
"linux/ppc64le",
"linux/riscv64",
]
pull = true
tags = [
"ghcr.io/techarohq/anubis:${GITHUB_SHA}"
]
}

View File

@@ -11,27 +11,54 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
- Remove the unused `/test-error` endpoint and update the testing endpoint `/make-challenge` to only be enabled in
development
- Add `--xff-strip-private` flag/envvar to toggle skipping X-Forwarded-For private addresses or not
- Requests can have their weight be adjusted, if a request weighs zero or less than it is allowed through
- Refactor challenge presentation logic to use a challenge registry
- Allow challenge implementations to register HTTP routes
- Implement a no-JS challenge method: [`metarefresh`](./admin/configuration/challenges/metarefresh.mdx) ([#95](https://github.com/TecharoHQ/anubis/issues/95))
- Bump AI-robots.txt to version 1.34
- Make progress bar styling more compatible (UXP, etc)
## v1.19.1: Jenomis cen Lexentale - Echo 1
- Return `data/bots/ai-robots-txt.yaml` to avoid breaking configs [#599](https://github.com/TecharoHQ/anubis/issues/599)
## v1.19.0: Jenomis cen Lexentale
Mostly a bunch of small features, no big ticket things this time.
- Record if challenges were issued via the API or via embedded JSON in the challenge page HTML ([#531](https://github.com/TecharoHQ/anubis/issues/531))
- Ensure that clients that are shown a challenge support storing cookies
- Imprint the version number into challenge pages
- Encode challenge pages with gzip level 1
- Add PowerPC 64 bit little-endian builds (`GOARCH=ppc64le`)
- Add `check-spelling` for spell checking
- Add `--target-insecure-skip-verify` flag/envvar to allow Anubis to hit a self-signed HTTPS backend
- Minor adjustments to FreeBSD rc.d script to allow for more flexible configuration.
- Added Podman and Docker support for running Playwright tests
- Add a default rule to throw challenges when a request with the `X-Firefox-Ai` header is set
- Updated the nonce value in the challenge JWT cookie to be a string instead of a number
- Rename cookies in response to user feedback
- Ensure cookie renaming is consistent across configuration options
- Add Bookstack app in data
- Add `--target-host` flag/envvar to allow changing the value of the Host header in requests forwarded to the target service.
- Truncate everything but the first five characters of Accept-Language headers when making challenges
- Ensure client JavaScript is served with Content-Type text/javascript.
- Add `--target-host` flag/envvar to allow changing the value of the Host header in requests forwarded to the target service
- Bump AI-robots.txt to version 1.31
- Add `RuntimeDirectory` to systemd unit settings so native packages can listen over unix sockets
- Added SearXNG instance tracker whitelist policy
- Added Qualys SSL Labs whitelist policy
- Fixed cookie deletion logic ([#520](https://github.com/TecharoHQ/anubis/issues/520), [#522](https://github.com/TecharoHQ/anubis/pull/522))
- Add `--target-sni` flag/envvar to allow changing the value of the TLS handshake hostname in requests forwarded to the target service.
- Add `--target-sni` flag/envvar to allow changing the value of the TLS handshake hostname in requests forwarded to the target service
- Fixed CEL expression matching validator to now properly error out when it receives empty expressions
- Added OpenRC init.d script
- Added `--version` flag
- Added `anubis_proxied_requests_total` metric to count proxied requests
- Add `Applebot` as "good" web crawler
- Reorganize AI/LLM crawler blocking into three separate stances, maintaining existing status quo as default
- Split out AI/LLM user agent blocking policies, adding documentation for each
## v1.18.0: Varis zos Galvus

View File

@@ -0,0 +1,8 @@
{
"label": "Challenges",
"position": 10,
"link": {
"type": "generated-index",
"description": "The different challenge methods that Anubis supports."
}
}

View File

@@ -0,0 +1,19 @@
# Meta Refresh (No JavaScript)
The `metarefresh` challenge sends a browser a much simpler challenge that makes it refresh the page after a set period of time. This enables clients to pass challenges without executing JavaScript.
To use it in your Anubis configuration:
```yaml
# Generic catchall rule
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: CHALLENGE
challenge:
difficulty: 1 # Number of seconds to wait before refreshing the page
report_as: 4 # Unused by this challenge method
algorithm: metarefresh # Specify a non-JS challenge method
```
This is not enabled by default while this method is tested and its false positive rate is ascertained. Many modern scrapers use headless Google Chrome, so this will have a much higher false positive rate.

View File

@@ -0,0 +1,5 @@
# Proof of Work (JavaScript)
When Anubis is configured to use the `fast` or `slow` challenge methods, clients will be sent a small [proof of work](https://en.wikipedia.org/wiki/Proof_of_work) challenge. In order to get a token used to access the upstream resource, clients must calculate a complicated math puzzle with JavaScript.
A `fast` challenge uses a heavily optimized multithreaded implementation and a `slow` challenge uses a simplistic single-threaded implementation. The `slow` method is kept around for legacy compatibility.

View File

@@ -143,7 +143,29 @@ Anubis would return a challenge because all of those conditions are true.
## Functions exposed to Anubis expressions
There are currently no functions from the Anubis runtime exposed to expressions. This will change in the future.
Anubis expressions can be augmented with the following functions:
### `randInt`
```ts
function randInt(n: int): int;
```
randInt returns a randomly selected integer value in the range of `[0,n)`. This is a thin wrapper around [Go's math/rand#Intn](https://pkg.go.dev/math/rand#Intn). Be careful with this as it may cause inconsistent behavior for genuine users.
This is best applied when doing explicit block rules, eg:
```yaml
# Denies LightPanda about 75% of the time on average
- name: deny-lightpanda-sometimes
action: DENY
expression:
all:
- userAgent.matches("LightPanda")
- randInt(16) >= 4
```
It seems counter-intuitive to allow known bad clients through sometimes, but this allows you to confuse attackers by making Anubis' behavior random. Adjust the thresholds and numbers as facts and circumstances demand.
## Life advice

View File

@@ -14,7 +14,7 @@ EG:
{
"bots": [
{
"import": "(data)/bots/ai-robots-txt.yaml"
"import": "(data)/bots/ai-catchall.yaml"
},
{
"import": "(data)/bots/cloudflare-workers.yaml"
@@ -29,8 +29,8 @@ EG:
```yaml
bots:
# Pathological bots to deny
- # This correlates to data/bots/ai-robots-txt.yaml in the source tree
import: (data)/bots/ai-robots-txt.yaml
- # This correlates to data/bots/ai-catchall.yaml in the source tree
import: (data)/bots/ai-catchall.yaml
- import: (data)/bots/cloudflare-workers.yaml
```
@@ -46,7 +46,7 @@ Of note, a bot rule can either have inline bot configuration or import a bot con
{
"bots": [
{
"import": "(data)/bots/ai-robots-txt.yaml",
"import": "(data)/bots/ai-catchall.yaml",
"name": "generic-browser",
"user_agent_regex": "Mozilla|Opera\n",
"action": "CHALLENGE"
@@ -60,7 +60,7 @@ Of note, a bot rule can either have inline bot configuration or import a bot con
```yaml
bots:
- import: (data)/bots/ai-robots-txt.yaml
- import: (data)/bots/ai-catchall.yaml
name: generic-browser
user_agent_regex: >
Mozilla|Opera
@@ -167,7 +167,7 @@ static
├── botPolicies.json
├── botPolicies.yaml
├── bots
│ ├── ai-robots-txt.yaml
│ ├── ai-catchall.yaml
│ ├── cloudflare-workers.yaml
│ ├── headless-browsers.yaml
│ └── us-ai-scraper.yaml

View File

@@ -10,6 +10,20 @@ Anubis can act in one of two modes:
1. Reverse proxy (the default): Anubis sits in the middle of all traffic and then will reverse proxy it to its destination. This is the moral equivalent of a middleware in your favorite web framework.
2. Subrequest authentication mode: Anubis listens for requests and if they don't pass muster then they are forwarded to Anubis for challenge processing. This is the equivalent of Anubis being a sidecar service.
:::note
Subrequest authentication requires changing the default policy because nginx interprets the default `DENY` status code `200` as successful authentication and allows the request.
```yaml
status_codes:
CHALLENGE: 200
DENY: 403
```
[See policy definitions](../policies.mdx).
:::
## Nginx
Anubis can perform [subrequest authentication](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-subrequest-authentication/) with the `auth_request` module in Nginx. In order to set this up, keep the following things in mind:

View File

@@ -92,6 +92,7 @@ Assuming you are protecting `anubistest.techaro.lol`, you need the following ser
# throw an "admin misconfiguration" error.
RequestHeader set "X-Real-Ip" expr=%{REMOTE_ADDR}
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set "X-Http-Version" "%{SERVER_PROTOCOL}s"
ProxyPreserveHost On
@@ -119,6 +120,14 @@ Make sure to add a separate configuration file for the listener on port 3001:
```text
# /etc/httpd/conf.d/listener-3001.conf
Listen [::1]:3001
```
In case you are running an IPv4-only system, use the following configuration instead:
```text
# /etc/httpd/conf.d/listener-3001.conf
Listen 127.0.0.1:3001
```

View File

@@ -61,6 +61,7 @@ server {
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Http-Version $server_protocol;
proxy_pass http://anubis;
}

View File

@@ -3,11 +3,10 @@ id: traefik
title: Traefik
---
:::note
This only talks about integration through Compose,
but it also applies to docker cli options.
This only talks about integration through Compose,
but it also applies to docker cli options.
:::

View File

@@ -49,29 +49,30 @@ For more detailed information on installing Anubis with native packages, please
Anubis uses these environment variables for configuration:
| Environment Variable | Default value | Explanation |
| :----------------------------- | :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `BASE_PREFIX` | unset | If set, adds a global prefix to all Anubis endpoints. For example, setting this to `/myapp` would make Anubis accessible at `/myapp/` instead of `/`. This is useful when running Anubis behind a reverse proxy that routes based on path prefixes. |
| `BIND` | `:8923` | The network address that Anubis listens on. For `unix`, set this to a path: `/run/anubis/instance.sock` |
| `BIND_NETWORK` | `tcp` | The address family that Anubis listens on. Accepts `tcp`, `unix` and anything Go's [`net.Listen`](https://pkg.go.dev/net#Listen) supports. |
| `COOKIE_DOMAIN` | unset | The domain the Anubis challenge pass cookie should be set to. This should be set to the domain you bought from your registrar (EG: `techaro.lol` if your webapp is running on `anubis.techaro.lol`). See this [stackoverflow explanation of cookies](https://stackoverflow.com/a/1063760) for more information.<br/><br/>Note that unlike `REDIRECT_DOMAINS`, you should never include a port number in this variable. |
| `COOKIE_EXPIRATION_TIME` | `168h` | The amount of time the authorization cookie is valid for. |
| `COOKIE_PARTITIONED` | `false` | If set to `true`, enables the [partitioned (CHIPS) flag](https://developers.google.com/privacy-sandbox/cookies/chips), meaning that Anubis inside an iframe has a different set of cookies than the domain hosting the iframe. |
| `DIFFICULTY` | `4` | The difficulty of the challenge, or the number of leading zeroes that must be in successful responses. |
| `ED25519_PRIVATE_KEY_HEX` | unset | The hex-encoded ed25519 private key used to sign Anubis responses. If this is not set, Anubis will generate one for you. This should be exactly 64 characters long. See below for details. |
| `ED25519_PRIVATE_KEY_HEX_FILE` | unset | Path to a file containing the hex-encoded ed25519 private key. Only one of this or its sister option may be set. |
| `METRICS_BIND` | `:9090` | The network address that Anubis serves Prometheus metrics on. See `BIND` for more information. |
| `METRICS_BIND_NETWORK` | `tcp` | The address family that the Anubis metrics server listens on. See `BIND_NETWORK` for more information. |
| `OG_EXPIRY_TIME` | `24h` | The expiration time for the Open Graph tag cache. |
| `OG_PASSTHROUGH` | `false` | If set to `true`, Anubis will enable Open Graph tag passthrough. |
| `OG_CACHE_CONSIDER_HOST` | `false` | If set to `true`, Anubis will consider the host in the Open Graph tag cache key. |
| `POLICY_FNAME` | unset | The file containing [bot policy configuration](./policies.mdx). See the bot policy documentation for more details. If unset, the default bot policy configuration is used. |
| Environment Variable | Default value | Explanation |
| :----------------------------- | :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `BASE_PREFIX` | unset | If set, adds a global prefix to all Anubis endpoints. For example, setting this to `/myapp` would make Anubis accessible at `/myapp/` instead of `/`. This is useful when running Anubis behind a reverse proxy that routes based on path prefixes. |
| `BIND` | `:8923` | The network address that Anubis listens on. For `unix`, set this to a path: `/run/anubis/instance.sock` |
| `BIND_NETWORK` | `tcp` | The address family that Anubis listens on. Accepts `tcp`, `unix` and anything Go's [`net.Listen`](https://pkg.go.dev/net#Listen) supports. |
| `COOKIE_DOMAIN` | unset | The domain the Anubis challenge pass cookie should be set to. This should be set to the domain you bought from your registrar (EG: `techaro.lol` if your webapp is running on `anubis.techaro.lol`). See this [stackoverflow explanation of cookies](https://stackoverflow.com/a/1063760) for more information.<br/><br/>Note that unlike `REDIRECT_DOMAINS`, you should never include a port number in this variable. |
| `COOKIE_EXPIRATION_TIME` | `168h` | The amount of time the authorization cookie is valid for. |
| `COOKIE_PARTITIONED` | `false` | If set to `true`, enables the [partitioned (CHIPS) flag](https://developers.google.com/privacy-sandbox/cookies/chips), meaning that Anubis inside an iframe has a different set of cookies than the domain hosting the iframe. |
| `DIFFICULTY` | `4` | The difficulty of the challenge, or the number of leading zeroes that must be in successful responses. |
| `ED25519_PRIVATE_KEY_HEX` | unset | The hex-encoded ed25519 private key used to sign Anubis responses. If this is not set, Anubis will generate one for you. This should be exactly 64 characters long. See below for details. |
| `ED25519_PRIVATE_KEY_HEX_FILE` | unset | Path to a file containing the hex-encoded ed25519 private key. Only one of this or its sister option may be set. |
| `METRICS_BIND` | `:9090` | The network address that Anubis serves Prometheus metrics on. See `BIND` for more information. |
| `METRICS_BIND_NETWORK` | `tcp` | The address family that the Anubis metrics server listens on. See `BIND_NETWORK` for more information. |
| `OG_EXPIRY_TIME` | `24h` | The expiration time for the Open Graph tag cache. |
| `OG_PASSTHROUGH` | `false` | If set to `true`, Anubis will enable Open Graph tag passthrough. |
| `OG_CACHE_CONSIDER_HOST` | `false` | If set to `true`, Anubis will consider the host in the Open Graph tag cache key. |
| `POLICY_FNAME` | unset | The file containing [bot policy configuration](./policies.mdx). See the bot policy documentation for more details. If unset, the default bot policy configuration is used. |
| `REDIRECT_DOMAINS` | unset | If set, restrict the domains that Anubis can redirect to when passing a challenge.<br/><br/>If this is unset, Anubis may redirect to any domain which could cause security issues in the unlikely case that an attacker passes a challenge for your browser and then tricks you into clicking a link to your domain.<br/><br/>Note that if you are hosting Anubis on a non-standard port (`https://example:com:8443`, `http://www.example.net:8080`, etc.), you must also include the port number here. |
| `SERVE_ROBOTS_TXT` | `false` | If set `true`, Anubis will serve a default `robots.txt` file that disallows all known AI scrapers by name and then additionally disallows every scraper. This is useful if facts and circumstances make it difficult to change the underlying service to serve such a `robots.txt` file. |
| `SOCKET_MODE` | `0770` | _Only used when at least one of the `*_BIND_NETWORK` variables are set to `unix`._ The socket mode (permissions) for Unix domain sockets. |
| `TARGET` | `http://localhost:3923` | The URL of the service that Anubis should forward valid requests to. Supports Unix domain sockets, set this to a URI like so: `unix:///path/to/socket.sock`. |
| `USE_REMOTE_ADDRESS` | unset | If set to `true`, Anubis will take the client's IP from the network socket. For production deployments, it is expected that a reverse proxy is used in front of Anubis, which pass the IP using headers, instead. |
| `WEBMASTER_EMAIL` | unset | If set, shows a contact email address when rendering error pages. This email address will be how users can get in contact with administrators. |
| `SERVE_ROBOTS_TXT` | `false` | If set `true`, Anubis will serve a default `robots.txt` file that disallows all known AI scrapers by name and then additionally disallows every scraper. This is useful if facts and circumstances make it difficult to change the underlying service to serve such a `robots.txt` file. |
| `SOCKET_MODE` | `0770` | _Only used when at least one of the `*_BIND_NETWORK` variables are set to `unix`._ The socket mode (permissions) for Unix domain sockets. |
| `TARGET` | `http://localhost:3923` | The URL of the service that Anubis should forward valid requests to. Supports Unix domain sockets, set this to a URI like so: `unix:///path/to/socket.sock`. |
| `USE_REMOTE_ADDRESS` | unset | If set to `true`, Anubis will take the client's IP from the network socket. For production deployments, it is expected that a reverse proxy is used in front of Anubis, which pass the IP using headers, instead. |
| `WEBMASTER_EMAIL` | unset | If set, shows a contact email address when rendering error pages. This email address will be how users can get in contact with administrators. |
| `XFF_STRIP_PRIVATE` | `true` | If set, strip private addresses from `X-Forwarded-For` headers. To unset this, you must set `XFF_STRIP_PRIVATE=false` or `--xff-strip-private=false`. |
<details>
<summary>Advanced configuration settings</summary>

View File

@@ -244,3 +244,39 @@ In case your service needs it for risk calculation reasons, Anubis exposes infor
| `X-Anubis-Status` | The status and how strict Anubis was in its checks | `PASS` |
Policy rules are matched using [Go's standard library regular expressions package](https://pkg.go.dev/regexp). You can mess around with the syntax at [regex101.com](https://regex101.com), make sure to select the Golang option.
## Request Weight
Anubis rules can also add or remove "weight" from requests, allowing administrators to configure custom levels of suspicion. For example, if your application uses session tokens named `i_love_gitea`:
```yaml
- name: gitea-session-token
action: WEIGH
expression:
all:
- '"Cookie" in headers'
- headers["Cookie"].contains("i_love_gitea=")
# Remove 5 weight points
weight:
adjust: -5
```
This would remove five weight points from the request, making Anubis present the [Meta Refresh challenge](./configuration/challenges/metarefresh.mdx).
### Weight Thresholds
Weight thresholds and challenge associations will be configurable with CEL expressions in the configuration file in an upcoming patch, for now here's how Anubis configures the weight thresholds:
| Weight Expression | Action |
| -----------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------- |
| `weight < 0` (weight is less than 0) | Allow the request through. |
| `weight < 10` (weight is less than 10) | Challenge the client with the [Meta Refresh challenge](./configuration/challenges/metarefresh.mdx) at the default difficulty level. |
| `weight >= 10` (weight is greater than or equal to 10) | Challenge the client with the [Proof of Work challenge](./configuration/challenges/proof-of-work.mdx) at the default difficulty level. |
### Advice
Weight is still very new and needs work. This is an experimental feature and should be treated as such. Here's some advice to help you better tune requests:
- The default weight for browser-like clients is 10. This triggers an aggressive challenge.
- Remove and add weight in multiples of five.
- Be careful with how you configure weight.

View File

@@ -91,7 +91,7 @@ work valid?"}
## Proof of passing challenges
When a client passes a challenge, Anubis sets an HTTP cookie named `"within.website-x-cmd-anubis-auth"` containing a signed [JWT](https://jwt.io/) (JSON Web Token). This JWT contains the following claims:
When a client passes a challenge, Anubis sets an HTTP cookie named `"techaro.lol-anubis-auth"` containing a signed [JWT](https://jwt.io/) (JSON Web Token). This JWT contains the following claims:
- `challenge`: The challenge string derived from user request metadata
- `nonce`: The nonce / iteration number used to generate the passing response

View File

@@ -19,14 +19,48 @@ title: Anubis
Anubis is brought to you by sponsors and donors like:
[![Distrust](/img/sponsors/distrust-logo.webp)](https://distrust.co?utm_campaign=github&utm_medium=referral&utm_content=anubis)
[![Terminal Trove](/img/sponsors/terminal-trove.webp)](https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=anubis&utm_source=abgh)
[![canine.tools](/img/sponsors/caninetools-logo.webp)](https://canine.tools?utm_campaign=github&utm_medium=referral&utm_content=anubis)
[![Weblate](/img/sponsors/weblate-logo.webp)](https://weblate.org/?utm_campaign=github&utm_medium=referral&utm_content=anubis)
### Diamond Tier
<a href="https://www.raptorcs.com/content/base/products.html">
<img
src="/img/sponsors/raptor-computing-logo.webp"
alt="Raptor Computing Systems"
height="64"
/>
</a>
### Gold Tier
<a href="https://distrust.co?utm_campaign=github&utm_medium=referral&utm_content=anubis">
<img src="/img/sponsors/distrust-logo.webp" alt="Distrust" height="64" />
</a>
<a href="https://terminaltrove.com/?utm_campaign=github&utm_medium=referral&utm_content=anubis&utm_source=abgh">
<img
src="/img/sponsors/terminal-trove.webp"
alt="Terminal Trove"
height="64"
/>
</a>
<a href="https://canine.tools?utm_campaign=github&utm_medium=referral&utm_content=anubis">
<img
src="/img/sponsors/caninetools-logo.webp"
alt="canine.tools"
height="64"
/>
</a>
<a href="https://weblate.org/">
<img src="/img/sponsors/weblate-logo.webp" alt="Weblate" height="64" />
</a>
<a href="https://uberspace.de/">
<img src="/img/sponsors/uberspace-logo.webp" alt="Uberspace" height="64" />
</a>
<a href="https://wildbase.xyz/">
<img src="/img/sponsors/wildbase-logo.webp" alt="Wildbase" height="64" />
</a>
## Overview
Anubis [weighs the soul of your connection](https://en.wikipedia.org/wiki/Weighing_of_souls) using a proof-of-work challenge in order to protect upstream resources from scraper bots.
Anubis is a Web AI Firewall Utility that [weighs the soul of your connection](https://en.wikipedia.org/wiki/Weighing_of_souls) using one or more challenges in order to protect upstream resources from scraper bots.
This program is designed to help protect the small internet from the endless storm of requests that flood in from AI companies. Anubis is as lightweight as possible to ensure that everyone can afford to protect the communities closest to them.

View File

@@ -18,3 +18,11 @@ Anubis uses [Web Workers](https://developer.mozilla.org/en-US/docs/Web/API/Web_W
2. Web Workers allow you to do multithreaded execution of JavaScript code. This lets Anubis run its checks in parallel across all your system cores so that the challenge can complete as fast as possible. In the last decade, most CPU advancements have come from making cores and code extremely parallel. Using Web Workers lets Anubis take advantage of your hardware as much as possible so that the challenge finishes as fast as possible.
If you use a browser extension such as [JShelter](https://jshelter.org/), you will need to [modify your JShelter configuration](./known-broken-extensions.md#jshelter) to allow Anubis' proof of work computation to complete.
## Does Anubis mine Bitcoin?
No. Anubis does not mine Bitcoin.
In order to mine bitcoin, you need to download a copy of the blockchain (so you have the state required to do mining) and also broadcast your mined blocks to the network should you reach a hash with the right number of leading zeroes. You also need to continuously read for newly broadcasted transactions so you can batch them into a block. This requires gigabytes of data to be transferred from the server to the client.
Anubis transfers two digit numbers of kilobytes from the server to the client (which you can independently verify with your browser's Developer Tools feature). This is orders of magnitude below what is required to mine Bitcoin.

View File

@@ -35,6 +35,12 @@ This page contains a non-exhaustive list with all websites using Anubis.
- https://hofstede.io/
- https://www.indiemag.fr/
- https://reddit.nerdvpn.de/
- https://hosted.weblate.org/
- https://gitea.com/
- https://openwrt.org/
- https://minihoot.site
- https://catgirl.click/
- https://wiki.dolphin-emu.org/
- <details>
<summary>FreeCAD</summary>
- https://forum.freecad.org/
@@ -58,3 +64,10 @@ This page contains a non-exhaustive list with all websites using Anubis.
<summary>The United Nations</summary>
- https://policytoolbox.iiep.unesco.org/
</details>
- <details>
<summary>hebis (Alliance of Hessian Libraries)</summary>
- https://ubmr.hds.hebis.de/
- https://tufind.hds.hebis.de/
- https://karla.hds.hebis.de/
- and many more (see https://www.hebis.de/dienste/hebis-discovery-system/)
</details>

View File

@@ -0,0 +1,72 @@
## Anubis has the ability to let you import snippets of configuration into the main
## configuration file. This allows you to break up your config into smaller parts
## that get logically assembled into one big file.
##
## Of note, a bot rule can either have inline bot configuration or import a
## bot config snippet. You cannot do both in a single bot rule.
##
## Import paths can either be prefixed with (data) to import from the common/shared
## rules in the data folder in the Anubis source tree or will point to absolute/relative
## paths in your filesystem. If you don't have access to the Anubis source tree, check
## /usr/share/docs/anubis/data or in the tarball you extracted Anubis from.
bots:
# Pathological bots to deny
- # This correlates to data/bots/deny-pathological.yaml in the source tree
# https://github.com/TecharoHQ/anubis/blob/main/data/bots/deny-pathological.yaml
import: (data)/bots/_deny-pathological.yaml
- import: (data)/bots/aggressive-brazilian-scrapers.yaml
# Aggressively block AI/LLM related bots/agents by default
- import: (data)/meta/ai-block-aggressive.yaml
# Consider replacing the aggressive AI policy with more selective policies:
# - import: (data)/meta/ai-block-moderate.yaml
# - import: (data)/meta/ai-block-permissive.yaml
# Search engine crawlers to allow, defaults to:
# - Google (so they don't try to bypass Anubis)
# - Apple
# - Bing
# - DuckDuckGo
# - Qwant
# - The Internet Archive
# - Kagi
# - Marginalia
# - Mojeek
- import: (data)/crawlers/_allow-good.yaml
# Challenge Firefox AI previews
- import: (data)/clients/x-firefox-ai.yaml
# Allow common "keeping the internet working" routes (well-known, favicon, robots.txt)
- import: (data)/common/keep-internet-working.yaml
# # Punish any bot with "bot" in the user-agent string
# # This is known to have a high false-positive rate, use at your own risk
# - name: generic-bot-catchall
# user_agent_regex: (?i:bot|crawler)
# action: CHALLENGE
# challenge:
# difficulty: 16 # impossible
# report_as: 4 # lie to the operator
# algorithm: slow # intentionally waste CPU cycles and time
# Generic catchall rule
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: CHALLENGE
challenge:
difficulty: 1 # Number of seconds to wait before refreshing the page
report_as: 4 # Unused by this challenge method
algorithm: metarefresh # Specify a non-JS challenge method
dnsbl: false
# By default, send HTTP 200 back to clients that either get issued a challenge
# or a denial. This seems weird, but this is load-bearing due to the fact that
# the most aggressive scraper bots seem to really, really, want an HTTP 200 and
# will stop sending requests once they get it.
status_codes:
CHALLENGE: 200
DENY: 200

View File

@@ -11,48 +11,58 @@ spec:
labels:
app: anubis-docs
spec:
volumes:
- name: anubis
configMap:
name: anubis-cfg
containers:
- name: anubis-docs
image: ghcr.io/techarohq/anubis/docs:main
imagePullPolicy: Always
resources:
limits:
memory: "128Mi"
cpu: "500m"
ports:
- containerPort: 80
- name: anubis
image: ghcr.io/techarohq/anubis:main
imagePullPolicy: Always
env:
- name: "BIND"
value: ":8081"
- name: "DIFFICULTY"
value: "4"
- name: "METRICS_BIND"
value: ":9090"
- name: "POLICY_FNAME"
value: ""
- name: "SERVE_ROBOTS_TXT"
value: "false"
- name: "TARGET"
value: "http://localhost:80"
# - name: "SLOG_LEVEL"
# value: "debug"
resources:
limits:
cpu: 500m
memory: 128Mi
requests:
cpu: 250m
memory: 128Mi
securityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault
- name: anubis-docs
image: ghcr.io/techarohq/anubis/docs:main
imagePullPolicy: Always
resources:
limits:
memory: "128Mi"
cpu: "500m"
requests:
cpu: 250m
memory: 128Mi
ports:
- containerPort: 80
- name: anubis
image: ghcr.io/techarohq/anubis:main
imagePullPolicy: Always
env:
- name: "BIND"
value: ":8081"
- name: "DIFFICULTY"
value: "4"
- name: "METRICS_BIND"
value: ":9090"
- name: "POLICY_FNAME"
value: "/xe/cfg/anubis/botPolicies.yaml"
- name: "SERVE_ROBOTS_TXT"
value: "false"
- name: "TARGET"
value: "http://localhost:80"
# - name: "SLOG_LEVEL"
# value: "debug"
volumeMounts:
- name: anubis
mountPath: /xe/cfg/anubis
resources:
limits:
cpu: 500m
memory: 128Mi
requests:
cpu: 250m
memory: 128Mi
securityContext:
runAsUser: 1000
runAsGroup: 1000
runAsNonRoot: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
seccompProfile:
type: RuntimeDefault

View File

@@ -2,4 +2,10 @@ resources:
- deployment.yaml
- ingress.yaml
- onionservice.yaml
- service.yaml
- service.yaml
configMapGenerator:
- name: anubis-cfg
behavior: create
files:
- ./cfg/anubis/botPolicies.yaml

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

30
go.mod
View File

@@ -3,7 +3,7 @@ module github.com/TecharoHQ/anubis
go 1.24.2
require (
github.com/a-h/templ v0.3.865
github.com/a-h/templ v0.3.898
github.com/facebookgo/flagenv v0.0.0-20160425205200-fcd59fca7456
github.com/golang-jwt/jwt/v5 v5.2.2
github.com/google/cel-go v0.25.0
@@ -11,23 +11,23 @@ require (
github.com/prometheus/client_golang v1.22.0
github.com/sebest/xff v0.0.0-20210106013422-671bd2870b3a
github.com/yl2chen/cidranger v1.0.2
golang.org/x/net v0.40.0
k8s.io/apimachinery v0.33.0
golang.org/x/net v0.41.0
k8s.io/apimachinery v0.33.1
)
require (
al.essio.dev/pkg/shellescape v1.6.0 // indirect
cel.dev/expr v0.23.1 // indirect
dario.cat/mergo v1.0.1 // indirect
dario.cat/mergo v1.0.2 // indirect
github.com/AlekSi/pointer v1.2.0 // indirect
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.3.1 // indirect
github.com/Masterminds/sprig/v3 v3.3.0 // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/ProtonMail/go-crypto v1.1.6 // indirect
github.com/ProtonMail/go-crypto v1.2.0 // indirect
github.com/Songmu/gitconfig v0.2.0 // indirect
github.com/TecharoHQ/yeet v0.2.3 // indirect
github.com/TecharoHQ/yeet v0.6.0 // indirect
github.com/a-h/parse v0.0.0-20250122154542-74294addb73e // indirect
github.com/andybalholm/brotli v1.1.0 // indirect
github.com/antlr4-go/antlr/v4 v4.13.0 // indirect
@@ -40,7 +40,7 @@ require (
github.com/cli/go-gh v0.1.0 // indirect
github.com/cloudflare/circl v1.6.0 // indirect
github.com/cyphar/filepath-securejoin v0.4.1 // indirect
github.com/deckarep/golang-set/v2 v2.7.0 // indirect
github.com/deckarep/golang-set/v2 v2.8.0 // indirect
github.com/dlclark/regexp2 v1.11.4 // indirect
github.com/dop251/goja v0.0.0-20250309171923-bcd7cc6bf64c // indirect
github.com/emirpasic/gods v1.18.1 // indirect
@@ -59,11 +59,11 @@ require (
github.com/goccy/go-yaml v1.12.0 // indirect
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/google/pprof v0.0.0-20230207041349-798e818bf904 // indirect
github.com/google/rpmpack v0.6.1-0.20240329070804-c2247cbb881a // indirect
github.com/google/rpmpack v0.6.1-0.20250405124433-758cc6896cbc // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/goreleaser/chglog v0.7.0 // indirect
github.com/goreleaser/fileglob v1.3.0 // indirect
github.com/goreleaser/nfpm/v2 v2.42.0 // indirect
github.com/goreleaser/nfpm/v2 v2.42.1 // indirect
github.com/huandu/xstrings v1.5.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
@@ -88,15 +88,16 @@ require (
github.com/ulikunitz/xz v0.5.12 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
gitlab.com/digitalxero/go-conventional-commit v1.0.7 // indirect
golang.org/x/crypto v0.38.0 // indirect
golang.org/x/crypto v0.39.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 // indirect
golang.org/x/mod v0.24.0 // indirect
golang.org/x/sync v0.14.0 // indirect
golang.org/x/mod v0.25.0 // indirect
golang.org/x/sync v0.15.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/telemetry v0.0.0-20240522233618-39ace7a40ae7 // indirect
golang.org/x/text v0.25.0 // indirect
golang.org/x/tools v0.32.0 // indirect
golang.org/x/term v0.32.0 // indirect
golang.org/x/text v0.26.0 // indirect
golang.org/x/tools v0.33.0 // indirect
golang.org/x/vuln v1.1.4 // indirect
golang.org/x/xerrors v0.0.0-20240716161551-93cc26a95ae9 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20240826202546-f6391c0de4c7 // indirect
@@ -105,6 +106,7 @@ require (
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
honnef.co/go/tools v0.6.1 // indirect
mvdan.cc/sh/v3 v3.11.0 // indirect
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 // indirect
sigs.k8s.io/yaml v1.4.0 // indirect
)

62
go.sum
View File

@@ -2,8 +2,8 @@ al.essio.dev/pkg/shellescape v1.6.0 h1:NxFcEqzFSEVCGN2yq7Huv/9hyCEGVa/TncnOOBBeX
al.essio.dev/pkg/shellescape v1.6.0/go.mod h1:6sIqp7X2P6mThCQ7twERpZTuigpr6KbZWtls1U8I890=
cel.dev/expr v0.23.1 h1:K4KOtPCJQjVggkARsjG9RWXP6O4R73aHeJMa/dmCQQg=
cel.dev/expr v0.23.1/go.mod h1:hLPLo1W4QUmuYdA72RBX06QTs6MXw941piREPl3Yfiw=
dario.cat/mergo v1.0.1 h1:Ra4+bf83h2ztPIQYNP99R6m+Y7KfnARDfID+a+vLl4s=
dario.cat/mergo v1.0.1/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
github.com/AlekSi/pointer v1.2.0 h1:glcy/gc4h8HnG2Z3ZECSzZ1IX1x2JxRVuDzaJwQE0+w=
github.com/AlekSi/pointer v1.2.0/go.mod h1:gZGfd3dpW4vEc/UlyfKKi1roIqcCgwOIvb0tSNSBle0=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=
@@ -20,20 +20,20 @@ github.com/Masterminds/sprig/v3 v3.3.0/go.mod h1:Zy1iXRYNqNLUolqCpL4uhk6SHUMAOSC
github.com/Microsoft/go-winio v0.5.2/go.mod h1:WpS1mjBmmwHBEWmogvA2mj8546UReBk4v8QkMxJ6pZY=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/ProtonMail/go-crypto v1.1.6 h1:ZcV+Ropw6Qn0AX9brlQLAUXfqLBc7Bl+f/DmNxpLfdw=
github.com/ProtonMail/go-crypto v1.1.6/go.mod h1:rA3QumHc/FZ8pAHreoekgiAbzpNsfQAosU5td4SnOrE=
github.com/ProtonMail/go-crypto v1.2.0 h1:+PhXXn4SPGd+qk76TlEePBfOfivE0zkWFenhGhFLzWs=
github.com/ProtonMail/go-crypto v1.2.0/go.mod h1:9whxjD8Rbs29b4XWbB8irEcE8KHMqaR2e7GWU1R+/PE=
github.com/ProtonMail/go-mime v0.0.0-20230322103455-7d82a3887f2f h1:tCbYj7/299ekTTXpdwKYF8eBlsYsDVoggDAuAjoK66k=
github.com/ProtonMail/go-mime v0.0.0-20230322103455-7d82a3887f2f/go.mod h1:gcr0kNtGBqin9zDW9GOHcVntrwnjrK+qdJ06mWYBybw=
github.com/ProtonMail/gopenpgp/v2 v2.7.1 h1:Awsg7MPc2gD3I7IFac2qE3Gdls0lZW8SzrFZ3k1oz0s=
github.com/ProtonMail/gopenpgp/v2 v2.7.1/go.mod h1:/BU5gfAVwqyd8EfC3Eu7zmuhwYQpKs+cGD8M//iiaxs=
github.com/Songmu/gitconfig v0.2.0 h1:pX2++u4KUq+K2k/ZCzGXLtkD3ceCqIdi0tDyb+IbSyo=
github.com/Songmu/gitconfig v0.2.0/go.mod h1:cB5bYJer+pl7W8g6RHFwL/0X6aJROVrYuHlvc7PT+hE=
github.com/TecharoHQ/yeet v0.2.3 h1:Pcsnq5HTnk4Xntlu/FNEidH7x55bIx+f5Mk1hpVIngs=
github.com/TecharoHQ/yeet v0.2.3/go.mod h1:avLiwxZpNY37A/o35XledvdmGnTkm3G7+Oskxca6Z7Y=
github.com/TecharoHQ/yeet v0.6.0 h1:RCBAjr7wIlllsgy0tpvWpLX7jsZgu2tiuBY3RrprcR0=
github.com/TecharoHQ/yeet v0.6.0/go.mod h1:bj2V4Fg8qKQXoiuPZa3HuawrE8g+LsOQv/9q2WyGSsA=
github.com/a-h/parse v0.0.0-20250122154542-74294addb73e h1:HjVbSQHy+dnlS6C3XajZ69NYAb5jbGNfHanvm1+iYlo=
github.com/a-h/parse v0.0.0-20250122154542-74294addb73e/go.mod h1:3mnrkvGpurZ4ZrTDbYU84xhwXW2TjTKShSwjRi2ihfQ=
github.com/a-h/templ v0.3.865 h1:nYn5EWm9EiXaDgWcMQaKiKvrydqgxDUtT1+4zU2C43A=
github.com/a-h/templ v0.3.865/go.mod h1:oLBbZVQ6//Q6zpvSMPTuBK0F3qOtBdFBcGRspcT+VNQ=
github.com/a-h/templ v0.3.898 h1:g9oxL/dmM6tvwRe2egJS8hBDQTncokbMoOFk1oJMX7s=
github.com/a-h/templ v0.3.898/go.mod h1:oLBbZVQ6//Q6zpvSMPTuBK0F3qOtBdFBcGRspcT+VNQ=
github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
@@ -65,14 +65,16 @@ github.com/cli/safeexec v1.0.0/go.mod h1:Z/D4tTN8Vs5gXYHDCbaM1S/anmEDnJb1iW0+EJ5
github.com/cli/shurcooL-graphql v0.0.1/go.mod h1:U7gCSuMZP/Qy7kbqkk5PrqXEeDgtfG5K+W+u8weorps=
github.com/cloudflare/circl v1.6.0 h1:cr5JKic4HI+LkINy2lg3W2jF8sHCVTBncJr5gIIq7qk=
github.com/cloudflare/circl v1.6.0/go.mod h1:uddAzsPgqdMAYatqJ0lsjX1oECcQLIlRpzZh3pJrofs=
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
github.com/cyphar/filepath-securejoin v0.4.1 h1:JyxxyPEaktOD+GAnqIqTf9A8tHyAG22rowi7HkoSU1s=
github.com/cyphar/filepath-securejoin v0.4.1/go.mod h1:Sdj7gXlvMcPZsbhwhQ33GguGLDGQL7h7bg04C/+u9jI=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/deckarep/golang-set/v2 v2.7.0 h1:gIloKvD7yH2oip4VLhsv3JyLLFnC0Y2mlusgcvJYW5k=
github.com/deckarep/golang-set/v2 v2.7.0/go.mod h1:VAky9rY/yGXJOLEDv3OMci+7wtDpOF4IN+y82NBOac4=
github.com/deckarep/golang-set/v2 v2.8.0 h1:swm0rlPCmdWn9mESxKOjWk8hXSqoxOp+ZlfuyaAdFlQ=
github.com/deckarep/golang-set/v2 v2.8.0/go.mod h1:VAky9rY/yGXJOLEDv3OMci+7wtDpOF4IN+y82NBOac4=
github.com/dlclark/regexp2 v1.11.4 h1:rPYF9/LECdNymJufQKmri9gV604RvvABwgOA8un7yAo=
github.com/dlclark/regexp2 v1.11.4/go.mod h1:DHkYz0B9wPfa6wondMfaivmHpzrQ3v9q8cnmRbL6yW8=
github.com/dop251/goja v0.0.0-20250309171923-bcd7cc6bf64c h1:mxWGS0YyquJ/ikZOjSrRjjFIbUqIP9ojyYQ+QZTU3Rg=
@@ -117,6 +119,8 @@ github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+
github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
github.com/go-playground/validator/v10 v10.10.0 h1:I7mrTYv78z8k8VXa/qJlOlEXn/nBh+BF8dHX5nt/dr0=
github.com/go-playground/validator/v10 v10.10.0/go.mod h1:74x4gJWsvQexRdW8Pn3dXSGrTK4nAUsbPlLADvpJkos=
github.com/go-quicktest/qt v1.101.0 h1:O1K29Txy5P2OK0dGo59b7b0LR6wKfIhttaAhHUyn7eI=
github.com/go-quicktest/qt v1.101.0/go.mod h1:14Bz/f7NwaXPtdYEgzsx46kqSxVwTbzVZsDC26tQJow=
github.com/go-sourcemap/sourcemap v2.1.3+incompatible h1:W1iEw64niKVGogNgBN3ePyLFfuisuzeidWPMPWmECqU=
github.com/go-sourcemap/sourcemap v2.1.3+incompatible/go.mod h1:F8jJfvm2KbVjc5NqelyYJmf/v5J0dwNLS2mL4sNA1Jg=
github.com/go-stack/stack v1.8.1 h1:ntEHSVwIt7PNXNpgPmVfMrNhLtgjlmnZha2kOpuRiDw=
@@ -141,8 +145,8 @@ github.com/google/pprof v0.0.0-20230207041349-798e818bf904 h1:4/hN5RUoecvl+RmJRE
github.com/google/pprof v0.0.0-20230207041349-798e818bf904/go.mod h1:uglQLonpP8qtYCYyzA+8c/9qtqgA3qsXGYqCPKARAFg=
github.com/google/renameio v0.1.0 h1:GOZbcHa3HfsPKPlmyPyN2KEohoMXOhdMbHrvbpl2QaA=
github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm40UhjYkI=
github.com/google/rpmpack v0.6.1-0.20240329070804-c2247cbb881a h1:JJBdjSfqSy3mnDT0940ASQFghwcZ4y4cb6ttjAoXqwE=
github.com/google/rpmpack v0.6.1-0.20240329070804-c2247cbb881a/go.mod h1:uqVAUVQLq8UY2hCDfmJ/+rtO3aw7qyhc90rCVEabEfI=
github.com/google/rpmpack v0.6.1-0.20250405124433-758cc6896cbc h1:qES+d3PvR9CN+zARQQH/bNXH0ybzmdjNMHICrBwXD28=
github.com/google/rpmpack v0.6.1-0.20250405124433-758cc6896cbc/go.mod h1:uqVAUVQLq8UY2hCDfmJ/+rtO3aw7qyhc90rCVEabEfI=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 h1:El6M4kTTCOh6aBiKaUGG7oYTSPP8MxqL4YI3kZKwcP4=
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510/go.mod h1:pupxD2MaaD3pAXIBCelhxNneeOaAeabZDe5s4K6zSpQ=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
@@ -153,8 +157,8 @@ github.com/goreleaser/chglog v0.7.0 h1:/KzXWAeg4DrEz4r3OI6K2Yb8RAsVGeInCUfLWFXL9
github.com/goreleaser/chglog v0.7.0/go.mod h1:2h/yyq9xvTUeM9tOoucBP+jri8Dj28splx+SjlYkklc=
github.com/goreleaser/fileglob v1.3.0 h1:/X6J7U8lbDpQtBvGcwwPS6OpzkNVlVEsFUVRx9+k+7I=
github.com/goreleaser/fileglob v1.3.0/go.mod h1:Jx6BoXv3mbYkEzwm9THo7xbr5egkAraxkGorbJb4RxU=
github.com/goreleaser/nfpm/v2 v2.42.0 h1:7BW4WQWyvZDrT0C7SyWop+J8rtqFyTB17Sb2/j/NxMI=
github.com/goreleaser/nfpm/v2 v2.42.0/go.mod h1:DtNL+nKpfB8sMFZp+X7Xu3W64atyZYtTnYe8O925/mg=
github.com/goreleaser/nfpm/v2 v2.42.1 h1:xu2pLRgQuz2ab+YZFoeIzwU/M5jjjCKDGwv1lRbVGvk=
github.com/goreleaser/nfpm/v2 v2.42.1/go.mod h1:dY53KWYKebkOocxgkmpM7SRX0Nv5hU+jEu2kIaM4/LI=
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI=
github.com/henvic/httpretty v0.0.6/go.mod h1:X38wLjWXHkXT7r2+uK8LjCMne9rsuNaBLJ+5cU2/Pmo=
github.com/huandu/xstrings v1.5.0 h1:2ag3IFq9ZDANvthTwTiqSSZLjDc+BedvHPAp5tJy2TI=
@@ -270,16 +274,16 @@ golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.38.0 h1:jt+WWG8IZlBnVbomuhg2Mdq0+BBQaHbtqHEFEigjUV8=
golang.org/x/crypto v0.38.0/go.mod h1:MvrbAqul58NNYPKnOra203SB9vpuZW0e+RRZV+Ggqjw=
golang.org/x/crypto v0.39.0 h1:SHs+kF4LP+f+p14esP5jAoDpHU8Gu/v9lFRK6IT5imM=
golang.org/x/crypto v0.39.0/go.mod h1:L+Xg3Wf6HoL4Bn4238Z6ft6KfEpN0tJGo53AAPC632U=
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 h1:2dVuKD2vS7b0QIHQbpyTISPd0LeHDbnYEryqj5Q1ug8=
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56/go.mod h1:M4RDyNAINzryxdtnbRXRL/OHtkFuWGRjvuhBJpk2IlY=
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 h1:1P7xPZEwZMoBoz0Yze5Nx2/4pxj6nw9ZqHWXqP0iRgQ=
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.24.0 h1:ZfthKaKaT4NrhGVZHO1/WDTwGES4De8KtWO0SIbNJMU=
golang.org/x/mod v0.24.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
golang.org/x/mod v0.25.0 h1:n7a+ZbQKQA/Ysbyb0/6IbB1H/X41mKgbhfv7AfG/44w=
golang.org/x/mod v0.25.0/go.mod h1:IXM97Txy2VM4PJ3gI61r1YEk/gAj6zAHN3AdZt6S9Ww=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
@@ -287,13 +291,13 @@ golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2/go.mod h1:9nx3DQGgdP8bBQD5qx
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.40.0 h1:79Xs7wF06Gbdcg4kdCCIQArK11Z1hr5POQ6+fIYHNuY=
golang.org/x/net v0.40.0/go.mod h1:y0hY0exeL2Pku80/zKK7tpntoX23cqL3Oa6njdgRtds=
golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.14.0 h1:woo0S4Yywslg6hp4eUFjTVOyKt0RookbpAHG4c1HmhQ=
golang.org/x/sync v0.14.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sync v0.15.0 h1:KWH3jNZsfyT6xfAfKiz6MRNmd46ByHDYaZ7KSkCtdW8=
golang.org/x/sync v0.15.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -334,14 +338,14 @@ golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8=
golang.org/x/text v0.9.0/go.mod h1:e1OnstbJyHTd6l/uOt8jFFHp6TRDWZR/bV3emEE/zU8=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.25.0 h1:qVyWApTSYLk/drJRO5mDlNYskwQznZmkpV2c8q9zls4=
golang.org/x/text v0.25.0/go.mod h1:WEdwpYrmk1qmdHvhkSTNPm3app7v4rsT8F2UD6+VHIA=
golang.org/x/text v0.26.0 h1:P42AVeLghgTYr4+xUnTRKDMqpar+PtX7KWuNQL21L8M=
golang.org/x/text v0.26.0/go.mod h1:QK15LZJUUQVJxhz7wXgxSy/CJaTFjd0G+YLonydOVQA=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.32.0 h1:Q7N1vhpkQv7ybVzLFtTjvQya2ewbwNDZzUgfXGqtMWU=
golang.org/x/tools v0.32.0/go.mod h1:ZxrU41P/wAbZD8EDa6dDCa6XfpkhJ7HFMjHJXfBDu8s=
golang.org/x/tools v0.33.0 h1:4qz2S3zmRxbGIhDIAgjxvFutSvH5EfnsYrRBj0UI0bc=
golang.org/x/tools v0.33.0/go.mod h1:CIJMaWEY88juyUfo7UbgPqbC8rU2OqfAV1h2Qp0oMYI=
golang.org/x/vuln v1.1.4 h1:Ju8QsuyhX3Hk8ma3CesTbO8vfJD9EvUBgHvkxHBzj0I=
golang.org/x/vuln v1.1.4/go.mod h1:F+45wmU18ym/ca5PLTPLsSzr2KppzswxPP603ldA67s=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@@ -371,8 +375,10 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.6.1 h1:R094WgE8K4JirYjBaOpz/AvTyUu/3wbmAoskKN/pxTI=
honnef.co/go/tools v0.6.1/go.mod h1:3puzxxljPCe8RGJX7BIy1plGbxEOZni5mR2aXe3/uk4=
k8s.io/apimachinery v0.33.0 h1:1a6kHrJxb2hs4t8EE5wuR/WxKDwGN1FKH3JvDtA0CIQ=
k8s.io/apimachinery v0.33.0/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM=
k8s.io/apimachinery v0.33.1 h1:mzqXWV8tW9Rw4VeW9rEkqvnxj59k1ezDUl20tFK/oM4=
k8s.io/apimachinery v0.33.1/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM=
mvdan.cc/sh/v3 v3.11.0 h1:q5h+XMDRfUGUedCqFFsjoFjrhwf2Mvtt1rkMvVz0blw=
mvdan.cc/sh/v3 v3.11.0/go.mod h1:LRM+1NjoYCzuq/WZ6y44x14YNAI0NK7FLPeQSaFagGg=
pault.ag/go/debian v0.18.0 h1:nr0iiyOU5QlG1VPnhZLNhnCcHx58kukvBJp+dvaM6CQ=
pault.ag/go/debian v0.18.0/go.mod h1:JFl0XWRCv9hWBrB5MDDZjA5GSEs1X3zcFK/9kCNIUmE=
pault.ag/go/topsort v0.1.1 h1:L0QnhUly6LmTv0e3DEzbN2q6/FGgAcQvaEw65S53Bg4=

View File

@@ -81,12 +81,12 @@ func XForwardedForToXRealIP(next http.Handler) http.Handler {
// XForwardedForUpdate sets or updates the X-Forwarded-For header, adding
// the known remote address to an existing chain if present
func XForwardedForUpdate(next http.Handler) http.Handler {
func XForwardedForUpdate(stripPrivate bool, next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer next.ServeHTTP(w, r)
pref := XFFComputePreferences{
StripPrivate: true,
StripPrivate: stripPrivate,
StripLoopback: true,
StripCGNAT: true,
Flatten: true,

7
internal/mimetype.go Normal file
View File

@@ -0,0 +1,7 @@
package internal
import "mime"
func init() {
mime.AddExtensionType(".mjs", "text/javascript")
}

View File

@@ -214,6 +214,11 @@ func TestPlaywrightBrowser(t *testing.T) {
return
}
if os.Getenv("SKIP_INTEGRATION") != "" {
t.Skip("SKIP_INTEGRATION was set")
return
}
startPlaywright(t)
pw := setupPlaywright(t)
@@ -289,6 +294,11 @@ func TestPlaywrightWithBasePrefix(t *testing.T) {
return
}
if os.Getenv("SKIP_INTEGRATION") != "" {
t.Skip("SKIP_INTEGRATION was set")
return
}
t.Skip("NOTE(Xe)\\ these tests require HTTPS support in #364")
startPlaywright(t)

View File

@@ -23,7 +23,7 @@ func TestXForwardedForUpdateIgnoreUnix(t *testing.T) {
w := httptest.NewRecorder()
XForwardedForUpdate(h).ServeHTTP(w, r)
XForwardedForUpdate(true, h).ServeHTTP(w, r)
if r.RemoteAddr != remoteAddr {
t.Errorf("wanted remoteAddr to be %s, got: %s", r.RemoteAddr, remoteAddr)
@@ -43,7 +43,7 @@ func TestXForwardedForUpdateAddToChain(t *testing.T) {
w.WriteHeader(http.StatusOK)
})
srv := httptest.NewServer(XForwardedForUpdate(h))
srv := httptest.NewServer(XForwardedForUpdate(true, h))
r, err := http.NewRequest(http.MethodGet, srv.URL, nil)
if err != nil {
@@ -81,6 +81,15 @@ func TestComputeXFFHeader(t *testing.T) {
},
result: "1.1.1.1,127.0.0.1",
},
{
name: "StripPrivate",
remoteAddr: "127.0.0.1:80",
origXFFHeader: "1.1.1.1,10.0.0.1",
pref: XFFComputePreferences{
StripPrivate: false,
},
result: "1.1.1.1,10.0.0.1,127.0.0.1",
},
{
name: "StripLoopback",
remoteAddr: "127.0.0.1:80",

View File

@@ -3,16 +3,14 @@ package lib
import (
"crypto/ed25519"
"crypto/sha256"
"crypto/subtle"
"encoding/json"
"errors"
"fmt"
"log/slog"
"math"
"net"
"net/http"
"net/url"
"slices"
"strconv"
"strings"
"time"
@@ -26,8 +24,13 @@ import (
"github.com/TecharoHQ/anubis/internal"
"github.com/TecharoHQ/anubis/internal/dnsbl"
"github.com/TecharoHQ/anubis/internal/ogtags"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/lib/policy/config"
// challenge implementations
_ "github.com/TecharoHQ/anubis/lib/challenge/metarefresh"
_ "github.com/TecharoHQ/anubis/lib/challenge/proofofwork"
)
var (
@@ -36,26 +39,25 @@ var (
Help: "The total number of challenges issued",
}, []string{"method"})
challengesValidated = promauto.NewCounter(prometheus.CounterOpts{
challengesValidated = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_challenges_validated",
Help: "The total number of challenges validated",
})
}, []string{"method"})
droneBLHits = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_dronebl_hits",
Help: "The total number of hits from DroneBL",
}, []string{"status"})
failedValidations = promauto.NewCounter(prometheus.CounterOpts{
failedValidations = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_failed_validations",
Help: "The total number of failed validations",
})
}, []string{"method"})
timeTaken = promauto.NewHistogram(prometheus.HistogramOpts{
Name: "anubis_time_taken",
Help: "The time taken for a browser to generate a response (milliseconds)",
Buckets: prometheus.ExponentialBucketsRange(1, math.Pow(2, 18), 19),
})
requestsProxied = promauto.NewCounterVec(prometheus.CounterOpts{
Name: "anubis_proxied_requests_total",
Help: "Number of requests proxied through Anubis to upstream targets",
}, []string{"host"})
)
type Server struct {
@@ -64,18 +66,23 @@ type Server struct {
policy *policy.ParsedConfig
DNSBLCache *decaymap.Impl[string, dnsbl.DroneBLResponse]
OGTags *ogtags.OGTagCache
cookieName string
priv ed25519.PrivateKey
pub ed25519.PublicKey
opts Options
cookieName string
}
func (s *Server) challengeFor(r *http.Request, difficulty int) string {
fp := sha256.Sum256(s.priv.Seed())
fp := sha256.Sum256(s.pub[:])
acceptLanguage := r.Header.Get("Accept-Language")
if len(acceptLanguage) > 5 {
acceptLanguage = acceptLanguage[:5]
}
challengeData := fmt.Sprintf(
"Accept-Language=%s,X-Real-IP=%s,User-Agent=%s,WeekTime=%s,Fingerprint=%x,Difficulty=%d",
r.Header.Get("Accept-Language"),
acceptLanguage,
r.Header.Get("X-Real-Ip"),
r.UserAgent(),
time.Now().UTC().Round(24*7*time.Hour).Format(time.RFC3339),
@@ -157,6 +164,29 @@ func (s *Server) maybeReverseProxy(w http.ResponseWriter, r *http.Request, httpS
return
}
claims, ok := token.Claims.(jwt.MapClaims)
if !ok {
lg.Debug("invalid token claims type", "path", r.URL.Path)
s.ClearCookie(w, s.cookieName, cookiePath)
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
policyRule, ok := claims["policyRule"].(string)
if !ok {
lg.Debug("policyRule claim is not a string")
s.ClearCookie(w, s.cookieName, cookiePath)
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
if policyRule != rule.Hash() {
lg.Debug("user originally passed with a different rule, issuing new challenge", "old", policyRule, "new", rule.Name)
s.ClearCookie(w, s.cookieName, cookiePath)
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
r.Header.Add("X-Anubis-Status", "PASS")
s.ServeHTTPNext(w, r)
}
@@ -226,6 +256,21 @@ func (s *Server) handleDNSBL(w http.ResponseWriter, r *http.Request, ip string,
func (s *Server) MakeChallenge(w http.ResponseWriter, r *http.Request) {
lg := internal.GetRequestLogger(r)
redir := r.FormValue("redir")
if redir == "" {
w.WriteHeader(http.StatusBadRequest)
encoder := json.NewEncoder(w)
lg.Error("invalid invocation of MakeChallenge", "redir", redir)
encoder.Encode(struct {
Error string `json:"error"`
}{
Error: "Invalid invocation of MakeChallenge",
})
return
}
r.URL.Path = redir
encoder := json.NewEncoder(w)
cr, rule, err := s.check(r)
if err != nil {
@@ -272,6 +317,14 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
cookiePath = strings.TrimSuffix(anubis.BasePrefix, "/") + "/"
}
if _, err := r.Cookie(anubis.TestCookieName); err == http.ErrNoCookie {
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, anubis.TestCookieName, "/")
lg.Warn("user has cookies disabled, this is not an anubis bug")
s.respondWithError(w, r, "Your browser is configured to disable cookies. Anubis requires cookies for the legitimate interest of making sure you are a valid client. Please enable cookies for this domain")
return
}
s.ClearCookie(w, anubis.TestCookieName, "/")
redir := r.FormValue("redir")
@@ -284,42 +337,6 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
// used by the path checker rule
r.URL = redirURL
cr, rule, err := s.check(r)
if err != nil {
lg.Error("check failed", "err", err)
s.respondWithError(w, r, "Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to look for the logs around \"passChallenge\".")
return
}
lg = lg.With("check_result", cr)
nonceStr := r.FormValue("nonce")
if nonceStr == "" {
s.ClearCookie(w, s.cookieName, cookiePath)
lg.Debug("no nonce")
s.respondWithError(w, r, "missing nonce")
return
}
elapsedTimeStr := r.FormValue("elapsedTime")
if elapsedTimeStr == "" {
s.ClearCookie(w, s.cookieName, cookiePath)
lg.Debug("no elapsedTime")
s.respondWithError(w, r, "missing elapsedTime")
return
}
elapsedTime, err := strconv.ParseFloat(elapsedTimeStr, 64)
if err != nil {
s.ClearCookie(w, s.cookieName, cookiePath)
lg.Debug("elapsedTime doesn't parse", "err", err)
s.respondWithError(w, r, "invalid elapsedTime")
return
}
lg.Info("challenge took", "elapsedTime", elapsedTime)
timeTaken.Observe(elapsedTime)
response := r.FormValue("response")
urlParsed, err := r.URL.Parse(redir)
if err != nil {
s.respondWithError(w, r, "Redirect URL not parseable")
@@ -330,54 +347,47 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
return
}
challenge := s.challengeFor(r, rule.Challenge.Difficulty)
if _, err := r.Cookie(anubis.TestCookieName); err == http.ErrNoCookie {
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, anubis.TestCookieName, cookiePath)
lg.Warn("user has cookies disabled, this is not an anubis bug")
s.respondWithError(w, r, "Your browser is configured to disable cookies. Anubis requires cookies for the legitimate interest of making sure you are a valid client. Please enable cookies for this domain")
return
}
nonce, err := strconv.Atoi(nonceStr)
cr, rule, err := s.check(r)
if err != nil {
s.ClearCookie(w, s.cookieName, cookiePath)
lg.Debug("nonce doesn't parse", "err", err)
s.respondWithError(w, r, "invalid response")
lg.Error("check failed", "err", err)
s.respondWithError(w, r, "Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to look for the logs around \"passChallenge\"")
return
}
lg = lg.With("check_result", cr)
impl, ok := challenge.Get(rule.Challenge.Algorithm)
if !ok {
lg.Error("check failed", "err", err)
s.respondWithError(w, r, fmt.Sprintf("Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to file a bug as Anubis is trying to use challenge method %s but it does not exist in the challenge registry", rule.Challenge.Algorithm))
return
}
calcString := fmt.Sprintf("%s%d", challenge, nonce)
calculated := internal.SHA256sum(calcString)
challengeStr := s.challengeFor(r, rule.Challenge.Difficulty)
if subtle.ConstantTimeCompare([]byte(response), []byte(calculated)) != 1 {
if err := impl.Validate(r, lg, rule, challengeStr); err != nil {
failedValidations.WithLabelValues(string(rule.Challenge.Algorithm)).Inc()
var cerr *challenge.Error
s.ClearCookie(w, s.cookieName, cookiePath)
lg.Debug("hash does not match", "got", response, "want", calculated)
s.respondWithStatus(w, r, "invalid response", http.StatusForbidden)
failedValidations.Inc()
return
}
lg.Debug("challenge validate call failed", "err", err)
// compare the leading zeroes
if !strings.HasPrefix(response, strings.Repeat("0", rule.Challenge.Difficulty)) {
s.ClearCookie(w, s.cookieName, cookiePath)
lg.Debug("difficulty check failed", "response", response, "difficulty", rule.Challenge.Difficulty)
s.respondWithStatus(w, r, "invalid response", http.StatusForbidden)
failedValidations.Inc()
return
switch {
case errors.As(err, &cerr):
switch {
case errors.Is(err, challenge.ErrFailed):
s.respondWithStatus(w, r, cerr.PublicReason, cerr.StatusCode)
case errors.Is(err, challenge.ErrInvalidFormat), errors.Is(err, challenge.ErrMissingField):
s.respondWithError(w, r, cerr.PublicReason)
}
}
}
// generate JWT cookie
token := jwt.NewWithClaims(jwt.SigningMethodEdDSA, jwt.MapClaims{
"challenge": challenge,
"nonce": nonceStr,
"response": response,
"iat": time.Now().Unix(),
"nbf": time.Now().Add(-1 * time.Minute).Unix(),
"exp": time.Now().Add(s.opts.CookieExpiration).Unix(),
tokenString, err := s.signJWT(jwt.MapClaims{
"challenge": challengeStr,
"method": rule.Challenge.Algorithm,
"policyRule": rule.Hash(),
"action": string(cr.Rule),
})
tokenString, err := token.SignedString(s.priv)
if err != nil {
lg.Error("failed to sign JWT", "err", err)
s.ClearCookie(w, s.cookieName, cookiePath)
@@ -387,23 +397,25 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
s.SetCookie(w, s.cookieName, tokenString, cookiePath)
challengesValidated.Inc()
challengesValidated.WithLabelValues(rule.Challenge.Algorithm).Inc()
lg.Debug("challenge passed, redirecting to app")
http.Redirect(w, r, redir, http.StatusFound)
}
func (s *Server) TestError(w http.ResponseWriter, r *http.Request) {
err := r.FormValue("err")
s.respondWithError(w, r, err)
}
func cr(name string, rule config.Rule) policy.CheckResult {
func cr(name string, rule config.Rule, weight int) policy.CheckResult {
return policy.CheckResult{
Name: name,
Rule: rule,
Name: name,
Rule: rule,
Weight: weight,
}
}
var (
weightOkayStatic = policy.NewStaticHashChecker("weight/okay")
weightMildSusStatic = policy.NewStaticHashChecker("weight/mild-suspicion")
weightVerySusStatic = policy.NewStaticHashChecker("weight/extreme-suspicion")
)
// Check evaluates the list of rules, and returns the result
func (s *Server) check(r *http.Request) (policy.CheckResult, *policy.Bot, error) {
host := r.Header.Get("X-Real-Ip")
@@ -416,6 +428,8 @@ func (s *Server) check(r *http.Request) (policy.CheckResult, *policy.Bot, error)
return decaymap.Zilch[policy.CheckResult](), nil, fmt.Errorf("[misconfiguration] %q is not an IP address", host)
}
weight := 0
for _, b := range s.policy.Bots {
match, err := b.Rules.Check(r)
if err != nil {
@@ -423,16 +437,53 @@ func (s *Server) check(r *http.Request) (policy.CheckResult, *policy.Bot, error)
}
if match {
return cr("bot/"+b.Name, b.Action), &b, nil
switch b.Action {
case config.RuleDeny, config.RuleAllow, config.RuleBenchmark, config.RuleChallenge:
return cr("bot/"+b.Name, b.Action, weight), &b, nil
case config.RuleWeigh:
slog.Debug("adjusting weight", "name", b.Name, "delta", b.Weight.Adjust)
weight += b.Weight.Adjust
}
}
}
return cr("default/allow", config.RuleAllow), &policy.Bot{
switch {
case weight <= 0:
return cr("weight/okay", config.RuleAllow, weight), &policy.Bot{
Challenge: &config.ChallengeRules{
Difficulty: s.policy.DefaultDifficulty,
ReportAs: s.policy.DefaultDifficulty,
Algorithm: config.DefaultAlgorithm,
},
Rules: weightOkayStatic,
}, nil
case weight > 0 && weight < 10:
return cr("weight/mild-suspicion", config.RuleChallenge, weight), &policy.Bot{
Challenge: &config.ChallengeRules{
Difficulty: s.policy.DefaultDifficulty,
ReportAs: s.policy.DefaultDifficulty,
Algorithm: "metarefresh",
},
Rules: weightMildSusStatic,
}, nil
case weight >= 10:
return cr("weight/extreme-suspicion", config.RuleChallenge, weight), &policy.Bot{
Challenge: &config.ChallengeRules{
Difficulty: s.policy.DefaultDifficulty,
ReportAs: s.policy.DefaultDifficulty,
Algorithm: "fast",
},
Rules: weightVerySusStatic,
}, nil
}
return cr("default/allow", config.RuleAllow, weight), &policy.Bot{
Challenge: &config.ChallengeRules{
Difficulty: s.policy.DefaultDifficulty,
ReportAs: s.policy.DefaultDifficulty,
Algorithm: config.AlgorithmFast,
Algorithm: config.DefaultAlgorithm,
},
Rules: &policy.CheckerList{},
}, nil
}

View File

@@ -4,10 +4,11 @@ import (
"encoding/json"
"fmt"
"net/http"
"net/http/cookiejar"
"net/http/httptest"
"net/url"
"os"
"strings"
"sync"
"testing"
"time"
@@ -18,6 +19,10 @@ import (
"github.com/TecharoHQ/anubis/lib/policy/config"
)
func init() {
internal.InitSlog("debug")
}
func loadPolicies(t *testing.T, fname string) *policy.ParsedConfig {
t.Helper()
@@ -40,20 +45,29 @@ func spawnAnubis(t *testing.T, opts Options) *Server {
return s
}
type challenge struct {
type challengeResp struct {
Challenge string `json:"challenge"`
}
func makeChallenge(t *testing.T, ts *httptest.Server, cli *http.Client) challenge {
func makeChallenge(t *testing.T, ts *httptest.Server, cli *http.Client) challengeResp {
t.Helper()
resp, err := cli.Post(ts.URL+"/.within.website/x/cmd/anubis/api/make-challenge", "", nil)
req, err := http.NewRequest(http.MethodPost, ts.URL+"/.within.website/x/cmd/anubis/api/make-challenge", nil)
if err != nil {
t.Fatalf("can't make request: %v", err)
}
q := req.URL.Query()
q.Set("redir", "/")
req.URL.RawQuery = q.Encode()
resp, err := cli.Do(req)
if err != nil {
t.Fatalf("can't request challenge: %v", err)
}
defer resp.Body.Close()
var chall challenge
var chall challengeResp
if err := json.NewDecoder(resp.Body).Decode(&chall); err != nil {
t.Fatalf("can't read challenge response body: %v", err)
}
@@ -61,7 +75,7 @@ func makeChallenge(t *testing.T, ts *httptest.Server, cli *http.Client) challeng
return chall
}
func handleChallengeZeroDifficulty(t *testing.T, ts *httptest.Server, cli *http.Client, chall challenge) *http.Response {
func handleChallengeZeroDifficulty(t *testing.T, ts *httptest.Server, cli *http.Client, chall challengeResp) *http.Response {
t.Helper()
nonce := 0
@@ -91,16 +105,48 @@ func handleChallengeZeroDifficulty(t *testing.T, ts *httptest.Server, cli *http.
return resp
}
type loggingCookieJar struct {
t *testing.T
lock sync.Mutex
cookies map[string][]*http.Cookie
}
func (lcj *loggingCookieJar) Cookies(u *url.URL) []*http.Cookie {
lcj.lock.Lock()
defer lcj.lock.Unlock()
// XXX(Xe): This is not RFC compliant in the slightest.
result, ok := lcj.cookies[u.Host]
if !ok {
return nil
}
lcj.t.Logf("requested cookies for %s", u)
for _, ckie := range result {
lcj.t.Logf("get cookie: <- %s", ckie)
}
return result
}
func (lcj *loggingCookieJar) SetCookies(u *url.URL, cookies []*http.Cookie) {
lcj.lock.Lock()
defer lcj.lock.Unlock()
for _, ckie := range cookies {
lcj.t.Logf("set cookie: %s -> %s", u, ckie)
}
// XXX(Xe): This is not RFC compliant in the slightest.
lcj.cookies[u.Host] = append(lcj.cookies[u.Host], cookies...)
}
func httpClient(t *testing.T) *http.Client {
t.Helper()
jar, err := cookiejar.New(nil)
if err != nil {
t.Fatal(err)
}
cli := &http.Client{
Jar: jar,
Jar: &loggingCookieJar{t: t, cookies: map[string][]*http.Cookie{}},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
@@ -134,8 +180,7 @@ func TestCVE2025_24369(t *testing.T) {
Next: http.NewServeMux(),
Policy: pol,
CookiePartitioned: true,
CookieName: t.Name(),
CookieName: t.Name(),
})
ts := httptest.NewServer(internal.RemoteXRealIP(true, "tcp", srv))
@@ -318,7 +363,7 @@ func TestBasePrefix(t *testing.T) {
}{
{
name: "no prefix",
basePrefix: "",
basePrefix: "/",
path: "/.within.website/x/cmd/anubis/api/make-challenge",
expected: "/.within.website/x/cmd/anubis/api/make-challenge",
},
@@ -353,8 +398,19 @@ func TestBasePrefix(t *testing.T) {
ts := httptest.NewServer(internal.RemoteXRealIP(true, "tcp", srv))
defer ts.Close()
cli := httpClient(t)
req, err := http.NewRequest(http.MethodPost, ts.URL+tc.path, nil)
if err != nil {
t.Fatal(err)
}
q := req.URL.Query()
q.Set("redir", tc.basePrefix)
req.URL.RawQuery = q.Encode()
// Test API endpoint with prefix
resp, err := ts.Client().Post(ts.URL+tc.path, "", nil)
resp, err := cli.Do(req)
if err != nil {
t.Fatalf("can't request challenge: %v", err)
}
@@ -364,7 +420,7 @@ func TestBasePrefix(t *testing.T) {
t.Errorf("expected status code %d, got: %d", http.StatusOK, resp.StatusCode)
}
var chall challenge
var chall challengeResp
if err := json.NewDecoder(resp.Body).Decode(&chall); err != nil {
t.Fatalf("can't read challenge response body: %v", err)
}
@@ -388,7 +444,6 @@ func TestBasePrefix(t *testing.T) {
elapsedTime := 420
redir := "/"
cli := ts.Client()
cli.CheckRedirect = func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
}
@@ -397,7 +452,7 @@ func TestBasePrefix(t *testing.T) {
passChallengePath := tc.path
passChallengePath = passChallengePath[:strings.LastIndex(passChallengePath, "/")+1] + "pass-challenge"
req, err := http.NewRequest(http.MethodGet, ts.URL+passChallengePath, nil)
req, err = http.NewRequest(http.MethodGet, ts.URL+passChallengePath, nil)
if err != nil {
t.Fatalf("can't make request: %v", err)
}
@@ -406,7 +461,7 @@ func TestBasePrefix(t *testing.T) {
req.AddCookie(ckie)
}
q := req.URL.Query()
q = req.URL.Query()
q.Set("response", calculated)
q.Set("nonce", fmt.Sprint(nonce))
q.Set("redir", redir)
@@ -549,3 +604,31 @@ func TestCloudflareWorkersRule(t *testing.T) {
})
}
}
func TestRuleChange(t *testing.T) {
pol := loadPolicies(t, "testdata/rule_change.yaml")
pol.DefaultDifficulty = 0
ckieExpiration := 10 * time.Minute
srv := spawnAnubis(t, Options{
Next: http.NewServeMux(),
Policy: pol,
CookieDomain: "127.0.0.1",
CookieName: t.Name(),
CookieExpiration: ckieExpiration,
})
ts := httptest.NewServer(internal.RemoteXRealIP(true, "tcp", srv))
defer ts.Close()
cli := httpClient(t)
chall := makeChallenge(t, ts, cli)
resp := handleChallengeZeroDifficulty(t, ts, cli, chall)
if resp.StatusCode != http.StatusFound {
resp.Write(os.Stderr)
t.Errorf("wanted %d, got: %d", http.StatusFound, resp.StatusCode)
}
}

View File

@@ -0,0 +1,52 @@
package challenge
import (
"log/slog"
"net/http"
"sort"
"sync"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/a-h/templ"
)
var (
registry map[string]Impl = map[string]Impl{}
regLock sync.RWMutex
)
func Register(name string, impl Impl) {
regLock.Lock()
defer regLock.Unlock()
registry[name] = impl
}
func Get(name string) (Impl, bool) {
regLock.RLock()
defer regLock.RUnlock()
result, ok := registry[name]
return result, ok
}
func Methods() []string {
regLock.RLock()
defer regLock.RUnlock()
var result []string
for method := range registry {
result = append(result, method)
}
sort.Strings(result)
return result
}
type Impl interface {
// Setup registers any additional routes with the Impl for assets or API routes.
Setup(mux *http.ServeMux)
// Issue a new challenge to the user, called by the Anubis.
Issue(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string, ogTags map[string]string) (templ.Component, error)
// Validate a challenge, making sure that it passes muster.
Validate(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string) error
}

37
lib/challenge/error.go Normal file
View File

@@ -0,0 +1,37 @@
package challenge
import (
"errors"
"fmt"
"net/http"
)
var (
ErrFailed = errors.New("challenge: user failed challenge")
ErrMissingField = errors.New("challenge: missing field")
ErrInvalidFormat = errors.New("challenge: field has invalid format")
)
func NewError(verb, publicReason string, privateReason error) *Error {
return &Error{
Verb: verb,
PublicReason: publicReason,
PrivateReason: privateReason,
StatusCode: http.StatusForbidden,
}
}
type Error struct {
Verb string
PublicReason string
PrivateReason error
StatusCode int
}
func (e *Error) Error() string {
return fmt.Sprintf("challenge: error when processing challenge: %s: %v", e.Verb, e.PrivateReason)
}
func (e *Error) Unwrap() error {
return e.PrivateReason
}

View File

@@ -0,0 +1,53 @@
package metarefresh
import (
"crypto/subtle"
"fmt"
"log/slog"
"net/http"
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/web"
"github.com/a-h/templ"
)
//go:generate go tool github.com/a-h/templ/cmd/templ generate
func init() {
challenge.Register("metarefresh", &Impl{})
}
type Impl struct{}
func (i *Impl) Setup(mux *http.ServeMux) {}
func (i *Impl) Issue(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string, ogTags map[string]string) (templ.Component, error) {
u, err := r.URL.Parse(anubis.BasePrefix + "/.within.website/x/cmd/anubis/api/pass-challenge")
if err != nil {
return nil, fmt.Errorf("can't render page: %w", err)
}
q := u.Query()
q.Set("redir", r.URL.String())
q.Set("challenge", challenge)
u.RawQuery = q.Encode()
component, err := web.BaseWithChallengeAndOGTags("Making sure you're not a bot!", page(challenge, u.String(), rule.Challenge.Difficulty), challenge, rule.Challenge, ogTags)
if err != nil {
return nil, fmt.Errorf("can't render page: %w", err)
}
return component, nil
}
func (i *Impl) Validate(r *http.Request, lg *slog.Logger, rule *policy.Bot, wantChallenge string) error {
gotChallenge := r.FormValue("challenge")
if subtle.ConstantTimeCompare([]byte(wantChallenge), []byte(gotChallenge)) != 1 {
return challenge.NewError("validate", "invalid response", fmt.Errorf("%w: wanted response %s but got %s", challenge.ErrFailed, wantChallenge, gotChallenge))
}
return nil
}

View File

@@ -0,0 +1,17 @@
package metarefresh
import (
"fmt"
"github.com/TecharoHQ/anubis"
)
templ page(challenge, redir string, difficulty int) {
<div class="centered-div">
<img id="image" style="width:100%;max-width:256px;" src={ anubis.BasePrefix + "/.within.website/x/cmd/anubis/static/img/pensive.webp?cacheBuster=" + anubis.Version }/>
<img style="display:none;" style="width:100%;max-width:256px;" src={ anubis.BasePrefix + "/.within.website/x/cmd/anubis/static/img/happy.webp?cacheBuster=" + anubis.Version }/>
<p id="status">Loading...</p>
<p>Please wait a moment while we ensure the security of your connection.</p>
<meta http-equiv="refresh" content={ fmt.Sprintf("%d; url=%s", difficulty, redir) }/>
</div>
}

View File

@@ -0,0 +1,85 @@
// Code generated by templ - DO NOT EDIT.
// templ: version: v0.3.898
package metarefresh
//lint:file-ignore SA4006 This context is only used if a nested component is present.
import "github.com/a-h/templ"
import templruntime "github.com/a-h/templ/runtime"
import (
"fmt"
"github.com/TecharoHQ/anubis"
)
func page(challenge, redir string, difficulty int) templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
return templ_7745c5c3_CtxErr
}
templ_7745c5c3_Buffer, templ_7745c5c3_IsBuffer := templruntime.GetBuffer(templ_7745c5c3_W)
if !templ_7745c5c3_IsBuffer {
defer func() {
templ_7745c5c3_BufErr := templruntime.ReleaseBuffer(templ_7745c5c3_Buffer)
if templ_7745c5c3_Err == nil {
templ_7745c5c3_Err = templ_7745c5c3_BufErr
}
}()
}
ctx = templ.InitializeContext(ctx)
templ_7745c5c3_Var1 := templ.GetChildren(ctx)
if templ_7745c5c3_Var1 == nil {
templ_7745c5c3_Var1 = templ.NopComponent
}
ctx = templ.ClearChildren(ctx)
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 1, "<div class=\"centered-div\"><img id=\"image\" style=\"width:100%;max-width:256px;\" src=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var2 string
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(anubis.BasePrefix + "/.within.website/x/cmd/anubis/static/img/pensive.webp?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 11, Col: 165}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 2, "\"> <img style=\"display:none;\" style=\"width:100%;max-width:256px;\" src=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var3 string
templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(anubis.BasePrefix + "/.within.website/x/cmd/anubis/static/img/happy.webp?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 12, Col: 174}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "\"><p id=\"status\">Loading...</p><p>Please wait a moment while we ensure the security of your connection.</p><meta http-equiv=\"refresh\" content=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var4 string
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d; url=%s", difficulty, redir))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 15, Col: 83}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "\"></div>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
return nil
})
}
var _ = templruntime.GeneratedTemplate

14
lib/challenge/metrics.go Normal file
View File

@@ -0,0 +1,14 @@
package challenge
import (
"math"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)
var TimeTaken = promauto.NewHistogramVec(prometheus.HistogramOpts{
Name: "anubis_time_taken",
Help: "The time taken for a browser to generate a response (milliseconds)",
Buckets: prometheus.ExponentialBucketsRange(1, math.Pow(2, 20), 20),
}, []string{"method"})

View File

@@ -0,0 +1,83 @@
package proofofwork
import (
"crypto/subtle"
"fmt"
"log/slog"
"net/http"
"strconv"
"strings"
"github.com/TecharoHQ/anubis/internal"
chall "github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/web"
"github.com/a-h/templ"
)
func init() {
chall.Register("fast", &Impl{Algorithm: "fast"})
chall.Register("slow", &Impl{Algorithm: "slow"})
}
type Impl struct {
Algorithm string
}
func (i *Impl) Setup(mux *http.ServeMux) {
/* no implementation required */
}
func (i *Impl) Issue(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string, ogTags map[string]string) (templ.Component, error) {
component, err := web.BaseWithChallengeAndOGTags("Making sure you're not a bot!", web.Index(), challenge, rule.Challenge, ogTags)
if err != nil {
return nil, fmt.Errorf("can't render page: %w", err)
}
return component, nil
}
func (i *Impl) Validate(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string) error {
nonceStr := r.FormValue("nonce")
if nonceStr == "" {
return chall.NewError("validate", "invalid response", fmt.Errorf("%w nonce", chall.ErrMissingField))
}
nonce, err := strconv.Atoi(nonceStr)
if err != nil {
return chall.NewError("validate", "invalid response", fmt.Errorf("%w: nonce: %w", chall.ErrInvalidFormat, err))
}
elapsedTimeStr := r.FormValue("elapsedTime")
if elapsedTimeStr == "" {
return chall.NewError("validate", "invalid response", fmt.Errorf("%w elapsedTime", chall.ErrMissingField))
}
elapsedTime, err := strconv.ParseFloat(elapsedTimeStr, 64)
if err != nil {
return chall.NewError("validate", "invalid response", fmt.Errorf("%w: elapsedTime: %w", chall.ErrInvalidFormat, err))
}
response := r.FormValue("response")
if response == "" {
return chall.NewError("validate", "invalid response", fmt.Errorf("%w response", chall.ErrMissingField))
}
calcString := fmt.Sprintf("%s%d", challenge, nonce)
calculated := internal.SHA256sum(calcString)
if subtle.ConstantTimeCompare([]byte(response), []byte(calculated)) != 1 {
return chall.NewError("validate", "invalid response", fmt.Errorf("%w: wanted response %s but got %s", chall.ErrFailed, calculated, response))
}
// compare the leading zeroes
if !strings.HasPrefix(response, strings.Repeat("0", rule.Challenge.Difficulty)) {
return chall.NewError("validate", "invalid response", fmt.Errorf("%w: wanted %d leading zeros but got %s", chall.ErrFailed, rule.Challenge.Difficulty, response))
}
lg.Debug("challenge took", "elapsedTime", elapsedTime)
chall.TimeTaken.WithLabelValues(i.Algorithm).Observe(elapsedTime)
return nil
}

View File

@@ -0,0 +1,136 @@
package proofofwork
import (
"errors"
"log/slog"
"net/http"
"testing"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/lib/policy/config"
)
func mkRequest(t *testing.T, values map[string]string) *http.Request {
t.Helper()
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, "/", nil)
if err != nil {
t.Fatal(err)
}
q := req.URL.Query()
for k, v := range values {
q.Set(k, v)
}
req.URL.RawQuery = q.Encode()
return req
}
func TestBasic(t *testing.T) {
i := &Impl{Algorithm: "fast"}
bot := &policy.Bot{
Challenge: &config.ChallengeRules{
Algorithm: "fast",
Difficulty: 0,
ReportAs: 0,
},
}
const challengeStr = "hunter"
const response = "2652bdba8fb4d2ab39ef28d8534d7694c557a4ae146c1e9237bd8d950280500e"
for _, cs := range []struct {
name string
req *http.Request
err error
challengeStr string
}{
{
name: "allgood",
req: mkRequest(t, map[string]string{
"nonce": "0",
"elapsedTime": "69",
"response": response,
}),
err: nil,
challengeStr: challengeStr,
},
{
name: "no-params",
req: mkRequest(t, map[string]string{}),
err: challenge.ErrMissingField,
challengeStr: challengeStr,
},
{
name: "missing-nonce",
req: mkRequest(t, map[string]string{
"elapsedTime": "69",
"response": response,
}),
err: challenge.ErrMissingField,
challengeStr: challengeStr,
},
{
name: "missing-elapsedTime",
req: mkRequest(t, map[string]string{
"nonce": "0",
"response": response,
}),
err: challenge.ErrMissingField,
challengeStr: challengeStr,
},
{
name: "missing-response",
req: mkRequest(t, map[string]string{
"nonce": "0",
"elapsedTime": "69",
}),
err: challenge.ErrMissingField,
challengeStr: challengeStr,
},
{
name: "wrong-nonce-format",
req: mkRequest(t, map[string]string{
"nonce": "taco",
"elapsedTime": "69",
"response": response,
}),
err: challenge.ErrInvalidFormat,
challengeStr: challengeStr,
},
{
name: "wrong-elapsedTime-format",
req: mkRequest(t, map[string]string{
"nonce": "0",
"elapsedTime": "taco",
"response": response,
}),
err: challenge.ErrInvalidFormat,
challengeStr: challengeStr,
},
{
name: "invalid-response",
req: mkRequest(t, map[string]string{
"nonce": "0",
"elapsedTime": "69",
"response": response,
}),
err: challenge.ErrFailed,
challengeStr: "Tacos are tasty",
},
} {
t.Run(cs.name, func(t *testing.T) {
lg := slog.With()
if _, err := i.Issue(cs.req, lg, bot, cs.challengeStr, nil); err != nil {
t.Errorf("can't issue challenge: %v", err)
}
if err := i.Validate(cs.req, lg, bot, cs.challengeStr); !errors.Is(err, cs.err) {
t.Errorf("got wrong error from Validate, got %v but wanted %v", err, cs.err)
}
})
}
}

View File

@@ -3,6 +3,7 @@ package lib
import (
"crypto/ed25519"
"crypto/rand"
"errors"
"fmt"
"io"
"log/slog"
@@ -17,6 +18,7 @@ import (
"github.com/TecharoHQ/anubis/internal"
"github.com/TecharoHQ/anubis/internal/dnsbl"
"github.com/TecharoHQ/anubis/internal/ogtags"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/web"
"github.com/TecharoHQ/anubis/xess"
@@ -65,6 +67,17 @@ func LoadPoliciesOrDefault(fname string, defaultDifficulty int) (*policy.ParsedC
}(fin)
anubisPolicy, err := policy.ParseConfig(fin, fname, defaultDifficulty)
var validationErrs []error
for _, b := range anubisPolicy.Bots {
if _, ok := challenge.Get(b.Challenge.Algorithm); !ok {
validationErrs = append(validationErrs, fmt.Errorf("%w %s", policy.ErrChallengeRuleHasWrongAlgorithm, b.Challenge.Algorithm))
}
}
if len(validationErrs) != 0 {
return nil, fmt.Errorf("can't do final validation of Anubis config: %w", errors.Join(validationErrs...))
}
return anubisPolicy, err
}
@@ -132,12 +145,21 @@ func New(opts Options) (*Server, error) {
}), "GET")
}
registerWithPrefix(anubis.APIPrefix+"make-challenge", http.HandlerFunc(result.MakeChallenge), "POST")
registerWithPrefix(anubis.APIPrefix+"pass-challenge", http.HandlerFunc(result.PassChallenge), "GET")
registerWithPrefix(anubis.APIPrefix+"check", http.HandlerFunc(result.maybeReverseProxyHttpStatusOnly), "")
registerWithPrefix(anubis.APIPrefix+"test-error", http.HandlerFunc(result.TestError), "GET")
registerWithPrefix("/", http.HandlerFunc(result.maybeReverseProxyOrPage), "")
//goland:noinspection GoBoolExpressions
if anubis.Version == "devel" {
// make-challenge is only used in tests. Only enable while version is devel
registerWithPrefix(anubis.APIPrefix+"make-challenge", http.HandlerFunc(result.MakeChallenge), "POST")
}
for _, implKind := range challenge.Methods() {
impl, _ := challenge.Get(implKind)
impl.Setup(mux)
}
result.mux = mux
return result, nil

51
lib/config_test.go Normal file
View File

@@ -0,0 +1,51 @@
package lib
import (
"errors"
"os"
"path/filepath"
"testing"
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/lib/policy"
)
func TestInvalidChallengeMethod(t *testing.T) {
if _, err := LoadPoliciesOrDefault("testdata/invalid-challenge-method.yaml", 4); !errors.Is(err, policy.ErrChallengeRuleHasWrongAlgorithm) {
t.Fatalf("wanted error %v but got %v", policy.ErrChallengeRuleHasWrongAlgorithm, err)
}
}
func TestBadConfigs(t *testing.T) {
finfos, err := os.ReadDir("policy/config/testdata/bad")
if err != nil {
t.Fatal(err)
}
for _, st := range finfos {
st := st
t.Run(st.Name(), func(t *testing.T) {
if _, err := LoadPoliciesOrDefault(filepath.Join("policy", "config", "testdata", "good", st.Name()), anubis.DefaultDifficulty); err == nil {
t.Fatal(err)
} else {
t.Log(err)
}
})
}
}
func TestGoodConfigs(t *testing.T) {
finfos, err := os.ReadDir("policy/config/testdata/good")
if err != nil {
t.Fatal(err)
}
for _, st := range finfos {
st := st
t.Run(st.Name(), func(t *testing.T) {
if _, err := LoadPoliciesOrDefault(filepath.Join("policy", "config", "testdata", "good", st.Name()), anubis.DefaultDifficulty); err != nil {
t.Fatal(err)
}
})
}
}

View File

@@ -1,6 +1,7 @@
package lib
import (
"fmt"
"math/rand"
"net/http"
"slices"
@@ -9,9 +10,11 @@ import (
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/internal"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/web"
"github.com/a-h/templ"
"github.com/golang-jwt/jwt/v5"
)
func (s *Server) SetCookie(w http.ResponseWriter, name, value, path string) {
@@ -74,7 +77,7 @@ func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, rule *polic
}
challengesIssued.WithLabelValues("embedded").Add(1)
challenge := s.challengeFor(r, rule.Challenge.Difficulty)
challengeStr := s.challengeFor(r, rule.Challenge.Difficulty)
var ogTags map[string]string = nil
if s.opts.OGPassthrough {
@@ -87,14 +90,21 @@ func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, rule *polic
http.SetCookie(w, &http.Cookie{
Name: anubis.TestCookieName,
Value: challenge,
Value: challengeStr,
Expires: time.Now().Add(30 * time.Minute),
Path: "/",
})
component, err := web.BaseWithChallengeAndOGTags("Making sure you're not a bot!", web.Index(), challenge, rule.Challenge, ogTags)
impl, ok := challenge.Get(rule.Challenge.Algorithm)
if !ok {
lg.Error("check failed", "err", "can't get algorithm", "algorithm", rule.Challenge.Algorithm)
s.respondWithError(w, r, fmt.Sprintf("Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to file a bug as Anubis is trying to use challenge method %s but it does not exist in the challenge registry", rule.Challenge.Algorithm))
return
}
component, err := impl.Issue(r, lg, rule, challengeStr, ogTags)
if err != nil {
lg.Error("render failed, please open an issue", "err", err) // This is likely a bug in the template. Should never be triggered as CI tests for this.
lg.Error("[unexpected] render failed, please open an issue", "err", err) // This is likely a bug in the template. Should never be triggered as CI tests for this.
s.respondWithError(w, r, "Internal Server Error: please contact the administrator and ask them to look for the logs around \"RenderIndex\"")
return
}
@@ -147,6 +157,15 @@ func (s *Server) ServeHTTPNext(w http.ResponseWriter, r *http.Request) {
web.Base("You are not a bot!", web.StaticHappy()),
).ServeHTTP(w, r)
} else {
requestsProxied.WithLabelValues(r.Host).Inc()
s.next.ServeHTTP(w, r)
}
}
func (s *Server) signJWT(claims jwt.MapClaims) (string, error) {
claims["iat"] = time.Now().Unix()
claims["nbf"] = time.Now().Add(-1 * time.Minute).Unix()
claims["exp"] = time.Now().Add(s.opts.CookieExpiration).Unix()
return jwt.NewWithClaims(jwt.SigningMethodEdDSA, claims).SignedString(s.priv)
}

View File

@@ -12,6 +12,7 @@ type Bot struct {
Challenge *config.ChallengeRules
Name string
Action config.Rule
Weight *config.Weight
}
func (b Bot) Hash() string {

View File

@@ -47,6 +47,20 @@ func (cl CheckerList) Hash() string {
return internal.SHA256sum(sb.String())
}
type staticHashChecker struct {
hash string
}
func (staticHashChecker) Check(r *http.Request) (bool, error) {
return true, nil
}
func (s staticHashChecker) Hash() string { return s.hash }
func NewStaticHashChecker(hashable string) Checker {
return staticHashChecker{hash: internal.SHA256sum(hashable)}
}
type RemoteAddrChecker struct {
ranger cidranger.Ranger
hash string
@@ -62,7 +76,10 @@ func NewRemoteAddrChecker(cidrs []string) (Checker, error) {
return nil, fmt.Errorf("%w: range %s not parsing: %w", ErrMisconfiguration, cidr, err)
}
ranger.Insert(cidranger.NewBasicRangerEntry(*rng))
err = ranger.Insert(cidranger.NewBasicRangerEntry(*rng))
if err != nil {
return nil, fmt.Errorf("%w: error inserting ip range: %w", ErrMisconfiguration, err)
}
fmt.Fprintln(&sb, cidr)
}

View File

@@ -7,12 +7,15 @@ import (
)
type CheckResult struct {
Name string
Rule config.Rule
Name string
Rule config.Rule
Weight int
}
func (cr CheckResult) LogValue() slog.Value {
return slog.GroupValue(
slog.String("name", cr.Name),
slog.String("rule", string(cr.Rule)))
slog.String("rule", string(cr.Rule)),
slog.Int("weight", cr.Weight),
)
}

View File

@@ -39,26 +39,22 @@ const (
RuleAllow Rule = "ALLOW"
RuleDeny Rule = "DENY"
RuleChallenge Rule = "CHALLENGE"
RuleWeigh Rule = "WEIGH"
RuleBenchmark Rule = "DEBUG_BENCHMARK"
)
type Algorithm string
const (
AlgorithmUnknown Algorithm = ""
AlgorithmFast Algorithm = "fast"
AlgorithmSlow Algorithm = "slow"
)
const DefaultAlgorithm = "fast"
type BotConfig struct {
UserAgentRegex *string `json:"user_agent_regex"`
PathRegex *string `json:"path_regex"`
HeadersRegex map[string]string `json:"headers_regex"`
Expression *ExpressionOrList `json:"expression"`
UserAgentRegex *string `json:"user_agent_regex,omitempty"`
PathRegex *string `json:"path_regex,omitempty"`
HeadersRegex map[string]string `json:"headers_regex,omitempty"`
Expression *ExpressionOrList `json:"expression,omitempty"`
Challenge *ChallengeRules `json:"challenge,omitempty"`
Weight *Weight `json:"weight,omitempty"`
Name string `json:"name"`
Action Rule `json:"action"`
RemoteAddr []string `json:"remote_addresses"`
RemoteAddr []string `json:"remote_addresses,omitempty"`
}
func (b BotConfig) Zero() bool {
@@ -79,7 +75,7 @@ func (b BotConfig) Zero() bool {
return true
}
func (b BotConfig) Valid() error {
func (b *BotConfig) Valid() error {
var errs []error
if b.Name == "" {
@@ -150,7 +146,7 @@ func (b BotConfig) Valid() error {
}
switch b.Action {
case RuleAllow, RuleBenchmark, RuleChallenge, RuleDeny:
case RuleAllow, RuleBenchmark, RuleChallenge, RuleDeny, RuleWeigh:
// okay
default:
errs = append(errs, fmt.Errorf("%w: %q", ErrUnknownAction, b.Action))
@@ -162,6 +158,10 @@ func (b BotConfig) Valid() error {
}
}
if b.Action == RuleWeigh && b.Weight == nil {
b.Weight = &Weight{Adjust: 5}
}
if len(errs) != 0 {
return fmt.Errorf("config: bot entry for %q is not valid:\n%w", b.Name, errors.Join(errs...))
}
@@ -170,15 +170,14 @@ func (b BotConfig) Valid() error {
}
type ChallengeRules struct {
Algorithm Algorithm `json:"algorithm"`
Difficulty int `json:"difficulty"`
ReportAs int `json:"report_as"`
Algorithm string `json:"algorithm"`
Difficulty int `json:"difficulty"`
ReportAs int `json:"report_as"`
}
var (
ErrChallengeRuleHasWrongAlgorithm = errors.New("config.Bot.ChallengeRules: algorithm is invalid")
ErrChallengeDifficultyTooLow = errors.New("config.Bot.ChallengeRules: difficulty is too low (must be >= 1)")
ErrChallengeDifficultyTooHigh = errors.New("config.Bot.ChallengeRules: difficulty is too high (must be <= 64)")
ErrChallengeDifficultyTooLow = errors.New("config.Bot.ChallengeRules: difficulty is too low (must be >= 1)")
ErrChallengeDifficultyTooHigh = errors.New("config.Bot.ChallengeRules: difficulty is too high (must be <= 64)")
)
func (cr ChallengeRules) Valid() error {
@@ -192,13 +191,6 @@ func (cr ChallengeRules) Valid() error {
errs = append(errs, fmt.Errorf("%w, got: %d", ErrChallengeDifficultyTooHigh, cr.Difficulty))
}
switch cr.Algorithm {
case AlgorithmFast, AlgorithmSlow, AlgorithmUnknown:
// do nothing, it's all good
default:
errs = append(errs, fmt.Errorf("%w: %q", ErrChallengeRuleHasWrongAlgorithm, cr.Algorithm))
}
if len(errs) != 0 {
return fmt.Errorf("config: challenge rules entry is not valid:\n%w", errors.Join(errs...))
}

View File

@@ -130,20 +130,6 @@ func TestBotValid(t *testing.T) {
},
err: ErrChallengeDifficultyTooHigh,
},
{
name: "challenge wrong algorithm",
bot: BotConfig{
Name: "mozilla-ua",
Action: RuleChallenge,
PathRegex: p("Mozilla"),
Challenge: &ChallengeRules{
Difficulty: 420,
ReportAs: 4,
Algorithm: "high quality rips",
},
},
err: ErrChallengeRuleHasWrongAlgorithm,
},
{
name: "invalid cidr range",
bot: BotConfig{
@@ -182,6 +168,25 @@ func TestBotValid(t *testing.T) {
},
err: nil,
},
{
name: "weight rule without weight",
bot: BotConfig{
Name: "weight-adjust-if-mozilla",
Action: RuleWeigh,
UserAgentRegex: p("Mozilla"),
},
},
{
name: "weight rule with weight adjust",
bot: BotConfig{
Name: "weight-adjust-if-mozilla",
Action: RuleWeigh,
UserAgentRegex: p("Mozilla"),
Weight: &Weight{
Adjust: 5,
},
},
},
}
for _, cs := range tests {
@@ -251,6 +256,7 @@ func TestImportStatement(t *testing.T) {
"bots",
"common",
"crawlers",
"meta",
} {
if err := fs.WalkDir(data.BotPolicies, folderName, func(path string, d fs.DirEntry, err error) error {
if err != nil {
@@ -259,6 +265,9 @@ func TestImportStatement(t *testing.T) {
if d.IsDir() {
return nil
}
if d.Name() == "README.md" {
return nil
}
tests = append(tests, testCase{
name: "(data)/" + path,
@@ -357,7 +366,7 @@ func TestBotConfigZero(t *testing.T) {
b.Challenge = &ChallengeRules{
Difficulty: 4,
ReportAs: 4,
Algorithm: AlgorithmFast,
Algorithm: DefaultAlgorithm,
}
if b.Zero() {
t.Error("BotConfig with challenge rules is zero value")

View File

@@ -14,8 +14,8 @@ var (
type ExpressionOrList struct {
Expression string `json:"-"`
All []string `json:"all"`
Any []string `json:"any"`
All []string `json:"all,omitempty"`
Any []string `json:"any,omitempty"`
}
func (eol ExpressionOrList) Equal(rhs *ExpressionOrList) bool {

View File

@@ -1,7 +1,7 @@
{
"bots": [
{
"import": "(data)/bots/ai-robots-txt.yaml",
"import": "(data)/bots/ai-catchall.yaml",
"name": "generic-browser",
"user_agent_regex": "Mozilla|Opera\n",
"action": "CHALLENGE"

View File

@@ -1,5 +1,5 @@
bots:
- import: (data)/bots/ai-robots-txt.yaml
- import: (data)/bots/ai-catchall.yaml
name: generic-browser
user_agent_regex: >
Mozilla|Opera

View File

@@ -0,0 +1,8 @@
bots:
- name: total-randomness
action: ALLOW
expression:
all:
- '"Accept" in headers'
- headers["Accept"].contains("text/html")
- randInt(1) == 0

View File

@@ -0,0 +1,6 @@
bots:
- name: simple-weight-adjust
action: WEIGH
user_agent_regex: Mozilla
weight:
adjust: 5

View File

@@ -0,0 +1,4 @@
bots:
- name: weight
action: WEIGH
user_agent_regex: Mozilla

View File

@@ -0,0 +1,5 @@
package config
type Weight struct {
Adjust int `json:"adjust"`
}

View File

@@ -1,7 +1,11 @@
package expressions
import (
"math/rand/v2"
"github.com/google/cel-go/cel"
"github.com/google/cel-go/common/types"
"github.com/google/cel-go/common/types/ref"
"github.com/google/cel-go/ext"
)
@@ -29,6 +33,20 @@ func NewEnvironment() (*cel.Env, error) {
cel.Variable("headers", cel.MapType(cel.StringType, cel.StringType)),
// Functions exposed to CEL programs:
cel.Function("randInt",
cel.Overload("randInt_int",
[]*cel.Type{cel.IntType},
cel.IntType,
cel.UnaryBinding(func(val ref.Val) ref.Val {
n, ok := val.(types.Int)
if !ok {
return types.ValOrErr(val, "value is not an integer, but is %T", val)
}
return types.Int(rand.IntN(int(n)))
}),
),
),
)
}

View File

@@ -18,14 +18,14 @@ func TestHTTPHeaders(t *testing.T) {
t.Run("contains-existing-header", func(t *testing.T) {
resp := headers.Contains(types.String("User-Agent"))
if !bool(resp.(types.Bool)) {
if !resp.(types.Bool) {
t.Fatal("headers does not contain User-Agent")
}
})
t.Run("not-contains-missing-header", func(t *testing.T) {
resp := headers.Contains(types.String("Xxx-Random-Header"))
if bool(resp.(types.Bool)) {
if resp.(types.Bool) {
t.Fatal("headers does not contain User-Agent")
}
})

View File

@@ -16,14 +16,14 @@ func TestURLValues(t *testing.T) {
t.Run("contains-existing-key", func(t *testing.T) {
resp := headers.Contains(types.String("format"))
if !bool(resp.(types.Bool)) {
if !resp.(types.Bool) {
t.Fatal("headers does not contain User-Agent")
}
})
t.Run("not-contains-missing-key", func(t *testing.T) {
resp := headers.Contains(types.String("not-there"))
if bool(resp.(types.Bool)) {
if resp.(types.Bool) {
t.Fatal("headers does not contain User-Agent")
}
})

View File

@@ -5,10 +5,9 @@ import (
"fmt"
"io"
"github.com/TecharoHQ/anubis/lib/policy/config"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/TecharoHQ/anubis/lib/policy/config"
)
var (
@@ -16,6 +15,8 @@ var (
Name: "anubis_policy_results",
Help: "The results of each policy rule",
}, []string{"rule", "action"})
ErrChallengeRuleHasWrongAlgorithm = errors.New("config.Bot.ChallengeRules: algorithm is invalid")
)
type ParsedConfig struct {
@@ -107,15 +108,19 @@ func ParseConfig(fin io.Reader, fname string, defaultDifficulty int) (*ParsedCon
parsedBot.Challenge = &config.ChallengeRules{
Difficulty: defaultDifficulty,
ReportAs: defaultDifficulty,
Algorithm: config.AlgorithmFast,
Algorithm: "fast",
}
} else {
parsedBot.Challenge = b.Challenge
if parsedBot.Challenge.Algorithm == config.AlgorithmUnknown {
parsedBot.Challenge.Algorithm = config.AlgorithmFast
if parsedBot.Challenge.Algorithm == "" {
parsedBot.Challenge.Algorithm = config.DefaultAlgorithm
}
}
if b.Weight != nil {
parsedBot.Weight = b.Weight
}
parsedBot.Rules = cl
result.Bots = append(result.Bots, parsedBot)

9
lib/testdata/hack-test.json vendored Normal file
View File

@@ -0,0 +1,9 @@
[
{
"name": "ipv6-ula",
"action": "ALLOW",
"remote_addresses": [
"fc00::/7"
]
}
]

3
lib/testdata/hack-test.yaml vendored Normal file
View File

@@ -0,0 +1,3 @@
- name: well-known
path_regex: ^/.well-known/.*$
action: ALLOW

View File

@@ -0,0 +1,8 @@
bots:
- name: generic-bot-catchall
user_agent_regex: (?i:bot|crawler)
action: CHALLENGE
challenge:
difficulty: 16
report_as: 4
algorithm: hunter2 # invalid algorithm

12
lib/testdata/rule_change.yaml vendored Normal file
View File

@@ -0,0 +1,12 @@
bots:
- name: old-rule
path_regex: ^/old$
action: CHALLENGE
- name: new-rule
path_regex: ^/new$
action: CHALLENGE
status_codes:
CHALLENGE: 401
DENY: 403

212
package-lock.json generated
View File

@@ -1,17 +1,17 @@
{
"name": "@techaro/anubis",
"version": "1.19.0-pre1",
"version": "1.19.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@techaro/anubis",
"version": "1.19.0-pre1",
"version": "1.19.1",
"license": "ISC",
"devDependencies": {
"cssnano": "^7.0.7",
"cssnano-preset-advanced": "^7.0.7",
"esbuild": "^0.25.4",
"esbuild": "^0.25.5",
"playwright": "^1.52.0",
"postcss-cli": "^11.0.1",
"postcss-import": "^16.1.0",
@@ -20,9 +20,9 @@
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.25.4.tgz",
"integrity": "sha512-1VCICWypeQKhVbE9oW/sJaAmjLxhVqacdkvPLEjwlttjfwENRSClS8EjBz0KzRyFSCPDIkuXW34Je/vk7zdB7Q==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.25.5.tgz",
"integrity": "sha512-9o3TMmpmftaCMepOdA5k/yDw8SfInyzWWTjYTFCX3kPSDJMROQTb8jg+h9Cnwnmm1vOzvxN7gIfB5V2ewpjtGA==",
"cpu": [
"ppc64"
],
@@ -37,9 +37,9 @@
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.25.4.tgz",
"integrity": "sha512-QNdQEps7DfFwE3hXiU4BZeOV68HHzYwGd0Nthhd3uCkkEKK7/R6MTgM0P7H7FAs5pU/DIWsviMmEGxEoxIZ+ZQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.25.5.tgz",
"integrity": "sha512-AdJKSPeEHgi7/ZhuIPtcQKr5RQdo6OO2IL87JkianiMYMPbCtot9fxPbrMiBADOWWm3T2si9stAiVsGbTQFkbA==",
"cpu": [
"arm"
],
@@ -54,9 +54,9 @@
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.25.4.tgz",
"integrity": "sha512-bBy69pgfhMGtCnwpC/x5QhfxAz/cBgQ9enbtwjf6V9lnPI/hMyT9iWpR1arm0l3kttTr4L0KSLpKmLp/ilKS9A==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.25.5.tgz",
"integrity": "sha512-VGzGhj4lJO+TVGV1v8ntCZWJktV7SGCs3Pn1GRWI1SBFtRALoomm8k5E9Pmwg3HOAal2VDc2F9+PM/rEY6oIDg==",
"cpu": [
"arm64"
],
@@ -71,9 +71,9 @@
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.25.4.tgz",
"integrity": "sha512-TVhdVtQIFuVpIIR282btcGC2oGQoSfZfmBdTip2anCaVYcqWlZXGcdcKIUklfX2wj0JklNYgz39OBqh2cqXvcQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.25.5.tgz",
"integrity": "sha512-D2GyJT1kjvO//drbRT3Hib9XPwQeWd9vZoBJn+bu/lVsOZ13cqNdDeqIF/xQ5/VmWvMduP6AmXvylO/PIc2isw==",
"cpu": [
"x64"
],
@@ -88,9 +88,9 @@
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.25.4.tgz",
"integrity": "sha512-Y1giCfM4nlHDWEfSckMzeWNdQS31BQGs9/rouw6Ub91tkK79aIMTH3q9xHvzH8d0wDru5Ci0kWB8b3up/nl16g==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.25.5.tgz",
"integrity": "sha512-GtaBgammVvdF7aPIgH2jxMDdivezgFu6iKpmT+48+F8Hhg5J/sfnDieg0aeG/jfSvkYQU2/pceFPDKlqZzwnfQ==",
"cpu": [
"arm64"
],
@@ -105,9 +105,9 @@
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.25.4.tgz",
"integrity": "sha512-CJsry8ZGM5VFVeyUYB3cdKpd/H69PYez4eJh1W/t38vzutdjEjtP7hB6eLKBoOdxcAlCtEYHzQ/PJ/oU9I4u0A==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.25.5.tgz",
"integrity": "sha512-1iT4FVL0dJ76/q1wd7XDsXrSW+oLoquptvh4CLR4kITDtqi2e/xwXwdCVH8hVHU43wgJdsq7Gxuzcs6Iq/7bxQ==",
"cpu": [
"x64"
],
@@ -122,9 +122,9 @@
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.25.4.tgz",
"integrity": "sha512-yYq+39NlTRzU2XmoPW4l5Ifpl9fqSk0nAJYM/V/WUGPEFfek1epLHJIkTQM6bBs1swApjO5nWgvr843g6TjxuQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.25.5.tgz",
"integrity": "sha512-nk4tGP3JThz4La38Uy/gzyXtpkPW8zSAmoUhK9xKKXdBCzKODMc2adkB2+8om9BDYugz+uGV7sLmpTYzvmz6Sw==",
"cpu": [
"arm64"
],
@@ -139,9 +139,9 @@
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.25.4.tgz",
"integrity": "sha512-0FgvOJ6UUMflsHSPLzdfDnnBBVoCDtBTVyn/MrWloUNvq/5SFmh13l3dvgRPkDihRxb77Y17MbqbCAa2strMQQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.25.5.tgz",
"integrity": "sha512-PrikaNjiXdR2laW6OIjlbeuCPrPaAl0IwPIaRv+SMV8CiM8i2LqVUHFC1+8eORgWyY7yhQY+2U2fA55mBzReaw==",
"cpu": [
"x64"
],
@@ -156,9 +156,9 @@
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.25.4.tgz",
"integrity": "sha512-kro4c0P85GMfFYqW4TWOpvmF8rFShbWGnrLqlzp4X1TNWjRY3JMYUfDCtOxPKOIY8B0WC8HN51hGP4I4hz4AaQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.25.5.tgz",
"integrity": "sha512-cPzojwW2okgh7ZlRpcBEtsX7WBuqbLrNXqLU89GxWbNt6uIg78ET82qifUy3W6OVww6ZWobWub5oqZOVtwolfw==",
"cpu": [
"arm"
],
@@ -173,9 +173,9 @@
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.25.4.tgz",
"integrity": "sha512-+89UsQTfXdmjIvZS6nUnOOLoXnkUTB9hR5QAeLrQdzOSWZvNSAXAtcRDHWtqAUtAmv7ZM1WPOOeSxDzzzMogiQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.25.5.tgz",
"integrity": "sha512-Z9kfb1v6ZlGbWj8EJk9T6czVEjjq2ntSYLY2cw6pAZl4oKtfgQuS4HOq41M/BcoLPzrUbNd+R4BXFyH//nHxVg==",
"cpu": [
"arm64"
],
@@ -190,9 +190,9 @@
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.25.4.tgz",
"integrity": "sha512-yTEjoapy8UP3rv8dB0ip3AfMpRbyhSN3+hY8mo/i4QXFeDxmiYbEKp3ZRjBKcOP862Ua4b1PDfwlvbuwY7hIGQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.25.5.tgz",
"integrity": "sha512-sQ7l00M8bSv36GLV95BVAdhJ2QsIbCuCjh/uYrWiMQSUuV+LpXwIqhgJDcvMTj+VsQmqAHL2yYaasENvJ7CDKA==",
"cpu": [
"ia32"
],
@@ -207,9 +207,9 @@
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.25.4.tgz",
"integrity": "sha512-NeqqYkrcGzFwi6CGRGNMOjWGGSYOpqwCjS9fvaUlX5s3zwOtn1qwg1s2iE2svBe4Q/YOG1q6875lcAoQK/F4VA==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.25.5.tgz",
"integrity": "sha512-0ur7ae16hDUC4OL5iEnDb0tZHDxYmuQyhKhsPBV8f99f6Z9KQM02g33f93rNH5A30agMS46u2HP6qTdEt6Q1kg==",
"cpu": [
"loong64"
],
@@ -224,9 +224,9 @@
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.25.4.tgz",
"integrity": "sha512-IcvTlF9dtLrfL/M8WgNI/qJYBENP3ekgsHbYUIzEzq5XJzzVEV/fXY9WFPfEEXmu3ck2qJP8LG/p3Q8f7Zc2Xg==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.25.5.tgz",
"integrity": "sha512-kB/66P1OsHO5zLz0i6X0RxlQ+3cu0mkxS3TKFvkb5lin6uwZ/ttOkP3Z8lfR9mJOBk14ZwZ9182SIIWFGNmqmg==",
"cpu": [
"mips64el"
],
@@ -241,9 +241,9 @@
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.25.4.tgz",
"integrity": "sha512-HOy0aLTJTVtoTeGZh4HSXaO6M95qu4k5lJcH4gxv56iaycfz1S8GO/5Jh6X4Y1YiI0h7cRyLi+HixMR+88swag==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.25.5.tgz",
"integrity": "sha512-UZCmJ7r9X2fe2D6jBmkLBMQetXPXIsZjQJCjgwpVDz+YMcS6oFR27alkgGv3Oqkv07bxdvw7fyB71/olceJhkQ==",
"cpu": [
"ppc64"
],
@@ -258,9 +258,9 @@
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.25.4.tgz",
"integrity": "sha512-i8JUDAufpz9jOzo4yIShCTcXzS07vEgWzyX3NH2G7LEFVgrLEhjwL3ajFE4fZI3I4ZgiM7JH3GQ7ReObROvSUA==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.25.5.tgz",
"integrity": "sha512-kTxwu4mLyeOlsVIFPfQo+fQJAV9mh24xL+y+Bm6ej067sYANjyEw1dNHmvoqxJUCMnkBdKpvOn0Ahql6+4VyeA==",
"cpu": [
"riscv64"
],
@@ -275,9 +275,9 @@
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.25.4.tgz",
"integrity": "sha512-jFnu+6UbLlzIjPQpWCNh5QtrcNfMLjgIavnwPQAfoGx4q17ocOU9MsQ2QVvFxwQoWpZT8DvTLooTvmOQXkO51g==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.25.5.tgz",
"integrity": "sha512-K2dSKTKfmdh78uJ3NcWFiqyRrimfdinS5ErLSn3vluHNeHVnBAFWC8a4X5N+7FgVE1EjXS1QDZbpqZBjfrqMTQ==",
"cpu": [
"s390x"
],
@@ -292,9 +292,9 @@
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.25.4.tgz",
"integrity": "sha512-6e0cvXwzOnVWJHq+mskP8DNSrKBr1bULBvnFLpc1KY+d+irZSgZ02TGse5FsafKS5jg2e4pbvK6TPXaF/A6+CA==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.25.5.tgz",
"integrity": "sha512-uhj8N2obKTE6pSZ+aMUbqq+1nXxNjZIIjCjGLfsWvVpy7gKCOL6rsY1MhRh9zLtUtAI7vpgLMK6DxjO8Qm9lJw==",
"cpu": [
"x64"
],
@@ -309,9 +309,9 @@
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.25.4.tgz",
"integrity": "sha512-vUnkBYxZW4hL/ie91hSqaSNjulOnYXE1VSLusnvHg2u3jewJBz3YzB9+oCw8DABeVqZGg94t9tyZFoHma8gWZQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.25.5.tgz",
"integrity": "sha512-pwHtMP9viAy1oHPvgxtOv+OkduK5ugofNTVDilIzBLpoWAM16r7b/mxBvfpuQDpRQFMfuVr5aLcn4yveGvBZvw==",
"cpu": [
"arm64"
],
@@ -326,9 +326,9 @@
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.25.4.tgz",
"integrity": "sha512-XAg8pIQn5CzhOB8odIcAm42QsOfa98SBeKUdo4xa8OvX8LbMZqEtgeWE9P/Wxt7MlG2QqvjGths+nq48TrUiKw==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.25.5.tgz",
"integrity": "sha512-WOb5fKrvVTRMfWFNCroYWWklbnXH0Q5rZppjq0vQIdlsQKuw6mdSihwSo4RV/YdQ5UCKKvBy7/0ZZYLBZKIbwQ==",
"cpu": [
"x64"
],
@@ -343,9 +343,9 @@
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.25.4.tgz",
"integrity": "sha512-Ct2WcFEANlFDtp1nVAXSNBPDxyU+j7+tId//iHXU2f/lN5AmO4zLyhDcpR5Cz1r08mVxzt3Jpyt4PmXQ1O6+7A==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.25.5.tgz",
"integrity": "sha512-7A208+uQKgTxHd0G0uqZO8UjK2R0DDb4fDmERtARjSHWxqMTye4Erz4zZafx7Di9Cv+lNHYuncAkiGFySoD+Mw==",
"cpu": [
"arm64"
],
@@ -360,9 +360,9 @@
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.25.4.tgz",
"integrity": "sha512-xAGGhyOQ9Otm1Xu8NT1ifGLnA6M3sJxZ6ixylb+vIUVzvvd6GOALpwQrYrtlPouMqd/vSbgehz6HaVk4+7Afhw==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.25.5.tgz",
"integrity": "sha512-G4hE405ErTWraiZ8UiSoesH8DaCsMm0Cay4fsFWOOUcz8b8rC6uCvnagr+gnioEjWn0wC+o1/TAHt+It+MpIMg==",
"cpu": [
"x64"
],
@@ -377,9 +377,9 @@
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.25.4.tgz",
"integrity": "sha512-Mw+tzy4pp6wZEK0+Lwr76pWLjrtjmJyUB23tHKqEDP74R3q95luY/bXqXZeYl4NYlvwOqoRKlInQialgCKy67Q==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.25.5.tgz",
"integrity": "sha512-l+azKShMy7FxzY0Rj4RCt5VD/q8mG/e+mDivgspo+yL8zW7qEwctQ6YqKX34DTEleFAvCIUviCFX1SDZRSyMQA==",
"cpu": [
"x64"
],
@@ -394,9 +394,9 @@
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.25.4.tgz",
"integrity": "sha512-AVUP428VQTSddguz9dO9ngb+E5aScyg7nOeJDrF1HPYu555gmza3bDGMPhmVXL8svDSoqPCsCPjb265yG/kLKQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.25.5.tgz",
"integrity": "sha512-O2S7SNZzdcFG7eFKgvwUEZ2VG9D/sn/eIiz8XRZ1Q/DO5a3s76Xv0mdBzVM5j5R639lXQmPmSo0iRpHqUUrsxw==",
"cpu": [
"arm64"
],
@@ -411,9 +411,9 @@
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.25.4.tgz",
"integrity": "sha512-i1sW+1i+oWvQzSgfRcxxG2k4I9n3O9NRqy8U+uugaT2Dy7kLO9Y7wI72haOahxceMX8hZAzgGou1FhndRldxRg==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.25.5.tgz",
"integrity": "sha512-onOJ02pqs9h1iMJ1PQphR+VZv8qBMQ77Klcsqv9CNW2w6yLqoURLcgERAIurY6QE63bbLuqgP9ATqajFLK5AMQ==",
"cpu": [
"ia32"
],
@@ -428,9 +428,9 @@
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.25.4.tgz",
"integrity": "sha512-nOT2vZNw6hJ+z43oP1SPea/G/6AbN6X+bGNhNuq8NtRHy4wsMhw765IKLNmnjek7GvjWBYQ8Q5VBoYTFg9y1UQ==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.25.5.tgz",
"integrity": "sha512-TXv6YnJ8ZMVdX+SXWVBo/0p8LTcrUYngpWjvm91TMjjBQii7Oz11Lw5lbDV5Y0TzuhSJHwiH4hEtC1I42mMS0g==",
"cpu": [
"x64"
],
@@ -1045,9 +1045,9 @@
}
},
"node_modules/esbuild": {
"version": "0.25.4",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.4.tgz",
"integrity": "sha512-8pgjLUcUjcgDg+2Q4NYXnPbo/vncAY4UmyaCm0jZevERqCHZIaWwdJHkf8XQtu4AxSKCdvrUbT0XUr1IdZzI8Q==",
"version": "0.25.5",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.5.tgz",
"integrity": "sha512-P8OtKZRv/5J5hhz0cUAdu/cLuPIKXpQl1R9pZtvmHWQvrAUVd0UNIPT4IB4W3rNOqVO0rlqHmCIbSwxh/c9yUQ==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
@@ -1058,31 +1058,31 @@
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.25.4",
"@esbuild/android-arm": "0.25.4",
"@esbuild/android-arm64": "0.25.4",
"@esbuild/android-x64": "0.25.4",
"@esbuild/darwin-arm64": "0.25.4",
"@esbuild/darwin-x64": "0.25.4",
"@esbuild/freebsd-arm64": "0.25.4",
"@esbuild/freebsd-x64": "0.25.4",
"@esbuild/linux-arm": "0.25.4",
"@esbuild/linux-arm64": "0.25.4",
"@esbuild/linux-ia32": "0.25.4",
"@esbuild/linux-loong64": "0.25.4",
"@esbuild/linux-mips64el": "0.25.4",
"@esbuild/linux-ppc64": "0.25.4",
"@esbuild/linux-riscv64": "0.25.4",
"@esbuild/linux-s390x": "0.25.4",
"@esbuild/linux-x64": "0.25.4",
"@esbuild/netbsd-arm64": "0.25.4",
"@esbuild/netbsd-x64": "0.25.4",
"@esbuild/openbsd-arm64": "0.25.4",
"@esbuild/openbsd-x64": "0.25.4",
"@esbuild/sunos-x64": "0.25.4",
"@esbuild/win32-arm64": "0.25.4",
"@esbuild/win32-ia32": "0.25.4",
"@esbuild/win32-x64": "0.25.4"
"@esbuild/aix-ppc64": "0.25.5",
"@esbuild/android-arm": "0.25.5",
"@esbuild/android-arm64": "0.25.5",
"@esbuild/android-x64": "0.25.5",
"@esbuild/darwin-arm64": "0.25.5",
"@esbuild/darwin-x64": "0.25.5",
"@esbuild/freebsd-arm64": "0.25.5",
"@esbuild/freebsd-x64": "0.25.5",
"@esbuild/linux-arm": "0.25.5",
"@esbuild/linux-arm64": "0.25.5",
"@esbuild/linux-ia32": "0.25.5",
"@esbuild/linux-loong64": "0.25.5",
"@esbuild/linux-mips64el": "0.25.5",
"@esbuild/linux-ppc64": "0.25.5",
"@esbuild/linux-riscv64": "0.25.5",
"@esbuild/linux-s390x": "0.25.5",
"@esbuild/linux-x64": "0.25.5",
"@esbuild/netbsd-arm64": "0.25.5",
"@esbuild/netbsd-x64": "0.25.5",
"@esbuild/openbsd-arm64": "0.25.5",
"@esbuild/openbsd-x64": "0.25.5",
"@esbuild/sunos-x64": "0.25.5",
"@esbuild/win32-arm64": "0.25.5",
"@esbuild/win32-ia32": "0.25.5",
"@esbuild/win32-x64": "0.25.5"
}
},
"node_modules/escalade": {

View File

@@ -1,6 +1,6 @@
{
"name": "@techaro/anubis",
"version": "1.19.0-pre1",
"version": "1.19.1",
"description": "",
"main": "index.js",
"scripts": {
@@ -20,11 +20,11 @@
"devDependencies": {
"cssnano": "^7.0.7",
"cssnano-preset-advanced": "^7.0.7",
"esbuild": "^0.25.4",
"esbuild": "^0.25.5",
"playwright": "^1.52.0",
"postcss-cli": "^11.0.1",
"postcss-import": "^16.1.0",
"postcss-import-url": "^7.2.0",
"postcss-url": "^10.1.3"
}
}
}

24
run/openrc/anubis.confd Normal file
View File

@@ -0,0 +1,24 @@
# The URL of the service that Anubis should forward valid requests to. Supports
# Unix domain sockets.
#ANUBIS_TARGET="http://localhost:3923"
#ANUBIS_TARGET="unix:///path/to/socket"
# The network address that Anubis listens on.
#
# If unset, listen on /run/anubis_${instance}/anubis.sock Unix socket instead.
#ANUBIS_BIND_PORT=":8923"
# The network address that Anubis serves Prometheus metrics on.
#
# If unset, listen on /run/anubis_${instance}/metrix.sock Unix socket instead.
#ANUBIS_METRICS_BIND_PORT=":9090"
# The difficulty of the challenge, or the number of leading zeroes that must be
# in successful responses.
#ANUBIS_DIFFICULTY=4
# Additional command-line options for Anubis.
#ANUBIS_OPTS=""
# Configure the user[:group] Anubis will run as.
#command_user="anubis:anubis"

34
run/openrc/anubis.initd Normal file
View File

@@ -0,0 +1,34 @@
#!/sbin/openrc-run
# shellcheck shell=sh
instance=${RC_SVCNAME#*.}
description="Anubis HTTP defense proxy (instance ${instance})"
supervisor="supervise-daemon"
command="/usr/bin/anubis"
command_args="\
-bind ${ANUBIS_BIND_PORT:-/run/anubis_${instance?}/anubis.sock -bind-network unix} \
-metrics-bind ${ANUBIS_METRICS_BIND_PORT:-/run/anubis_${instance?}/metrics.sock -metrics-bind-network unix} \
-target ${ANUBIS_TARGET:-http://localhost:3923} \
-difficulty ${ANUBIS_DIFFICULTY:-4} \
${ANUBIS_OPTS}
"
command_background=1
pidfile="/run/anubis_${instance?}/anubis.pid"
: "${command_user:=anubis:anubis}"
depend() {
use net firewall
}
start_pre() {
if [ "${instance?}" = "${RC_SVCNAME?}" ]; then
eerror "${RC_SVCNAME?} cannot be started directly. You must create"
eerror "symbolic links to it for the services you want to start"
eerror "and add those to the appropriate runlevels."
return 1
fi
checkpath -d -o "${command_user?}" "/run/anubis_${instance?}"
}

Some files were not shown because too many files have changed in this diff Show More