Compare commits

..

43 Commits

Author SHA1 Message Date
Xe Iaso
953f85ec74 chore: spelling
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-07-03 06:24:51 -04:00
Xe Iaso
94ed2cb1b7 fix(run/anubis@.service): unique runtimedir per instance
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-07-02 22:10:20 -04:00
Martin
0e43138324 feat(localization): Add option for forcing a language (#742)
* Add forcesLanguage option

* Change comments for forced language option

* Add changes to CHANGELOG.md

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-07-02 05:33:00 +00:00
Xe Iaso
c981c23f7e chore: npm run generate
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-07-02 05:25:10 +00:00
Xe Iaso
9f0c5e974e fix(web/main): remove the success interstitial (#745)
I'm gonna be totally honest here, I'm still not sure why #564 is still
an issue. This is really confusing and I'm going to totally throw out
how Anubis issues challenges and redo it with Valkey (#201, #622).

The problem seems to be that I assume that the makeChallenge function in
package lib is idempotent for the same client. I have no idea why this
would be inconsistent, but for some reason it is and I'm just at a loss
for words as to why this is happening.

This stops the bleeding by improving the UX as a stopgap.

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-07-01 23:44:38 +00:00
Victor Fernandes
292c470ada Set cookies to have the Secure flag default to true (#739)
* Set Cookies to use the Secure Flag and default SameSite to None

* Add secure flag test

* Updated changelog and documentation for secure flag option
2025-06-30 14:58:31 -04:00
Rafael Fontenelle
12453fdc00 Fix translations in pt-BR.json (#729)
Signed-off-by: Rafael Fontenelle <rffontenelle@users.noreply.github.com>
2025-06-30 14:14:24 -04:00
Xe Iaso
f5b3bf81bc feat: dev container support (#734)
* chore: add devcontainer for Anubis

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(devcontainer): ensure user can write to $HOME

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(devcontainer): forward ports, add launch config

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(devcontainer): add playwright deps

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: document devcontainer usage

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* ci(devcontainer): fix action references

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(devcontainer): fix ko on arm64

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-29 23:41:29 -04:00
dependabot[bot]
1820649987 build(deps): bump the gomod group with 2 updates (#736)
---
updated-dependencies:
- dependency-name: github.com/a-h/templ
  dependency-version: 0.3.906
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: gomod
- dependency-name: sigs.k8s.io/yaml
  dependency-version: 1.5.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: gomod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-29 21:32:56 -04:00
dependabot[bot]
14eeeb56d6 build(deps): bump the github-actions group with 2 updates (#735)
Bumps the github-actions group with 2 updates: [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) and [github/codeql-action](https://github.com/github/codeql-action).


Updates `astral-sh/setup-uv` from 6.3.0 to 6.3.1
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](445689ea25...bd01e18f51)

Updates `github/codeql-action` from 3.29.0 to 3.29.1
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](ce28f5bb42...39edc492db)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 6.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: github/codeql-action
  dependency-version: 3.29.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-29 20:53:14 -04:00
Martin
d9e0fbe905 feat(cmd): Add custom cookie prefix (#732)
* Add cookie prefix option

* Add explaination comment for TestCookieName

* Rename TestCookieName value from cookie-test-if-you-block-this-anubis-wont-work to cookie-verification

* Add changes to CHANGELOG.md

* Add values to CookieName and TestCookieName in anubis.go required for testcases
2025-06-29 20:03:09 -04:00
Martin
6aa17532da fix: Dynamic cookie domain not working (#731)
* Fix cookieDynamicDomain option not being set in Options struct

* Fix using wrong cookie name when using dynamic cookie domains

* Adjust testcases for new cookie option structs

* Add known words to expect.txt and change typo in Zombocom

* Cleanup expect.txt

* Add changes to changelog

* Bump versions of grpc and apimachinery

* Fix testcases and add additional condition for dynamic cookie domain
2025-06-29 15:38:55 -04:00
Xe Iaso
b1edf84a7c docs(blog/v1.20.0): i am smart
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-27 21:10:02 -04:00
Xe Iaso
d47a3406db docs(blog/v1.20.0): how did CI not catch this?
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-27 19:55:58 -04:00
Xe Iaso
ff5991b5cf docs(blog/v1.20.0): add cover image
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-27 19:20:12 -04:00
Xe Iaso
19f78f37ad docs(blog/v1.20.0): fix typo
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-27 18:59:07 -04:00
Xe Iaso
b0b0a5c08a feat(blog): v1.20.0 announcement post
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-27 18:56:09 -04:00
Rafael Fontenelle
261306dc63 Add Brazilian Portuguese translation (#726)
* Create pt-br.json

Signed-off-by: Rafael Fontenelle <rffontenelle@users.noreply.github.com>

* Enable pt-br locale

Signed-off-by: Rafael Fontenelle <rffontenelle@users.noreply.github.com>

* Fix language code

Signed-off-by: Rafael Fontenelle <rffontenelle@users.noreply.github.com>

* Update and rename pt-br.json to pt-BR.json

Signed-off-by: Rafael Fontenelle <rffontenelle@users.noreply.github.com>

* Update lib/localization/locales/pt-BR.json

Co-authored-by: Victor Fernandes  <victorvalenca@gmail.com>
Signed-off-by: Rafael Fontenelle <rffontenelle@users.noreply.github.com>

---------

Signed-off-by: Rafael Fontenelle <rffontenelle@users.noreply.github.com>
Co-authored-by: Victor Fernandes <victorvalenca@gmail.com>
2025-06-27 20:56:56 +00:00
CXM
3520421757 fix: determine bind network from bind address (#714)
* fix: determine bind network from bind address

* docs: update CHANGELOG

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-06-27 17:57:37 +00:00
Laurent Laffont
ad5430612f feat: implement localization system (#716)
* lib/localization: implement localization system

Locale files are placed in lib/localization/locales/. If you add a
locale, update manifest.json with available locales.

* Exclude locales from check spelling

* tests(lib/localization): add comprehensive translations test

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(challenge/metarefresh): enable localization

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix: use simple syntax for localization in templ

Also localize CELPHASE into French according to the wishes of the
artist.

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore:(js): fix forbidden patterns

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: add goi18n to tools

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(lib/localization): dynamically determine the list of supported languages

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-06-27 17:49:15 +00:00
Xe Iaso
c2423d0688 chore: release v1.20.0
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-27 12:06:22 -04:00
Xe Iaso
a1b7d2ccda feat: dynamic cookie domains (#722)
* feat: dynamic cookie domains

Replaces #685

I was having weird testing issues when trying to merge #685, so I
rewrote it from scratch to be a lot more minimal.

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-26 12:11:59 +00:00
msporleder
7cf6ac5de6 remove incorrect module mentions (#687)
mod_proxy_html is for modifying html content in response bodies. The example configs are using mod_proxy_http.

https://httpd.apache.org/docs/2.4/mod/mod_proxy_html.html
vs
https://httpd.apache.org/docs/2.4/mod/mod_proxy_http.html

And anyway mod_proxy + mod_proxy_http should already be installed on almost all systems.

Signed-off-by: msporleder <msporleder@gmail.com>
2025-06-26 10:47:30 +00:00
Martin
59f5b07281 feat: Add option to use HS512 secret for JWT instead of ED25519 (#680)
* Add functionality for HS512 JWT tokens

* Add HS512_SECRET to installation docs

* Update CHANGELOG.md regarding HS512

* Move HS512_SECRET to advenced section in docs

* Move token Keyfunc logic to Server function

* Add Keyfunc to spelling

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Co-authored-by: Martin Weidenauer <mweidenauer@nanx0as46153.anx.local>
Co-authored-by: Xe Iaso <me@xeiaso.net>
2025-06-26 10:06:44 +00:00
Jason Cameron
1562f88c35 chore: Remove unused/dead code (#703)
* chore(xess): remove unused xess templates

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* chore(checker): remove unused staticHashChecker implementation

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* feat: add pinact and deadcode to go tools (pinact is used for the gha pinning)

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* chore: update Docker and kubectl actions to latest versions

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* chore: update Homebrew action from master to main in workflow files

See  df537ec97f

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* chore: remove unused go-colorable and tools dependencies from go.sum

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* chore: update postcss-import and other dependencies to latest versions

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* chore: update Docusaurus dependencies to version 3.8.1

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

* chore: downgrade playwright and playwright-core to version 1.52.0

Signed-off-by: Jason Cameron <git@jasoncameron.dev>

---------

Signed-off-by: Jason Cameron <git@jasoncameron.dev>
2025-06-25 09:31:33 -04:00
Outvi V
15bd9b6a44 Populate OpenGraph configurations to Opens.OpenGraph (#717)
* chore: read OpenGraph configurations

* docs: update CHANGELOG
2025-06-24 15:12:26 +00:00
dependabot[bot]
1ca531b930 build(deps): bump the gomod group with 4 updates (#709)
Bumps the gomod group with 4 updates: [github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus](https://github.com/grpc-ecosystem/go-grpc-middleware), [github.com/grpc-ecosystem/go-grpc-middleware/v2](https://github.com/grpc-ecosystem/go-grpc-middleware), [google.golang.org/grpc](https://github.com/grpc/grpc-go) and [k8s.io/apimachinery](https://github.com/kubernetes/apimachinery).


Updates `github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus` from 1.0.1 to 1.1.0
- [Release notes](https://github.com/grpc-ecosystem/go-grpc-middleware/releases)
- [Commits](https://github.com/grpc-ecosystem/go-grpc-middleware/compare/providers/prometheus/v1.0.1...v1.1.0)

Updates `github.com/grpc-ecosystem/go-grpc-middleware/v2` from 2.1.0 to 2.3.2
- [Release notes](https://github.com/grpc-ecosystem/go-grpc-middleware/releases)
- [Commits](https://github.com/grpc-ecosystem/go-grpc-middleware/compare/v2.1.0...v2.3.2)

Updates `google.golang.org/grpc` from 1.72.2 to 1.73.0
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.72.2...v1.73.0)

Updates `k8s.io/apimachinery` from 0.33.1 to 0.33.2
- [Commits](https://github.com/kubernetes/apimachinery/compare/v0.33.1...v0.33.2)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus
  dependency-version: 1.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: gomod
- dependency-name: github.com/grpc-ecosystem/go-grpc-middleware/v2
  dependency-version: 2.3.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: gomod
- dependency-name: google.golang.org/grpc
  dependency-version: 1.73.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: gomod
- dependency-name: k8s.io/apimachinery
  dependency-version: 0.33.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: gomod
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-23 15:59:08 -04:00
Xe Iaso
f9259299b9 chore: release v1.20.0-pre2
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-23 15:17:13 -04:00
Xe Iaso
16a4e04027 fix(lib): fix invalid response after success in Chrome (#711)
Closes #564

This one is really dumb. Take a seat and listen to my tale of woe.

While @victorvalenca was working on #693 we ran into a strange issue.
The tests would consistently pass on Firefox but instantly failed on
Chrome. After adding increasingly desperate debugging logs to the mix,
we found out that somehow Chrome was randomizing the contents of its
Accept-Language header. This was making the challenge string get
calculated differently, thus making things spuriously fail. I cannot
figure out what causes Chrome to do this other than you being in an
environment where you have more than one "system language" set.

Either way, this should finally fix this issue and bring peace to the
land forever*.

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-23 15:11:56 -04:00
dependabot[bot]
8c79870edb build(deps): bump the github-actions group with 3 updates (#708)
Bumps the github-actions group with 3 updates: [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action), [actions-hub/kubectl](https://github.com/actions-hub/kubectl) and [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv).


Updates `docker/setup-buildx-action` from 3.10.0 to 3.11.1
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](b5ca514318...e468171a9d)

Updates `actions-hub/kubectl` from 1.33.1 to 1.33.2
- [Release notes](https://github.com/actions-hub/kubectl/releases)
- [Commits](f632a31512...d50394b7d7)

Updates `astral-sh/setup-uv` from 6.1.0 to 6.3.0
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](f0ec1fc3b3...445689ea25)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: 3.11.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
- dependency-name: actions-hub/kubectl
  dependency-version: 1.33.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: github-actions
- dependency-name: astral-sh/setup-uv
  dependency-version: 6.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: github-actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-23 08:41:04 -04:00
Eric T. Johnson
060b10ea2d fix(web/js): broken progress bar with slow algo (#673)
This was revealed by the reformat in #546.

Signed-off-by: Eric T. Johnson <yut23@users.noreply.github.com>
2025-06-22 20:05:37 -04:00
Xe Iaso
4c74934e9f fix(default-config): Techaro -> Zombocom
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-22 20:04:40 -04:00
Xe Iaso
5870f7072c feat: implement imprint/impressum support (#706)
* feat: implement imprint/impressum support

Closes #362

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(docs/anubis): enable an imprint

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: fix the end of the sentence, comment out a default impressum

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: link back to impressum page

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-22 18:09:37 -04:00
Xe Iaso
3c1d95d61e fix(default-config): off-by-one error in the default thresholds (#701)
I don't know how I missed this in testing.
2025-06-20 11:47:34 -04:00
Jan Alexander Steffens
ab801a3597 Makefile: Build robots2policy (#699)
* Makefile: Build robots2policy

* Update metadata

check-spelling run (pull_request) for build-robots2policy

Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
on-behalf-of: @check-spelling <check-spelling-bot@check-spelling.dev>

---------

Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
Co-authored-by: Jason Cameron <git@jasoncameron.dev>
2025-06-20 11:08:56 -04:00
Xe Iaso
ecc716940e chore: release v1.20.0-pre1
Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-19 19:32:49 -04:00
Xe Iaso
4948036f39 feat: add default OpenGraph tags to configuration file (#694)
* feat(config): opengraph passthrough configuration

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(ogtags): use config.OpenGraph for configuration

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: wire up ogtags config in most of the app

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(ogtags): return default tags if they are supplied

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: make OpenGraph legal so we have some sanity in reviewing

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib): use OpenGraph.Enabled

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(lib): load default config file if one is not specified in spawnAnubis

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(config): fix ST1005

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: document open graph defaults and its new home in the policy file

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs(installation): point to weight threshold new home

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: rename default to override

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(default-config): add off-by-default opengraph settings to bot policy file

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(anubis): make build

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(lib): fix build

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-19 18:00:44 -04:00
Xe Iaso
7aa732c700 fix(config): actually load threshold config (#696)
* fix(config): actually load threshold config

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(lib): fix test failures

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2025-06-19 17:13:01 -04:00
Xe Iaso
226cf36bf7 feat(config): custom weight thresholds via CEL (#688)
* feat(config): add Thresholds to the top level config file

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(config): make String() on ExpressionOrList join the component expressions

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(config): ensure unparseable json fails

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(config): if no thresholds are set, use the default thresholds

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(policy): half implement thresholds

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(policy): continue wiring things up

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(lib): wire up thresholds

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(lib): handle behavior from legacy configurations

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: document thresholds

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: update CHANGELOG, refer to threshold configuration

Signed-off-by: Xe Iaso <me@xeiaso.net>

* fix(lib): fix build

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore(lib): fix U1000

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
Co-authored-by: Jason Cameron <git@jasoncameron.dev>
2025-06-18 16:58:31 -04:00
Dryusdan
1d5fa49eb0 Bump ai.robots.txt to v1.37 (#689)
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
2025-06-18 13:30:53 -04:00
Lothar Serra Mari
97c1d4f353 docs(known-instances): add extensions.typo3.org (#691)
Signed-off-by: Lothar Serra Mari <mail@serra.me>
2025-06-18 08:06:23 -04:00
hydrargyrum
244f1c505a fix(geo): correct typo "counties" to "countries" (#678) 2025-06-17 23:50:42 -04:00
Jason Cameron
ae4d3b0ce5 chore: remove duplicate CHANGELOG entry (#684)
Signed-off-by: Jason Cameron <git@jasoncameron.dev>
2025-06-17 22:49:30 +00:00
123 changed files with 5279 additions and 2968 deletions

12
.devcontainer/Dockerfile Normal file
View File

@@ -0,0 +1,12 @@
FROM ghcr.io/xe/devcontainer-base/pre/go
WORKDIR /app
COPY go.mod go.sum package.json package-lock.json ./
RUN go install github.com/a-h/templ/cmd/templ \
&& npx --yes playwright@1.52.0 install --with-deps\
&& apt-get update \
&& apt-get -y install zstd brotli \
&& mkdir -p /home/vscode/.local/share/fish \
&& chown -R vscode:vscode /home/vscode/.local/share/fish \
&& chown -R vscode:vscode /go

13
.devcontainer/README.md Normal file
View File

@@ -0,0 +1,13 @@
# Anubis Dev Container
Anubis offers a [development container](https://containers.dev/) image in order to make it easier to contribute to the project. This image is based on [Xe/devcontainer-base/go](https://github.com/Xe/devcontainer-base/tree/main/src/go), which is based on Debian Bookworm with the following customizations:
- [Fish](https://fishshell.com/) as the shell complete with a custom theme
- [Go](https://go.dev) at the most recent stable version
- [Node.js](https://nodejs.org/en) at the most recent stable version
- [Atuin](https://atuin.sh/) to sync shell history between your host OS and the development container
- [Docker](https://docker.com) to manage and build Anubis container images from inside the development container
- [Ko](https://ko.build/) to build production-ready Anubis container images
- [Neovim](https://neovim.io/) for use with Git
This development container is tested and known to work with [Visual Studio Code](https://code.visualstudio.com/). If you run into problems with it outside of VS Code, please file an issue and let us know what editor you are using.

View File

@@ -0,0 +1,34 @@
// For format details, see https://aka.ms/devcontainer.json. For config options, see the
// README at: https://github.com/devcontainers/templates/tree/main/src/debian
{
"name": "Dev",
// Or use a Dockerfile or Docker Compose file. More info: https://containers.dev/guide/dockerfile
"build": {
"dockerfile": "./Dockerfile",
"context": "..",
"cacheFrom": [
"type=registry,ref=ghcr.io/techarohq/anubis/devcontainer"
]
},
"postStartCommand": "npm ci && go mod download",
"features": {
"ghcr.io/xe/devcontainer-features/ko:1.1.0": {}
},
"initializeCommand": "mkdir -p ${localEnv:HOME}${localEnv:USERPROFILE}/.local/share/atuin",
"customizations": {
"vscode": {
"extensions": [
"esbenp.prettier-vscode",
"ms-azuretools.vscode-containers",
"golang.go",
"unifiedjs.vscode-mdx",
"a-h.templ",
"redhat.vscode-yaml"
]
}
},
"forwardPorts": [
8923,
3000
]
}

View File

@@ -83,7 +83,11 @@
^\Q.github/FUNDING.yml\E$
^\Q.github/workflows/spelling.yml\E$
^data/crawlers/
^docs/blog/tags\.yml$
^docs/manifest/.*$
^docs/static/\.nojekyll$
^lib/policy/config/testdata/bad/unparseable\.json$
ignore$
robots.txt
^lib/localization/locales/.*\.json$
^lib/localization/.*_test.go$

View File

@@ -13,6 +13,8 @@ asnc
asnchecker
asns
aspirational
atuin
azuretools
badregexes
bdba
berr
@@ -33,7 +35,7 @@ Caddyfile
caninetools
Cardyb
celchecker
CELPHASE
celphase
cerr
certresolver
cespare
@@ -44,16 +46,17 @@ chall
challengemozilla
checkpath
checkresult
chen
chibi
cidranger
ckie
cloudflare
Codespaces
confd
containerbuild
coreutils
Cotoyogi
CRDs
Cromite
crt
Cscript
daemonizing
@@ -61,7 +64,6 @@ DDOS
Debian
debrpm
decaymap
decompiling
Diffbot
discordapp
discordbot
@@ -69,13 +71,17 @@ distros
dnf
dnsbl
dnserr
domainhere
dracula
dronebl
droneblresponse
dropin
duckduckbot
eerror
ellenjoe
emacs
enbyware
etld
everyones
evilbot
evilsite
@@ -131,6 +137,7 @@ iat
ifm
Imagesift
imgproxy
impressum
inp
IPTo
iptoasn
@@ -146,6 +153,7 @@ JWTs
kagi
kagibot
keikaku
Keyfunc
keypair
KHTML
kinda
@@ -180,6 +188,7 @@ mozilla
nbf
netsurf
nginx
nicksnyder
nobots
NONINFRINGEMENT
nosleep
@@ -188,6 +197,7 @@ ogtags
omgili
omgilibot
openai
opengraph
openrc
pag
palemoon
@@ -208,6 +218,7 @@ privkey
promauto
promhttp
proofofwork
publicsuffix
pwcmd
pwuser
qualys
@@ -216,8 +227,10 @@ qwantbot
rac
rawler
rcvar
redhat
redir
redirectscheme
refactors
relayd
reputational
reqmeta
@@ -225,6 +238,7 @@ risc
ruleset
runlevels
RUnlock
runtimedir
sas
sasl
Scumm
@@ -238,7 +252,9 @@ Seo
setsebool
shellcheck
Sidetrade
simprint
sitemap
skopeo
sls
sni
Sourceware
@@ -261,18 +277,21 @@ techarohq
templ
templruntime
testarea
Thancred
thoth
thothmock
Tik
Timpibot
traefik
uberspace
unifiedjs
unixhttpd
unmarshal
unparseable
uuidgen
uvx
UXP
Valkey
Varis
Velen
vendored
@@ -300,6 +319,7 @@ xess
xff
XForwarded
XNG
XOB
XReal
yae
YAMLTo
@@ -309,4 +329,5 @@ yourdomain
yoursite
Zenos
zizmor
zombocom
zos

View File

@@ -273,14 +273,6 @@
# Most people only have two hands. Reword.
\b(?i)on the third hand\b
# Should be `Open Graph`
# unless talking about a specific Open Graph implementation:
# - Java
# - Node
# - Py
# - Ruby
\bOpenGraph\b
# Should be `OpenShift`
\bOpenshift\b

47
.github/workflows/devcontainer.yml vendored Normal file
View File

@@ -0,0 +1,47 @@
name: Dev container prebuild
on:
push:
branches: ["main"]
tags: ["v*.*.*"]
jobs:
devcontainer:
runs-on: ubuntu-24.04
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-tags: true
fetch-depth: 0
persist-credentials: false
- name: Set up QEMU
uses: docker/setup-qemu-action@29109295f81e9208d7d86ff1c6c12d2833863392 # v3.6.0
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: latest
- run: |
sudo apt-get update
sudo apt-get -y install skopeo
- name: Log into registry
if: github.event_name != 'pull_request'
uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3.0.0
with:
registry: ghcr.io
username: techarohq
password: ${{ secrets.GITHUB_TOKEN }}
- name: Pre-build dev container image
uses: devcontainers/ci@8bf61b26e9c3a98f69cb6ce2f88d24ff59b785c6 # v0.3.1900000417
with:
imageName: ghcr.io/techarohq/anubis/devcontainer
cacheFrom: ghcr.io/techarohq/anubis/devcontainer
push: always
platform: linux/amd64,linux/arm64

View File

@@ -22,7 +22,7 @@ jobs:
persist-credentials: false
- name: Set up Homebrew
uses: Homebrew/actions/setup-homebrew@master
uses: Homebrew/actions/setup-homebrew@main
- name: Setup Homebrew cellar cache
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3

View File

@@ -32,7 +32,7 @@ jobs:
echo "IMAGE=ghcr.io/${GITHUB_REPOSITORY,,}" >> $GITHUB_ENV
- name: Set up Homebrew
uses: Homebrew/actions/setup-homebrew@master
uses: Homebrew/actions/setup-homebrew@main
- name: Setup Homebrew cellar cache
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3

View File

@@ -22,7 +22,7 @@ jobs:
persist-credentials: false
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Log into registry
uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0
@@ -50,14 +50,14 @@ jobs:
push: true
- name: Apply k8s manifests to aeacus
uses: actions-hub/kubectl@f632a31512a74cb35940627c49c20f67723cbaaf # v1.33.1
uses: actions-hub/kubectl@d50394b7d704525f93faefce1e65a6329ff67271 # v1.33.2
env:
KUBE_CONFIG: ${{ secrets.LIMSA_LOMINSA_KUBECONFIG }}
with:
args: apply -k docs/manifest
- name: Apply k8s manifests to aeacus
uses: actions-hub/kubectl@f632a31512a74cb35940627c49c20f67723cbaaf # v1.33.1
uses: actions-hub/kubectl@d50394b7d704525f93faefce1e65a6329ff67271 # v1.33.2
env:
KUBE_CONFIG: ${{ secrets.LIMSA_LOMINSA_KUBECONFIG }}
with:

View File

@@ -18,7 +18,7 @@ jobs:
persist-credentials: false
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Docker meta
id: meta

View File

@@ -25,7 +25,7 @@ jobs:
sudo apt-get install -y build-essential
- name: Set up Homebrew
uses: Homebrew/actions/setup-homebrew@master
uses: Homebrew/actions/setup-homebrew@main
- name: Setup Homebrew cellar cache
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3

View File

@@ -25,7 +25,7 @@ jobs:
sudo apt-get install -y build-essential
- name: Set up Homebrew
uses: Homebrew/actions/setup-homebrew@master
uses: Homebrew/actions/setup-homebrew@main
- name: Setup Homebrew cellar cache
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3

View File

@@ -27,7 +27,7 @@ jobs:
sudo apt-get install -y build-essential
- name: Set up Homebrew
uses: Homebrew/actions/setup-homebrew@master
uses: Homebrew/actions/setup-homebrew@main
- name: Setup Homebrew cellar cache
uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3

View File

@@ -30,7 +30,7 @@ jobs:
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0
uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1
- name: Build and push
run: |
cd ./test/ssh-ci

View File

@@ -21,7 +21,7 @@ jobs:
persist-credentials: false
- name: Install the latest version of uv
uses: astral-sh/setup-uv@f0ec1fc3b38f5e7cd731bb6ce540c5af426746bb # v6.1.0
uses: astral-sh/setup-uv@bd01e18f51369d5a26f1651c3cb451d3417e3bba # v6.3.1
- name: Run zizmor 🌈
run: uvx zizmor --format sarif . > results.sarif
@@ -29,7 +29,7 @@ jobs:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@ce28f5bb42b7a9f2c824e633a3f6ee835bab6858 # v3.29.0
uses: github/codeql-action/upload-sarif@39edc492dbe16b1465b0cafca41432d857bdb31a # v3.29.1
with:
sarif_file: results.sarif
category: zizmor

2
.gitignore vendored
View File

@@ -20,3 +20,5 @@ node_modules
# how does this get here
doc/VERSION
web/static/locales/*.json

10
.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,10 @@
{
"recommendations": [
"esbenp.prettier-vscode",
"ms-azuretools.vscode-containers",
"golang.go",
"unifiedjs.vscode-mdx",
"a-h.templ",
"redhat.vscode-yaml"
]
}

27
.vscode/launch.json vendored Normal file
View File

@@ -0,0 +1,27 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Launch Package",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${fileDirname}"
},
{
"name": "Anubis [dev]",
"command": "npm run dev",
"request": "launch",
"type": "node-terminal"
},
{
"name": "Start Docs",
"command": "cd docs && npm ci && npm run start",
"request": "launch",
"type": "node-terminal"
}
]
}

View File

@@ -18,6 +18,7 @@ assets: deps
build: assets
$(GO) build -o ./var/anubis ./cmd/anubis
$(GO) build -o ./var/robots2policy ./cmd/robots2policy
@echo "Anubis is now built to ./var/anubis"
lint: assets
@@ -27,6 +28,7 @@ lint: assets
prebaked-build:
$(GO) build -o ./var/anubis -ldflags "-X 'github.com/TecharoHQ/anubis.Version=$(VERSION)'" ./cmd/anubis
$(GO) build -o ./var/robots2policy -ldflags "-X 'github.com/TecharoHQ/anubis.Version=$(VERSION)'" ./cmd/robots2policy
test: assets
$(GO) test ./...

View File

@@ -9,6 +9,7 @@
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/TecharoHQ/anubis)
![language count](https://img.shields.io/github/languages/count/TecharoHQ/anubis)
![repo size](https://img.shields.io/github/repo-size/TecharoHQ/anubis)
[![GitHub Sponsors](https://img.shields.io/github/sponsors/Xe)](https://github.com/sponsors/Xe)
## Sponsors

View File

@@ -1 +1 @@
1.19.1
1.20.0

View File

@@ -11,12 +11,11 @@ var Version = "devel"
// CookieName is the name of the cookie that Anubis uses in order to validate
// access.
const CookieName = "techaro.lol-anubis-auth"
var CookieName = "techaro.lol-anubis-auth"
// WithDomainCookieName is the name that is prepended to the per-domain cookie used when COOKIE_DOMAIN is set.
const WithDomainCookieName = "techaro.lol-anubis-auth-for-"
const TestCookieName = "techaro.lol-anubis-cookie-test-if-you-block-this-anubis-wont-work"
// TestCookieName is the name of the cookie that Anubis uses in order to check
// if cookies are enabled on the client's browser.
var TestCookieName = "techaro.lol-anubis-cookie-verification"
// CookieDefaultExpirationTime is the amount of time before the cookie/JWT expires.
const CookieDefaultExpirationTime = 7 * 24 * time.Hour
@@ -33,3 +32,7 @@ const APIPrefix = "/.within.website/x/cmd/anubis/api/"
// DefaultDifficulty is the default "difficulty" (number of leading zeroes)
// that must be met by the client in order to pass the challenge.
const DefaultDifficulty = 4
// ForcedLanguage is the language being used instead of the one of the request's Accept-Language header
// if being set.
var ForcedLanguage = ""

View File

@@ -46,8 +46,13 @@ var (
bindNetwork = flag.String("bind-network", "tcp", "network family to bind HTTP to, e.g. unix, tcp")
challengeDifficulty = flag.Int("difficulty", anubis.DefaultDifficulty, "difficulty of the challenge")
cookieDomain = flag.String("cookie-domain", "", "if set, the top-level domain that the Anubis cookie will be valid for")
cookieDynamicDomain = flag.Bool("cookie-dynamic-domain", false, "if set, automatically set the cookie Domain value based on the request domain")
cookieExpiration = flag.Duration("cookie-expiration-time", anubis.CookieDefaultExpirationTime, "The amount of time the authorization cookie is valid for")
cookiePrefix = flag.String("cookie-prefix", "techaro.lol-anubis", "prefix for browser cookies created by Anubis")
cookiePartitioned = flag.Bool("cookie-partitioned", false, "if true, sets the partitioned flag on Anubis cookies, enabling CHIPS support")
forcedLanguage = flag.String("forced-language", "", "if set, this language is being used instead of the one from the request's Accept-Language header")
hs512Secret = flag.String("hs512-secret", "", "secret used to sign JWTs, uses ed25519 if not set")
cookieSecure = flag.Bool("cookie-secure", true, "if true, sets the secure flag on Anubis cookies")
ed25519PrivateKeyHex = flag.String("ed25519-private-key-hex", "", "private key used to sign JWTs, if not set a random one will be assigned")
ed25519PrivateKeyHexFile = flag.String("ed25519-private-key-hex-file", "", "file name containing value for ed25519-private-key-hex")
metricsBind = flag.String("metrics-bind", ":9090", "network address to bind metrics to")
@@ -105,8 +110,41 @@ func doHealthCheck() error {
return nil
}
// parseBindNetFromAddr determine bind network and address based on the given network and address.
func parseBindNetFromAddr(address string) (string, string) {
defaultScheme := "http://"
if !strings.Contains(address, "://") {
if strings.HasPrefix(address, ":") {
address = defaultScheme + "localhost" + address
} else {
address = defaultScheme + address
}
}
bindUri, err := url.Parse(address)
if err != nil {
log.Fatal(fmt.Errorf("failed to parse bind URL: %w", err))
}
switch bindUri.Scheme {
case "unix":
return "unix", bindUri.Path
case "tcp", "http", "https":
return "tcp", bindUri.Host
default:
log.Fatal(fmt.Errorf("unsupported network scheme %s in address %s", bindUri.Scheme, address))
}
return "", address
}
func setupListener(network string, address string) (net.Listener, string) {
formattedAddress := ""
if network == "" {
// keep compatibility
network, address = parseBindNetFromAddr(address)
}
switch network {
case "unix":
formattedAddress = "unix:" + address
@@ -239,6 +277,10 @@ func main() {
}
}
if *cookieDomain != "" && *cookieDynamicDomain {
log.Fatalf("you can't set COOKIE_DOMAIN and COOKIE_DYNAMIC_DOMAIN at the same time")
}
ctx := context.Background()
// Thoth configuration
@@ -290,11 +332,15 @@ func main() {
"this may result in unexpected behavior")
}
var priv ed25519.PrivateKey
if *ed25519PrivateKeyHex != "" && *ed25519PrivateKeyHexFile != "" {
var ed25519Priv ed25519.PrivateKey
if *hs512Secret != "" && (*ed25519PrivateKeyHex != "" || *ed25519PrivateKeyHexFile != "") {
log.Fatal("do not specify both HS512 and ED25519 secrets")
} else if *hs512Secret != "" {
ed25519Priv = ed25519.PrivateKey(*hs512Secret)
} else if *ed25519PrivateKeyHex != "" && *ed25519PrivateKeyHexFile != "" {
log.Fatal("do not specify both ED25519_PRIVATE_KEY_HEX and ED25519_PRIVATE_KEY_HEX_FILE")
} else if *ed25519PrivateKeyHex != "" {
priv, err = keyFromHex(*ed25519PrivateKeyHex)
ed25519Priv, err = keyFromHex(*ed25519PrivateKeyHex)
if err != nil {
log.Fatalf("failed to parse and validate ED25519_PRIVATE_KEY_HEX: %v", err)
}
@@ -304,12 +350,12 @@ func main() {
log.Fatalf("failed to read ED25519_PRIVATE_KEY_HEX_FILE %s: %v", *ed25519PrivateKeyHexFile, err)
}
priv, err = keyFromHex(string(bytes.TrimSpace(hexFile)))
ed25519Priv, err = keyFromHex(string(bytes.TrimSpace(hexFile)))
if err != nil {
log.Fatalf("failed to parse and validate content of ED25519_PRIVATE_KEY_HEX_FILE: %v", err)
}
} else {
_, priv, err = ed25519.GenerateKey(rand.Reader)
_, ed25519Priv, err = ed25519.GenerateKey(rand.Reader)
if err != nil {
log.Fatalf("failed to generate ed25519 key: %v", err)
}
@@ -331,22 +377,36 @@ func main() {
slog.Warn("REDIRECT_DOMAINS is not set, Anubis will only redirect to the same domain a request is coming from, see https://anubis.techaro.lol/docs/admin/configuration/redirect-domains")
}
anubis.CookieName = *cookiePrefix + "-auth"
anubis.TestCookieName = *cookiePrefix + "-cookie-verification"
anubis.ForcedLanguage = *forcedLanguage
// If OpenGraph configuration values are not set in the config file, use the
// values from flags / envvars.
if !policy.OpenGraph.Enabled {
policy.OpenGraph.Enabled = *ogPassthrough
policy.OpenGraph.ConsiderHost = *ogCacheConsiderHost
policy.OpenGraph.TimeToLive = *ogTimeToLive
policy.OpenGraph.Override = map[string]string{}
}
s, err := libanubis.New(libanubis.Options{
BasePrefix: *basePrefix,
StripBasePrefix: *stripBasePrefix,
Next: rp,
Policy: policy,
ServeRobotsTXT: *robotsTxt,
PrivateKey: priv,
CookieDomain: *cookieDomain,
CookieExpiration: *cookieExpiration,
CookiePartitioned: *cookiePartitioned,
OGPassthrough: *ogPassthrough,
OGTimeToLive: *ogTimeToLive,
RedirectDomains: redirectDomainsList,
Target: *target,
WebmasterEmail: *webmasterEmail,
OGCacheConsidersHost: *ogCacheConsiderHost,
BasePrefix: *basePrefix,
StripBasePrefix: *stripBasePrefix,
Next: rp,
Policy: policy,
ServeRobotsTXT: *robotsTxt,
ED25519PrivateKey: ed25519Priv,
HS512Secret: []byte(*hs512Secret),
CookieDomain: *cookieDomain,
CookieDynamicDomain: *cookieDynamicDomain,
CookieExpiration: *cookieExpiration,
CookiePartitioned: *cookiePartitioned,
RedirectDomains: redirectDomainsList,
Target: *target,
WebmasterEmail: *webmasterEmail,
OpenGraph: policy.OpenGraph,
CookieSecure: *cookieSecure,
})
if err != nil {
log.Fatalf("can't construct libanubis.Server: %v", err)

View File

@@ -56,7 +56,7 @@ bots:
- name: countries-with-aggressive-scrapers
action: WEIGH
geoip:
counties:
countries:
- BR
- CN
weight:
@@ -84,6 +84,59 @@ bots:
dnsbl: false
# #
# impressum:
# # Displayed at the bottom of every page rendered by Anubis.
# footer: >-
# This website is hosted by Zombocom. If you have any complaints or notes
# about the service, please contact
# <a href="mailto:contact@domainhere.example">contact@domainhere.example</a>
# and we will assist you as soon as possible.
# # The imprint page that will be linked to at the footer of every Anubis page.
# page:
# # The HTML <title> of the page
# title: Imprint and Privacy Policy
# # The HTML contents of the page. The exact contents of this page can
# # and will vary by locale. Please consult with a lawyer if you are not
# # sure what to put here
# body: >-
# <p>Last updated: June 2025</p>
# <h2>Information that is gathered from visitors</h2>
# <p>In common with other websites, log files are stored on the web server saving details such as the visitor's IP address, browser type, referring page and time of visit.</p>
# <p>Cookies may be used to remember visitor preferences when interacting with the website.</p>
# <p>Where registration is required, the visitor's email and a username will be stored on the server.</p>
# <!-- ... -->
# Open Graph passthrough configuration, see here for more information:
# https://anubis.techaro.lol/docs/admin/configuration/open-graph/
openGraph:
# Enables Open Graph passthrough
enabled: false
# Enables the use of the HTTP host in the cache key, this enables
# caching metadata for multiple http hosts at once.
considerHost: false
# How long cached OpenGraph metadata should last in memory
ttl: 24h
# # If set, return these opengraph values instead of looking them up with
# # the target service.
# #
# # Correlates to properties in https://ogp.me/
# override:
# # og:title is required, it is the title of the website
# "og:title": "Techaro Anubis"
# "og:description": >-
# Anubis is a Web AI Firewall Utility that helps you fight the bots
# away so that you can maintain uptime at work!
# "description": >-
# Anubis is a Web AI Firewall Utility that helps you fight the bots
# away so that you can maintain uptime at work!
# By default, send HTTP 200 back to clients that either get issued a challenge
# or a denial. This seems weird, but this is load-bearing due to the fact that
# the most aggressive scraper bots seem to really, really, want an HTTP 200 and
@@ -91,3 +144,57 @@ dnsbl: false
status_codes:
CHALLENGE: 200
DENY: 200
# The weight thresholds for when to trigger individual challenges. Any
# CHALLENGE will take precedence over this.
#
# A threshold has four configuration options:
#
# - name: the name that is reported down the stack and used for metrics
# - expression: A CEL expression with the request weight in the variable
# weight
# - action: the Anubis action to apply, similar to in a bot policy
# - challenge: which challenge to send to the user, similar to in a bot policy
#
# See https://anubis.techaro.lol/docs/admin/configuration/thresholds for more
# information.
thresholds:
# By default Anubis ships with the following thresholds:
- name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
expression: weight <= 0 # a feather weighs zero units
action: ALLOW # Allow the traffic through
# For clients that had some weight reduced through custom rules, give them a
# lightweight challenge.
- name: mild-suspicion
expression:
all:
- weight > 0
- weight < 10
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
algorithm: metarefresh
difficulty: 1
report_as: 1
# For clients that are browser-like but have either gained points from custom rules or
# report as a standard browser.
- name: moderate-suspicion
expression:
all:
- weight >= 10
- weight < 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 2 # two leading zeros, very fast for most clients
report_as: 2
# For clients that are browser like and have gained many points from custom rules
- name: extreme-suspicion
expression: weight >= 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 4
report_as: 4

View File

@@ -2,5 +2,5 @@
# Note: Blocks human-directed/non-training user agents
- name: "ai-robots-txt"
user_agent_regex: >-
AI2Bot|Ai2Bot-Dolma|aiHitBot|Amazonbot|Andibot|anthropic-ai|Applebot|Applebot-Extended|bedrockbot|Brightbot 1.0|Bytespider|CCBot|ChatGPT-User|Claude-SearchBot|Claude-User|Claude-Web|ClaudeBot|cohere-ai|cohere-training-data-crawler|Cotoyogi|Crawlspace|Diffbot|DuckAssistBot|FacebookBot|Factset_spyderbot|FirecrawlAgent|FriendlyCrawler|Google-CloudVertexBot|Google-Extended|GoogleOther|GoogleOther-Image|GoogleOther-Video|GPTBot|iaskspider/2.0|ICC-Crawler|ImagesiftBot|img2dataset|ISSCyberRiskCrawler|Kangaroo Bot|meta-externalagent|Meta-ExternalAgent|meta-externalfetcher|Meta-ExternalFetcher|MistralAI-User/1.0|NovaAct|OAI-SearchBot|omgili|omgilibot|Operator|PanguBot|Panscient|panscient.com|Perplexity-User|PerplexityBot|PetalBot|PhindBot|QualifiedBot|QuillBot|quillbot.com|SBIntuitionsBot|Scrapy|SemrushBot-OCOB|SemrushBot-SWA|Sidetrade indexer bot|TikTokSpider|Timpibot|VelenPublicWebCrawler|Webzio-Extended|wpbot|YandexAdditional|YandexAdditionalBot|YouBot
AI2Bot|Ai2Bot-Dolma|aiHitBot|Amazonbot|Andibot|anthropic-ai|Applebot|Applebot-Extended|bedrockbot|Brightbot 1.0|Bytespider|CCBot|ChatGPT-User|Claude-SearchBot|Claude-User|Claude-Web|ClaudeBot|cohere-ai|cohere-training-data-crawler|Cotoyogi|Crawlspace|Diffbot|DuckAssistBot|EchoboxBot|FacebookBot|facebookexternalhit|Factset_spyderbot|FirecrawlAgent|FriendlyCrawler|Google-CloudVertexBot|Google-Extended|GoogleOther|GoogleOther-Image|GoogleOther-Video|GPTBot|iaskspider/2.0|ICC-Crawler|ImagesiftBot|img2dataset|ISSCyberRiskCrawler|Kangaroo Bot|meta-externalagent|Meta-ExternalAgent|meta-externalfetcher|Meta-ExternalFetcher|MistralAI-User/1.0|MyCentralAIScraperBot|NovaAct|OAI-SearchBot|omgili|omgilibot|Operator|PanguBot|Panscient|panscient.com|Perplexity-User|PerplexityBot|PetalBot|PhindBot|Poseidon Research Crawler|QualifiedBot|QuillBot|quillbot.com|SBIntuitionsBot|Scrapy|SemrushBot|SemrushBot-BA|SemrushBot-CT|SemrushBot-OCOB|SemrushBot-SI|SemrushBot-SWA|Sidetrade indexer bot|TikTokSpider|Timpibot|VelenPublicWebCrawler|Webzio-Extended|wpbot|YandexAdditional|YandexAdditionalBot|YouBot
action: DENY

View File

@@ -0,0 +1,248 @@
---
slug: release/v1.20.0
title: Anubis v1.20.0 is now available!
authors: [xe]
tags: [release]
image: sunburst.webp
---
![](./sunburst.webp)
Hey all!
Today we released [Anubis v1.20.0: Thancred Waters](https://github.com/TecharoHQ/anubis/releases/tag/v1.20.0). This adds a lot of new and exciting features to Anubis, including but not limited to the `WEIGH` action, custom weight thresholds, Imprint/impressum support, and a no-JS challenge. Here's what you need to know so you can protect your websites in new and exciting ways!
{/* truncate */}
## Sponsoring the product
If you rely on Anubis to keep your website safe, please consider sponsoring the project on [GitHub Sponsors](https://github.com/sponsors/Xe) or [Patreon](https://patreon.com/cadey). Funding helps pay hosting bills and offset the time spent on making this project the best it can be. Every little bit helps and when enough money is raised, [I can make Anubis my full-time job](https://github.com/TecharoHQ/anubis/discussions/278).
I am waiting to hear back from NLNet on if Anubis was selected for funding or not. Let's hope it is!
## Deprecation warning: `DEFAULT_DIFFICULTY`
Anubis v1.20.0 is the last version to support the `DEFAULT_DIFFICULTY` flag in the exact way it currently does. In future versions, this will be ineffectual and you should use the [custom threshold system](/docs/admin/configuration/thresholds) instead.
If this becomes an imposition in practice, this will be reverted.
## Chrome won't show "invalid response" after "Success!"
There were a bunch of smaller fixes in Anubis this time around, but the biggest one was finally squashing the ["invalid response" after "Success!" issue](https://github.com/TecharoHQ/anubis/issues/564) that had been plaguing Chrome users. This was a really annoying issue to track down but it was discovered while we were working on better end-to-end / functional testing: [Chrome randomizes the `Accept-Language` header](https://github.com/explainers-by-googlers/reduce-accept-language) so that websites can't do fingerprinting as easily.
When Anubis issues a challenge, it grabs [information that the browser sends to the user](/docs/design/how-anubis-works#challenge-format) to create a challenge string. Anubis doesn't store these challenge strings anywhere, and when a solution is being checked it calculates the challenge string from the request. This means that they'd get a challenge on one end, compute the response for that challenge, and then the server would validate that against a different challenge. This server-side validation would fail, leading to the user seeing "invalid response" after the client reported success.
I suspect this was why Vanadium and Cromite were having sporadic issues as well.
## New Features
The biggest feature in Anubis is the "weight" subsystem. This allows administrators to make custom rules that change the suspicion level of a request without having to take immediate action. As an example, consider the self-hostable git forge [Gitea](https://about.gitea.com/). When you load a page in Gitea, it creates a session cookie that your browser sends with every request. Weight allows you to mark a request that includes a Gitea session token as _less_ suspicious:
```yaml
- name: gitea-session-token
action: WEIGH
expression:
all:
# Check if the request has a Cookie header
- '"Cookie" in headers'
# Check if the request's Cookie header contains the Gitea session token
- headers["Cookie"].contains("i_love_gitea=")
# Remove 5 weight points
weight:
adjust: -5
```
This is different from the past where you could only allow every request with a Gitea session token, meaning that the invention of lying would allow malicious clients to bypass protection.
Weight is added and removed whenever a `WEIGH` rule is encountered. When all rules are processed and the request doesn't match any `ALLOW`, `CHALLENGE`, or `DENY` rules, Anubis uses [weight thresholds](/docs/admin/configuration/thresholds) to figure out how to handle that request. Thresholds are defined in the [policy file](/docs/admin/policies) alongside your bot rules:
```yaml
thresholds:
- name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
expression: weight <= 0 # a feather weighs zero units
action: ALLOW # Allow the traffic through
# For clients that had some weight reduced through custom rules, give them a
# lightweight challenge.
- name: mild-suspicion
expression:
all:
- weight > 0
- weight < 10
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
algorithm: metarefresh
difficulty: 1
report_as: 1
# For clients that are browser-like but have either gained points from custom rules or
# report as a standard browser.
- name: moderate-suspicion
expression:
all:
- weight >= 10
- weight < 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 2 # two leading zeros, very fast for most clients
report_as: 2
# For clients that are browser like and have gained many points from custom rules
- name: extreme-suspicion
expression: weight >= 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 4
report_as: 4
```
:::note
If you don't have thresholds defined in your Anubis policy file, Anubis will default to the "legacy" behaviour where browser-like clients get a challenge at the default difficulty.
:::
This lets most clients through if they pass a simple [proof of work challenge](/docs/admin/configuration/challenges/proof-of-work), but any clients that are less suspicious (like ones with a Gitea session token) are given the lightweight [Meta Refresh](/docs/admin/configuration/challenges/metarefresh) challenge instead.
Threshold expressions are like [Bot rule expressions](/docs/admin/configuration/expressions), but there's only one input: the request's weight. If no thresholds match, the request is allowed through.
### Imprint/Impressum Support
European countries like Germany [require an imprint/impressum](https://www.ionos.com/digitalguide/websites/digital-law/a-case-for-thinking-global-germanys-impressum-laws/) to be present in the footer of their website. This allows users to contact someone on the team behind a website in case they run into issues. This also must generally have a separate page where users can view an extended imprint with other information like a privacy policy or a copyright notice.
Anubis v1.20.0 and later [has support for showing imprints](/docs/admin/configuration/impressum). You can configure two kinds of imprints:
1. An imprint that is shown in the footer of every Anubis page.
2. An extended imprint / privacy policy that is shown when users click on the "Imprint" link. For example, [here's the imprint for the website you're looking at right now](https://anubis.techaro.lol/.within.website/x/cmd/anubis/api/imprint).
Imprints are configured in [the policy file](/docs/admin/policies/):
```yaml
impressum:
# Displayed at the bottom of every page rendered by Anubis.
footer: >-
This website is hosted by Zombocom. If you have any complaints or notes
about the service, please contact
<a href="mailto:contact@zombocom.example">contact@zombocom.example</a> and
we will assist you as soon as possible.
# The imprint page that will be linked to at the footer of every Anubis page.
page:
# The HTML <title> of the page
title: Imprint and Privacy Policy
# The HTML contents of the page. The exact contents of this page can
# and will vary by locale. Please consult with a lawyer if you are not
# sure what to put here.
body: >-
<p>Last updated: June 2025</p>
<h2>Information that is gathered from visitors</h2>
<p>In common with other websites, log files are stored on the web server
saving details such as the visitor's IP address, browser type, referring
page and time of visit.</p>
<p>Cookies may be used to remember visitor preferences when interacting
with the website.</p>
<p>Where registration is required, the visitor's email and a username
will be stored on the server.</p>
<!-- ... -->
```
If this is insufficient, please [file an issue](https://github.com/TecharoHQ/anubis/issues/new) with a link to the relevant legislation for your country so that this feature can be amended and improved.
### No-JS Challenge
One of the first issues in Anubis before it was moved to the [TecharoHQ org](https://github.com/TecharoHQ) was a request [to support challenging browsers without using JavaScript](https://github.com/Xe/x/issues/651). This is a pretty challenging thing to do without rethinking how Anubis works from a fundamentally low level, and with v1.20.0, [Anubis finally has support for running without client-side JavaScript](https://github.com/TecharoHQ/anubis/issues/95) thanks to the [Meta Refresh](/docs/admin/configuration/challenges/metarefresh) challenge.
When Anubis decides it needs to send a challenge to your browser, it sends a challenge page. Historically, this challenge page is [an HTML template](https://github.com/TecharoHQ/anubis/blob/main/web/index.templ) that kicks off some JavaScript, reads the challenge information out of the page body, and then solves it as fast as possible in order to let users see the website they want to visit.
In v1.20.0, Anubis has a challenge registry to hold [different client challenge implementations](/docs/category/challenges). This allows us to implement anything we want as long as it can render a page to show a challenge and then check if the result is correct. This is going to be used to implement a WebAssembly-based proof of work option (one that will be way more efficient than the existing browser JS version), but as a proof of concept I implemented a simple challenge using [HTML `<meta refresh>`](https://en.wikipedia.org/wiki/Meta_refresh).
In my testing, this has worked with every browser I have thrown it at (including CLI browsers, the browser embedded in emacs, etc.). The default configuration of Anubis does use the [meta refresh challenge](/docs/admin/configuration/challenges/metarefresh) for [clients with a very low suspicion](/docs/admin/configuration/thresholds), but by default clients will be sent an [easy proof of work challenge](/docs/admin/configuration/challenges/proof-of-work).
If the false positive rate of this challenge turns out to not be very high in practice, the meta refresh challenge will be enabled by default for browsers in future versions of Anubis.
### `robots2policy`
Anubis was created because crawler bots don't respect [`robots.txt` files](https://www.robotstxt.org/). Administrators have been working on refining and crafting their `robots.txt` files for years, and one common comment is that people don't know where to start crafting their own rules.
Anubis now ships with a [`robots2policy` tool](/docs/admin/robots2policy) that lets you convert your `robots.txt` file to an Anubis policy.
```text
robots2policy -input https://github.com/robots.txt
```
:::note
If you installed Anubis from [an OS package](/docs/admin/native-install), you may need to run `anubis-robots2policy` instead of `robots2policy`.
:::
We hope that this will help you get started with Anubis faster. We are working on a version of this that will run in the documentation via WebAssembly.
### Open Graph configuration is being moved to the policy file
Anubis supports reading [Open Graph tags](/docs/admin/configuration/open-graph) from target services and returning them in challenge pages. This makes the right metadata show up when linking services protected by Anubis in chat applications or on social media.
In order to test the migration of all of the configuration to the policy file, Open Graph configuration has been moved to the policy file. For more information, please read [the Open Graph configuration options](/docs/admin/configuration/open-graph#configuration-options).
You can also set default Open Graph tags:
```yaml
openGraph:
enabled: true
ttl: 24h
# If set, return these opengraph values instead of looking them up with
# the target service.
#
# Correlates to properties in https://ogp.me/
override:
# og:title is required, it is the title of the website
"og:title": "Techaro Anubis"
"og:description": >-
Anubis is a Web AI Firewall Utility that helps you fight the bots
away so that you can maintain uptime at work!
"description": >-
Anubis is a Web AI Firewall Utility that helps you fight the bots
away so that you can maintain uptime at work!
```
## Improvements and optimizations
One of the biggest improvements we've made in v1.20.0 is replacing [SHA-256 with xxhash](https://github.com/TecharoHQ/anubis/pull/676). Anubis uses hashes all over the place to help with identifying clients, matching against rules when allowing traffic through, in error messages sent to users, and more. Historically these have been done with [SHA-256](https://en.wikipedia.org/wiki/SHA-2), however this has been having a mild performance impact in real-world use. As a result, we now use [xxhash](https://xxhash.com/) when possible. This makes policy matching 3x faster in some scenarios and reduces memory usage across the board.
Anubis now uses [bart](https://pkg.go.dev/github.com/gaissmai/bart) for doing IP address matching when you specify addresses in a `remote_address` check configuration or when you are matching against [advanced checks](/docs/admin/thoth). This uses the same kind of IP address routing configuration that your OS kernel does, making it very fast to query information about IP addresses. This makes IP address range matches anywhere from 3-14 times faster depending on the number of addresses it needs to match against. For more information and benchmarks, check out [@JasonLovesDoggo](https://github.com/JasonLovesDoggo)'s PR: [perf: replace cidranger with bart for significant performance improvements #675](https://github.com/TecharoHQ/anubis/pull/675).
## What's up next?
v1.21.0 is already shaping up to be a massive improvement as Anubis adds [internationalization](https://en.wikipedia.org/wiki/Internationalization) support, allowing your users to see its messages in the language they're most comfortable with.
So far Anubis supports the following languages:
- English (Simplified and Traditional)
- French
- Portugese (Brazil)
- Spanish
If you want to contribute translations, please [file an issue](https://github.com/TecharoHQ/anubis/issues/new) with your language of choice or submit a pull request to [the `lib/localization/locales` folder](https://github.com/TecharoHQ/anubis/tree/main/lib/localization/locales). We are about to introduce features to the translation stack, so you may want to hold off a hot minute, but we welcome any and all contributions to making Anubis useful to a global audience.
Other things we plan to do:
- Move configuration to the policy file
- Support reloading the policy file at runtime without having to restart Anubis
- Detecting if a client is "brand new"
- A [Valkey](https://valkey.io/)-backed store for sharing information between instances of Anubis
- Augmenting No-JS support in the paid product
- TLS fingerprinting
- Automated testing improvements in CI (FreeBSD CI support, better automated integration/functional testing, etc.)
## Conclusion
I hope that these features let you get the same Anubis power you've come to know and love and increases the things you can do with it! I've been really excited to ship [thresholds](/docs/admin/configuration/thresholds) and the cloud-based services for Anubis.
If you run into any problems, please [file an issue](https://github.com/TecharoHQ/anubis/issues/new). Otherwise, have a good day and get back to making your communities great.

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 KiB

View File

@@ -1 +0,0 @@

View File

@@ -11,22 +11,170 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
- Replace cidranger with bart for IP range checking, improving IP matching performance by 3-20x with zero heap
<!-- This changes the project to: -->
- Add `COOKIE_SECURE` option to set the cookie [Secure flag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Cookies#block_access_to_your_cookies)
- Sets cookie defaults to use [SameSite: None](https://web.dev/articles/samesite-cookies-explained)
- Determine the `BIND_NETWORK`/`--bind-network` value from the bind address ([#677](https://github.com/TecharoHQ/anubis/issues/677)).
- Implement localization system. Find locale files in lib/localization/locales/.
- Implement a [development container](https://containers.dev/) manifest to make contributions easier.
- Fix dynamic cookie domains functionality ([#731](https://github.com/TecharoHQ/anubis/pull/731))
- Add option for custom cookie prefix ([#732](https://github.com/TecharoHQ/anubis/pull/732))
- Remove the "Success" interstitial after a proof of work challenge is concluded.
- Add option for forcing a specific language ([#742](https://github.com/TecharoHQ/anubis/pull/742))
### Potentially breaking changes
The following potentially breaking change applies to native installs with systemd only:
Each instance of systemd service template now has a unique `RuntimeDirectory`, as opposed to each instance of the service sharing a `RuntimeDirectory`. This change was made to avoid [the `RuntimeDirectory` getting nuked any time one of the Anubis instances restarts](https://github.com/TecharoHQ/anubis/issues/748).
If you configured Anubis' unix sockets to listen on `/run/anubis/foo.sock` for instance `anubis@foo`, you will need to configure Anubis to listen on `/run/anubis/foo/sock` and additionally configure your HTTP load balancer as appropriate.
If you need the legacy behaviour, install this [systemd unit dropin](https://www.flatcar.org/docs/latest/setup/systemd/drop-in-units/):
```systemd
# /etc/systemd/system/anubis@.service.d/50-runtimedir.conf
[Service]
RuntimeDirectory=anubis
```
## v1.20.0: Thancred Waters
The big ticket items are as follows:
- Implement a no-JS challenge method: [`metarefresh`](./admin/configuration/challenges/metarefresh.mdx) ([#95](https://github.com/TecharoHQ/anubis/issues/95))
- Implement request "weight", allowing administrators to customize the behaviour of Anubis based on specific criteria
- Implement GeoIP and ASN based checks via [Thoth](https://anubis.techaro.lol/docs/admin/thoth) ([#206](https://github.com/TecharoHQ/anubis/issues/206))
- Add [custom weight thresholds](./admin/configuration/thresholds.mdx) via CEL ([#688](https://github.com/TecharoHQ/anubis/pull/688))
- Move Open Graph configuration [to the policy file](./admin/configuration/open-graph.mdx)
- Enable support for Open Graph metadata to be returned by default instead of doing lookups against the target
- Add `robots2policy` CLI utility to convert robots.txt files to Anubis challenge policies using CEL expressions ([#409](https://github.com/TecharoHQ/anubis/issues/409))
- Refactor challenge presentation logic to use a challenge registry
- Allow challenge implementations to register HTTP routes
- [Imprint/Impressum support](./admin/configuration/impressum.mdx) ([#362](https://github.com/TecharoHQ/anubis/issues/362))
- Fix "invalid response" after "Success!" in Chromium ([#564](https://github.com/TecharoHQ/anubis/issues/564))
A lot of performance improvements have been made:
- Replace internal SHA256 hashing with xxhash for 4-6x performance improvement in policy evaluation and cache operations
- Optimized the OGTags subsystem with reduced allocations and runtime per request by up to 66%
- Replace cidranger with bart for IP range checking, improving IP matching performance by 3-20x with zero heap
allocations
And some cleanups/refactors were added:
- Fix OpenGraph passthrough ([#717](https://github.com/TecharoHQ/anubis/issues/717))
- Remove the unused `/test-error` endpoint and update the testing endpoint `/make-challenge` to only be enabled in
development
- Add `--xff-strip-private` flag/envvar to toggle skipping X-Forwarded-For private addresses or not
- Requests can have their weight be adjusted, if a request weighs zero or less than it is allowed through
- Refactor challenge presentation logic to use a challenge registry
- Allow challenge implementations to register HTTP routes
- Implement a no-JS challenge method: [`metarefresh`](./admin/configuration/challenges/metarefresh.mdx) ([#95](https://github.com/TecharoHQ/anubis/issues/95))
- Bump AI-robots.txt to version 1.34
- Bump AI-robots.txt to version 1.37
- Make progress bar styling more compatible (UXP, etc)
- Optimized the OGTags subsystem with reduced allocations and runtime per request by up to 66%
- Add `--strip-base-prefix` flag/envvar to strip the base prefix from request paths when forwarding to target servers
- Add `robots2policy` CLI utility to convert robots.txt files to Anubis challenge policies using CEL expressions ([#409](https://github.com/TecharoHQ/anubis/issues/409))
- Implement GeoIP and ASN based checks via [Thoth](https://anubis.techaro.lol/docs/admin/thoth) ([#206](https://github.com/TecharoHQ/anubis/issues/206))
- Replace internal SHA256 hashing with xxhash for 4-6x performance improvement in policy evaluation and cache operations
- Fix an off-by-one in the default threshold config
- Add functionality for HS512 JWT algorithm
- Add support for dynamic cookie domains with the `--cookie-dynamic-domain`/`COOKIE_DYNAMIC_DOMAIN` flag/envvar
Request weight is one of the biggest ticket features in Anubis. This enables Anubis to be much closer to a Web Application Firewall and when combined with custom thresholds allows administrators to have Anubis take advanced reactions. For more information about request weight, see [the request weight section](./admin/policies.mdx#request-weight) of the policy file documentation.
TL;DR when you have one or more WEIGHT rules like this:
```yaml
bots:
- name: gitea-session-token
action: WEIGH
expression:
all:
- '"Cookie" in headers'
- headers["Cookie"].contains("i_love_gitea=")
# Remove 5 weight points
weight:
adjust: -5
```
You can configure custom thresholds like this:
```yaml
thresholds:
- name: minimal-suspicion # This client is likely fine, its soul is lighter than a feather
expression: weight < 0 # a feather weighs zero units
action: ALLOW # Allow the traffic through
# For clients that had some weight reduced through custom rules, give them a
# lightweight challenge.
- name: mild-suspicion
expression:
all:
- weight >= 0
- weight < 10
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh
algorithm: metarefresh
difficulty: 1
report_as: 1
# For clients that are browser-like but have either gained points from custom
# rules or report as a standard browser.
- name: moderate-suspicion
expression:
all:
- weight >= 10
- weight < 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 2 # two leading zeros, very fast for most clients
report_as: 2
# For clients that are browser like and have gained many points from custom
# rules
- name: extreme-suspicion
expression: weight >= 20
action: CHALLENGE
challenge:
# https://anubis.techaro.lol/docs/admin/configuration/challenges/proof-of-work
algorithm: fast
difficulty: 4
report_as: 4
```
These thresholds apply when no other `ALLOW`, `DENY`, or `CHALLENGE` rule matches the request. `WEIGHT` rules add and remove request weight as needed:
```yaml
bots:
- name: gitea-session-token
action: WEIGH
expression:
all:
- '"Cookie" in headers'
- headers["Cookie"].contains("i_love_gitea=")
# Remove 5 weight points
weight:
adjust: -5
- name: bot-like-user-agent
action: WEIGH
expression: '"Bot" in userAgent'
# Add 5 weight points
weight:
adjust: 5
```
Of note: the default "generic browser" rule assigns 10 weight points:
```yaml
# Generic catchall rule
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: WEIGH
weight:
adjust: 10
```
Adjust this as you see fit.
## v1.19.1: Jenomis cen Lexentale - Echo 1

View File

@@ -0,0 +1,70 @@
# Imprint / Impressum configuration
Some jurisdictions (such as the European Union and specifically Germany) [must have contact information freely available](https://www.privacycompany.eu/blog/the-imprint-requirement-a-must-have-for-companies-from-outside-germany) on an imprint/impressum page. Anubis supports creating an Anubis-specific imprint page for your organization with the `impressum` block in your bot policy file. For example:
```yaml
impressum:
# Displayed at the bottom of every page rendered by Anubis.
footer: >-
This website is hosted by Techaro. If you have any complaints or notes
about the service, please contact
<a href="mailto:contact@techaro.lol">contact@techaro.lol</a> and we
will assist you as soon as possible.
# The imprint page that will be linked to at the footer of every Anubis page.
page:
# The HTML <title> of the page
title: Imprint and Privacy Policy
# The HTML contents of the page. The exact contents of this page can
# and will vary by locale. Please consult with a lawyer if you are not
# sure what to put here
body: >-
<p>Last updated: June 2025</p>
<h2>Information that is gathered from visitors</h2>
<p>In common with other websites, log files are stored on the web server saving details such as the visitor's IP address, browser type, referring page and time of visit.</p>
<p>Cookies may be used to remember visitor preferences when interacting with the website.</p>
<p>Where registration is required, the visitor's email and a username will be stored on the server.</p>
<!-- ... -->
```
If you are subscribed to and using [advanced classification features](../thoth.mdx), be sure to disclose the following:
```html
<h2>Techaro Anubis</h2>
<p>
This website uses a service called
<a href="https://anubis.techaro.lol">Anubis</a> by
<a href="https://techaro.lol">Techaro</a> to filter malicious traffic. Anubis
requires the use of browser cookies to ensure that web clients are running
conformant software. Anubis also may report the following data to Techaro to
improve service quality:
</p>
<ul>
<li>
IP address (for purposes of matching against geo-location and BGP autonomous
systems numbers), which is stored in-memory and not persisted to disk.
</li>
<li>
Unique browser fingerprints (such as HTTP request fingerprints and
encryption system fingerprints), which may be stored on Techaro's side for a
period of up to one month.
</li>
<li>
HTTP request metadata that may include things such as the User-Agent header
and other identifiers.
</li>
</ul>
<p>
This data is processed and stored for the legitimate interest of combatting
abusive web clients. This data is encrypted at rest as much as possible and is
only decrypted in memory for the purposes of fulfilling requests.
</p>
```

View File

@@ -9,12 +9,45 @@ This page provides detailed information on how to configure [Open Graph tag](htt
## Configuration Options
Open Graph settings are configured in the `openGraph` section of the [Policy File](../policies.mdx).
```yaml
openGraph:
# Enables Open Graph passthrough
enabled: true
# Enables the use of the HTTP host in the cache key, this enables
# caching metadata for multiple http hosts at once.
considerHost: true
# How long cached OpenGraph metadata should last in memory
ttl: 24h
# If set, return these opengraph values instead of looking them up with
# the target service.
#
# Correlates to properties in https://ogp.me/
override:
# og:title is required, it is the title of the website
"og:title": "Techaro Anubis"
"og:description": >-
Anubis is a Web AI Firewall Utility that helps you fight the bots
away so that you can maintain uptime at work!
"description": >-
Anubis is a Web AI Firewall Utility that helps you fight the bots
away so that you can maintain uptime at work!
```
<details>
<summary>Configuration flags / envvars (old)</summary>
Open Graph passthrough used to be configured with configuration flags / environment variables. Reference to these settings are maintained for backwards compatibility's sake.
| Name | Description | Type | Default | Example |
| ------------------------ | --------------------------------------------------------- | -------- | ------- | ----------------------------- |
| `OG_PASSTHROUGH` | Enables or disables the Open Graph tag passthrough system | Boolean | `true` | `OG_PASSTHROUGH=true` |
| `OG_EXPIRY_TIME` | Configurable cache expiration time for Open Graph tags | Duration | `24h` | `OG_EXPIRY_TIME=1h` |
| `OG_CACHE_CONSIDER_HOST` | Enables or disables the use of the host in the cache key | Boolean | `false` | `OG_CACHE_CONSIDER_HOST=true` |
</details>
## Usage
To configure Open Graph tags, you can set the following environment variables, environment file or as flags in your Anubis configuration:

View File

@@ -0,0 +1,140 @@
# Weight Threshold Configuration
Anubis offers the ability to assign "weight" to requests. This is a custom level of suspicion that rules can add to or remove from. For example, here's how you assign 10 weight points to anything that might be a browser:
```yaml
# botPolicies.yaml
bots:
- name: generic-browser
user_agent_regex: >-
Mozilla|Opera
action: WEIGH
weight:
adjust: 10
```
Thresholds let you take this per-request weight value and take actions in response to it. Thresholds are defined alongside your bot configuration in `botPolicies.yaml`.
:::note
Thresholds DO NOT apply when a request matches a bot rule with the CHALLENGE action. Thresholds only apply when requests don't match any terminal bot rules.
:::
```yaml
# botPolicies.yaml
bots: ...
thresholds:
- name: minimal-suspicion
expression: weight < 0
action: ALLOW
- name: mild-suspicion
expression:
all:
- weight >= 0
- weight < 10
action: CHALLENGE
challenge:
algorithm: metarefresh
difficulty: 1
report_as: 1
- name: moderate-suspicion
expression:
all:
- weight >= 10
- weight < 20
action: CHALLENGE
challenge:
algorithm: fast
difficulty: 2
report_as: 2
- name: extreme-suspicion
expression: weight >= 20
action: CHALLENGE
challenge:
algorithm: fast
difficulty: 4
report_as: 4
```
This defines a suite of 4 thresholds:
1. If the request weight is less than zero, allow it through.
2. If the request weight is greater than or equal to zero, but less than ten: give it [a very lightweight challenge](./challenges/metarefresh.mdx).
3. If the request weight is greater than or equal to ten, but less than twenty: give it [a slightly heavier challenge](./challenges/proof-of-work.mdx).
4. Otherwise, give it [the heaviest challenge](./challenges/proof-of-work.mdx).
Thresholds can be configured with the following options:
<table>
<thead>
<tr>
<th>Name</th>
<th>Description</th>
<th>Example</th>
</tr>
</thead>
<tbody>
<tr>
<td>`name`</td>
<td>The human-readable name for this threshold.</td>
<td>
```yaml
name: extreme-suspicion
```
</td>
</tr>
<tr>
<td>`expression`</td>
<td>A [CEL](https://cel.dev/) expression taking the request weight and returning true or false</td>
<td>
To check if the request weight is less than zero:
```yaml
expression: weight < 0
```
To check if it's between 0 and 10 (inclusive):
```yaml
expression:
all:
- weight >= 0
- weight < 10
```
</td>
</tr>
<tr>
<td>`action`</td>
<td>The Anubis action to apply: `ALLOW`, `CHALLENGE`, or `DENY`</td>
<td>
```yaml
action: ALLOW
```
If you set the CHALLENGE action, you must set challenge details:
```yaml
action: CHALLENGE
challenge:
algorithm: metarefresh
difficulty: 1
report_as: 1
```
</td>
</tr>
</tbody>
</table>

View File

@@ -34,27 +34,6 @@ These examples assume that you are using a setup where your nginx configuration
:::
## Dependencies
Install the following dependencies for proxying HTTP:
<Tabs>
<TabItem value="rpm" label="Red Hat / RPM" default>
```text
dnf -y install mod_proxy_html
```
</TabItem>
<TabItem value="deb" label="Debian / Ubuntu / apt">
```text
apt-get install -y libapache2-mod-proxy-html libxml2-dev
```
</TabItem>
</Tabs>
## Configuration
Assuming you are protecting `anubistest.techaro.lol`, you need the following server configuration blocks:

View File

@@ -4,7 +4,6 @@ title: Setting up Anubis
import RandomKey from "@site/src/components/RandomKey";
Anubis is meant to sit between your reverse proxy (such as Nginx or Caddy) and your target service. One instance of Anubis must be used per service you are protecting.
<center>
@@ -30,7 +29,7 @@ TLS terminator)
Anubis is shipped in the Docker repo [`ghcr.io/techarohq/anubis`](https://github.com/TecharoHQ/anubis/pkgs/container/anubis). The following tags exist for your convenience:
| Tag | Meaning |
|:--------------------|:-----------------------------------------------------------------------------------------------------------------------------------|
| :------------------ | :--------------------------------------------------------------------------------------------------------------------------------- |
| `latest` | The latest [tagged release](https://github.com/TecharoHQ/anubis/releases), if you are in doubt, start here. |
| `v<version number>` | The Anubis image for [any given tagged release](https://github.com/TecharoHQ/anubis/tags) |
| `main` | The current build on the `main` branch. Only use this if you need the latest and greatest features as they are merged into `main`. |
@@ -43,26 +42,40 @@ Anubis has very minimal system requirements. I suspect that 128Mi of ram may be
For more detailed information on installing Anubis with native packages, please read [the native install directions](./native-install.mdx).
## Environment variables
## Configuration
Anubis is configurable via environment variables and [the policy file](./policies.mdx). Most settings are currently exposed with environment variables but they are being slowly moved over to the policy file.
### Configuration via the policy file
Currently the following settings are configurable via the policy file:
- [Bot policies](./policies.mdx)
- [Open Graph passthrough](./configuration/open-graph.mdx)
- [Weight thresholds](./configuration/thresholds.mdx)
### Environment variables
Anubis uses these environment variables for configuration:
| Environment Variable | Default value | Explanation |
|:-------------------------------|:------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| :----------------------------- | :---------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `BASE_PREFIX` | unset | If set, adds a global prefix to all Anubis endpoints. For example, setting this to `/myapp` would make Anubis accessible at `/myapp/` instead of `/`. This is useful when running Anubis behind a reverse proxy that routes based on path prefixes. |
| `BIND` | `:8923` | The network address that Anubis listens on. For `unix`, set this to a path: `/run/anubis/instance.sock` |
| `BIND_NETWORK` | `tcp` | The address family that Anubis listens on. Accepts `tcp`, `unix` and anything Go's [`net.Listen`](https://pkg.go.dev/net#Listen) supports. |
| `COOKIE_DOMAIN` | unset | The domain the Anubis challenge pass cookie should be set to. This should be set to the domain you bought from your registrar (EG: `techaro.lol` if your webapp is running on `anubis.techaro.lol`). See this [stackoverflow explanation of cookies](https://stackoverflow.com/a/1063760) for more information.<br/><br/>Note that unlike `REDIRECT_DOMAINS`, you should never include a port number in this variable. |
| `COOKIE_DYNAMIC_DOMAIN` | false | If set to true, automatically set cookie domain fields based on the hostname of the request. EG: if you are making a request to `anubis.techaro.lol`, the Anubis cookie will be valid for any subdomain of `techaro.lol`. |
| `COOKIE_EXPIRATION_TIME` | `168h` | The amount of time the authorization cookie is valid for. |
| `COOKIE_PARTITIONED` | `false` | If set to `true`, enables the [partitioned (CHIPS) flag](https://developers.google.com/privacy-sandbox/cookies/chips), meaning that Anubis inside an iframe has a different set of cookies than the domain hosting the iframe. |
| `COOKIE_SECURE` | `true` | If set to `true`, enables the [Secure flag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Cookies#block_access_to_your_cookies), meaning that the cookies will only be transmitted over HTTPS. If Anubis is used in an unsecure context (plain HTTP), this will be need to be set to false |
| `DIFFICULTY` | `4` | The difficulty of the challenge, or the number of leading zeroes that must be in successful responses. |
| `ED25519_PRIVATE_KEY_HEX` | unset | The hex-encoded ed25519 private key used to sign Anubis responses. If this is not set, Anubis will generate one for you. This should be exactly 64 characters long. See below for details. |
| `ED25519_PRIVATE_KEY_HEX_FILE` | unset | Path to a file containing the hex-encoded ed25519 private key. Only one of this or its sister option may be set. |
| `METRICS_BIND` | `:9090` | The network address that Anubis serves Prometheus metrics on. See `BIND` for more information. |
| `METRICS_BIND_NETWORK` | `tcp` | The address family that the Anubis metrics server listens on. See `BIND_NETWORK` for more information. |
| `OG_EXPIRY_TIME` | `24h` | The expiration time for the Open Graph tag cache. |
| `OG_PASSTHROUGH` | `false` | If set to `true`, Anubis will enable Open Graph tag passthrough. |
| `OG_CACHE_CONSIDER_HOST` | `false` | If set to `true`, Anubis will consider the host in the Open Graph tag cache key. |
| `OG_EXPIRY_TIME` | `24h` | The expiration time for the Open Graph tag cache. Prefer using [the policy file](./configuration/open-graph.mdx) to configure the Open Graph subsystem. |
| `OG_PASSTHROUGH` | `false` | If set to `true`, Anubis will enable Open Graph tag passthrough. Prefer using [the policy file](./configuration/open-graph.mdx) to configure the Open Graph subsystem. |
| `OG_CACHE_CONSIDER_HOST` | `false` | If set to `true`, Anubis will consider the host in the Open Graph tag cache key. Prefer using [the policy file](./configuration/open-graph.mdx) to configure the Open Graph subsystem. |
| `POLICY_FNAME` | unset | The file containing [bot policy configuration](./policies.mdx). See the bot policy documentation for more details. If unset, the default bot policy configuration is used. |
| `REDIRECT_DOMAINS` | unset | If set, restrict the domains that Anubis can redirect to when passing a challenge.<br/><br/>If this is unset, Anubis may redirect to any domain which could cause security issues in the unlikely case that an attacker passes a challenge for your browser and then tricks you into clicking a link to your domain.<br/><br/>Note that if you are hosting Anubis on a non-standard port (`https://example:com:8443`, `http://www.example.net:8080`, etc.), you must also include the port number here. |
| `SERVE_ROBOTS_TXT` | `false` | If set `true`, Anubis will serve a default `robots.txt` file that disallows all known AI scrapers by name and then additionally disallows every scraper. This is useful if facts and circumstances make it difficult to change the underlying service to serve such a `robots.txt` file. |
@@ -82,11 +95,12 @@ If you don't know or understand what these settings mean, ignore them. These are
:::
| Environment Variable | Default value | Explanation |
| :---------------------------- | :------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------- |
| `TARGET_SNI` | unset | If set, overrides the TLS handshake hostname in requests forwarded to `TARGET`. |
| `TARGET_HOST` | unset | If set, overrides the Host header in requests forwarded to `TARGET`. |
| `TARGET_INSECURE_SKIP_VERIFY` | `false` | If `true`, skip TLS certificate validation for targets that listen over `https`. If your backend does not listen over `https`, ignore this setting. |
| Environment Variable | Default value | Explanation |
| :---------------------------- | :------------ | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `TARGET_SNI` | unset | If set, overrides the TLS handshake hostname in requests forwarded to `TARGET`. |
| `TARGET_HOST` | unset | If set, overrides the Host header in requests forwarded to `TARGET`. |
| `TARGET_INSECURE_SKIP_VERIFY` | `false` | If `true`, skip TLS certificate validation for targets that listen over `https`. If your backend does not listen over `https`, ignore this setting. |
| `HS512_SECRET` | unset | Secret string for JWT HS512 algorithm. If this is not set, Anubis will use ED25519 as defined via the variables above. The longer the better; 128 chars should suffice. |
</details>
@@ -138,6 +152,7 @@ STRIP_BASE_PREFIX=true
```
With this configuration:
- A request to `/myapp/api/users` would be forwarded to your target service as `/api/users`
- A request to `/myapp/` would be forwarded as `/`

View File

@@ -233,6 +233,10 @@ remote_addresses:
</TabItem>
</Tabs>
## Imprint / Impressum support
Anubis has support for showing imprint / impressum information. This is defined in the `impressum` block of your configuration. See [Imprint / Impressum configuration](./configuration/impressum.mdx) for more information.
## Risk calculation for downstream services
In case your service needs it for risk calculation reasons, Anubis exposes information about the rules that any requests match using a few headers:
@@ -261,17 +265,11 @@ Anubis rules can also add or remove "weight" from requests, allowing administrat
adjust: -5
```
This would remove five weight points from the request, making Anubis present the [Meta Refresh challenge](./configuration/challenges/metarefresh.mdx).
This would remove five weight points from the request, which would make Anubis present the [Meta Refresh challenge](./configuration/challenges/metarefresh.mdx) in the default configuration.
### Weight Thresholds
Weight thresholds and challenge associations will be configurable with CEL expressions in the configuration file in an upcoming patch, for now here's how Anubis configures the weight thresholds:
| Weight Expression | Action |
| -----------------------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------- |
| `weight < 0` (weight is less than 0) | Allow the request through. |
| `weight < 10` (weight is less than 10) | Challenge the client with the [Meta Refresh challenge](./configuration/challenges/metarefresh.mdx) at the default difficulty level. |
| `weight >= 10` (weight is greater than or equal to 10) | Challenge the client with the [Proof of Work challenge](./configuration/challenges/proof-of-work.mdx) at the default difficulty level. |
For more information on configuring weight thresholds, see [Weight Threshold Configuration](./configuration/thresholds.mdx)
### Advice

View File

@@ -59,7 +59,7 @@ For example, to add 10 weight points to requests from Brazil and China:
- name: countries-with-aggressive-scrapers
action: WEIGH
geoip:
counties:
countries:
- BR
- CN
weight:

View File

@@ -107,7 +107,6 @@ This ensures that the token has enough metadata to prove that the token is valid
Challenges are formed by taking some user request metadata and using that to generate a SHA-256 checksum. The following request headers are used:
- `Accept-Encoding`: The content encodings that the requestor supports, such as gzip.
- `Accept-Language`: The language that the requestor would prefer the server respond in, such as English.
- `X-Real-Ip`: The IP address of the requestor, as set by a reverse proxy server.
- `User-Agent`: The user agent string of the requestor.
- The current time in UTC rounded to the nearest week.

View File

@@ -58,7 +58,9 @@ This will build all static assets (CSS, JavaScript) for distribution.
make build
```
From this point it is up to you to make sure that `./var/anubis` ends up in the right place. You may want to consult the `./run` folder for useful files such as a systemd unit and `anubis.env.default` file.
From this point it is up to you to make sure that `./var/anubis` and `./var/robots2policy` end up in
the right place. You may want to consult the `./run` folder for useful files such as a systemd unit
and `anubis.env.default` file.
## "Pre-baked" tarball
@@ -75,7 +77,7 @@ When using this tarball, all you need to do is build `./cmd/anubis`:
make prebaked-build
```
Anubis will be built to `./var/anubis`.
Anubis will be built to `./var/anubis` and the robots2policy tool to `./var/robots2policy`.
## Development dependencies

View File

@@ -2,6 +2,10 @@
title: Local development
---
If you use an editor with [Development containers](https://containers.dev) support, load this repo's [devcontainer configuration](https://github.com/TecharoHQ/anubis/tree/main/.devcontainer). Skip to [Running Anubis locally](#running-anubis-locally) if you are using the devcontainer.
This enables you to contribute from [GitHub Codespaces](https://github.com/features/codespaces) or other web-based editors.
:::note
TL;DR: `npm ci && npm run dev`

View File

@@ -14,6 +14,7 @@ title: Anubis
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/TecharoHQ/anubis)
![language count](https://img.shields.io/github/languages/count/TecharoHQ/anubis)
![repo size](https://img.shields.io/github/repo-size/TecharoHQ/anubis)
[![GitHub Sponsors](https://img.shields.io/github/sponsors/Xe)](https://github.com/sponsors/Xe)
## Sponsors

View File

@@ -44,6 +44,7 @@ This page contains a non-exhaustive list with all websites using Anubis.
- https://squirreljme.cc/
- https://gitlab.postmarketos.org/
- https://wiki.koha-community.org/
- https://extensions.typo3.org/
- <details>
<summary>FreeCAD</summary>
- https://forum.freecad.org/

View File

@@ -70,6 +70,55 @@ bots:
dnsbl: false
impressum:
footer: |
This website is hosted by Techaro. If you have any complaints or notes about the service, please contact <a href="mailto:contact@techaro.lol">contact@techaro.lol</a> and we will assist you as soon as possible.
page:
title: Privacy Policy
body: |
<p>Last updated: June 2025</p>
<h2>Information that is gathered from visitors</h2>
<p>In common with other websites, log files are stored on the web server saving details such as the visitor's IP address, browser type, referring page and time of visit.</p>
<p>Cookies may be used to remember visitor preferences when interacting with the website.</p>
<p>Where registration is required, the visitor's email and a username will be stored on the server.</p>
<h2>How the Information is used</h2>
<p>The information is used to enhance the vistor's experience when using the website to display personalised content and possibly advertising.</p>
<p>E-mail addresses will not be sold, rented or leased to 3rd parties.</p>
<p>E-mail may be sent to inform you of news of our services or offers by us or our affiliates.</p>
<h2>Visitor Options</h2>
<p>If you have subscribed to one of our services, you may unsubscribe by following the instructions which are included in e-mail that you receive.</p>
<p>You may be able to block cookies via your browser settings but this may prevent you from access to certain features of the website.</p>
<h2>Cookies</h2>
<p>Cookies are small digital signature files that are stored by your web browser that allow your preferences to be recorded when visiting the website. Also they may be used to track your return visits to the website.</p>
<p>3rd party advertising companies may also use cookies for tracking purposes.</p>
<h2>Techaro Anubis</h2>
<p>This website uses a service called <a href="https://anubis.techaro.lol">Anubis</a> to filter malicious traffic. Anubis requires the use of browser cookies to ensure that web clients are running conformant software. Anubis also may report the following data to Techaro to improve service quality:</p>
<ul>
<li>IP address (for purposes of matching against geo-location and BGP autonomous systems numbers), which is stored in-memory and not persisted to disk.</li>
<li>Unique browser fingerprints (such as HTTP request fingerprints and encryption system fingerprints), which may be stored on Techaro's side for a period of up to one month.</li>
<li>HTTP request metadata that may include things such as the User-Agent header and other identifiers.</li>
</ul>
<p>This data is processed and stored for the legitimate interest of combatting abusive web clients. This data is encrypted at rest as much as possible and is only decrypted in memory for the purposes of fulfilling requests.</p>
# By default, send HTTP 200 back to clients that either get issued a challenge
# or a denial. This seems weird, but this is load-bearing due to the fact that
# the most aggressive scraper bots seem to really, really, want an HTTP 200 and

3085
docs/package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -15,9 +15,9 @@
"typecheck": "tsc"
},
"dependencies": {
"@docusaurus/core": "3.7.0",
"@docusaurus/preset-classic": "3.7.0",
"@docusaurus/theme-mermaid": "^3.7.0",
"@docusaurus/core": "^3.8.1",
"@docusaurus/preset-classic": "^3.8.1",
"@docusaurus/theme-mermaid": "^3.8.1",
"@mdx-js/react": "^3.0.0",
"clsx": "^2.0.0",
"prism-react-renderer": "^2.3.0",
@@ -25,9 +25,9 @@
"react-dom": "^19.0.0"
},
"devDependencies": {
"@docusaurus/module-type-aliases": "3.7.0",
"@docusaurus/tsconfig": "3.7.0",
"@docusaurus/types": "3.7.0",
"@docusaurus/module-type-aliases": "^3.8.1",
"@docusaurus/tsconfig": "^3.8.1",
"@docusaurus/types": "^3.8.1",
"typescript": "~5.6.2"
},
"browserslist": {
@@ -45,4 +45,4 @@
"engines": {
"node": ">=18.0"
}
}
}

43
go.mod
View File

@@ -4,23 +4,25 @@ go 1.24.2
require (
github.com/TecharoHQ/thoth-proto v0.4.0
github.com/a-h/templ v0.3.898
github.com/a-h/templ v0.3.906
github.com/cespare/xxhash/v2 v2.3.0
github.com/facebookgo/flagenv v0.0.0-20160425205200-fcd59fca7456
github.com/gaissmai/bart v0.20.4
github.com/golang-jwt/jwt/v5 v5.2.2
github.com/google/cel-go v0.25.0
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.0.1
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.1.0
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.1.0
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.2
github.com/joho/godotenv v1.5.1
github.com/nicksnyder/go-i18n/v2 v2.6.0
github.com/playwright-community/playwright-go v0.5200.0
github.com/prometheus/client_golang v1.22.0
github.com/sebest/xff v0.0.0-20210106013422-671bd2870b3a
golang.org/x/net v0.41.0
google.golang.org/grpc v1.72.2
golang.org/x/text v0.26.0
google.golang.org/grpc v1.73.0
gopkg.in/yaml.v3 v3.0.1
k8s.io/apimachinery v0.33.1
sigs.k8s.io/yaml v1.4.0
k8s.io/apimachinery v0.33.2
sigs.k8s.io/yaml v1.5.0
)
require (
@@ -29,7 +31,7 @@ require (
cel.dev/expr v0.23.1 // indirect
dario.cat/mergo v1.0.2 // indirect
github.com/AlekSi/pointer v1.2.0 // indirect
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c // indirect
github.com/BurntSushi/toml v1.5.0 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.3.1 // indirect
github.com/Masterminds/sprig/v3 v3.3.0 // indirect
@@ -47,6 +49,7 @@ require (
github.com/cli/browser v1.3.0 // indirect
github.com/cli/go-gh v0.1.0 // indirect
github.com/cloudflare/circl v1.6.1 // indirect
github.com/cpuguy83/go-md2man/v2 v2.0.6 // indirect
github.com/cyphar/filepath-securejoin v0.4.1 // indirect
github.com/deckarep/golang-set/v2 v2.8.0 // indirect
github.com/dlclark/regexp2 v1.11.4 // indirect
@@ -66,18 +69,21 @@ require (
github.com/gobwas/glob v0.2.3 // indirect
github.com/goccy/go-yaml v1.12.0 // indirect
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 // indirect
github.com/google/go-github/v70 v70.0.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/pprof v0.0.0-20230207041349-798e818bf904 // indirect
github.com/google/rpmpack v0.6.1-0.20250405124433-758cc6896cbc // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/goreleaser/chglog v0.7.0 // indirect
github.com/goreleaser/fileglob v1.3.0 // indirect
github.com/goreleaser/nfpm/v2 v2.42.1 // indirect
github.com/hashicorp/go-version v1.7.0 // indirect
github.com/huandu/xstrings v1.5.0 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/kevinburke/ssh_config v1.2.0 // indirect
github.com/klauspost/compress v1.18.0 // indirect
github.com/klauspost/pgzip v1.2.6 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
@@ -88,28 +94,38 @@ require (
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/russross/blackfriday/v2 v2.1.0 // indirect
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 // indirect
github.com/shopspring/decimal v1.4.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/skeema/knownhosts v1.3.1 // indirect
github.com/spf13/afero v1.14.0 // indirect
github.com/spf13/cast v1.7.1 // indirect
github.com/stoewer/go-strcase v1.3.0 // indirect
github.com/suzuki-shunsuke/logrus-error v0.1.4 // indirect
github.com/suzuki-shunsuke/pinact v1.6.0 // indirect
github.com/suzuki-shunsuke/urfave-cli-help-all v0.0.4 // indirect
github.com/ulikunitz/xz v0.5.12 // indirect
github.com/urfave/cli/v2 v2.27.6 // indirect
github.com/xanzy/ssh-agent v0.3.3 // indirect
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 // indirect
gitlab.com/digitalxero/go-conventional-commit v1.0.7 // indirect
go.yaml.in/yaml/v2 v2.4.2 // indirect
go.yaml.in/yaml/v3 v3.0.3 // indirect
golang.org/x/crypto v0.39.0 // indirect
golang.org/x/exp v0.0.0-20240719175910-8a7402abbf56 // indirect
golang.org/x/exp/typeparams v0.0.0-20231108232855-2478ac86f678 // indirect
golang.org/x/mod v0.25.0 // indirect
golang.org/x/oauth2 v0.28.0 // indirect
golang.org/x/sync v0.15.0 // indirect
golang.org/x/sys v0.33.0 // indirect
golang.org/x/telemetry v0.0.0-20240522233618-39ace7a40ae7 // indirect
golang.org/x/term v0.32.0 // indirect
golang.org/x/text v0.26.0 // indirect
golang.org/x/tools v0.33.0 // indirect
golang.org/x/tools v0.34.0 // indirect
golang.org/x/vuln v1.1.4 // indirect
golang.org/x/xerrors v0.0.0-20240716161551-93cc26a95ae9 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250218202821-56aae31c358a // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20250324211829-b45e905df463 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250324211829-b45e905df463 // indirect
google.golang.org/protobuf v1.36.6 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
honnef.co/go/tools v0.6.1 // indirect
@@ -120,6 +136,9 @@ require (
tool (
github.com/TecharoHQ/yeet/cmd/yeet
github.com/a-h/templ/cmd/templ
github.com/nicksnyder/go-i18n/v2/goi18n
github.com/suzuki-shunsuke/pinact/cmd/pinact
golang.org/x/tools/cmd/deadcode
golang.org/x/tools/cmd/goimports
golang.org/x/tools/cmd/stringer
golang.org/x/vuln/cmd/govulncheck

97
go.sum
View File

@@ -8,8 +8,8 @@ dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
github.com/AlekSi/pointer v1.2.0 h1:glcy/gc4h8HnG2Z3ZECSzZ1IX1x2JxRVuDzaJwQE0+w=
github.com/AlekSi/pointer v1.2.0/go.mod h1:gZGfd3dpW4vEc/UlyfKKi1roIqcCgwOIvb0tSNSBle0=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c h1:pxW6RcqyfI9/kWtOwnv/G+AzdKuy2ZrqINhenH4HyNs=
github.com/BurntSushi/toml v1.4.1-0.20240526193622-a339e1f7089c/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/BurntSushi/toml v1.5.0 h1:W5quZX/G/csjUnuI8SUYlsHs9M38FC7znL0lIO+DvMg=
github.com/BurntSushi/toml v1.5.0/go.mod h1:ukJfTF/6rtPPRCnwkur4qwRxa8vTRFBF0uk2lLoLwho=
github.com/DataDog/zstd v1.5.5 h1:oWf5W7GtOLgp6bciQYDmhHHjdhYkALu6S/5Ni9ZgSvQ=
github.com/DataDog/zstd v1.5.5/go.mod h1:g4AWEaM3yOg3HYfnJ3YIawPnVdXJh9QME85blwSAmyw=
github.com/MakeNowJust/heredoc v1.0.0/go.mod h1:mG5amYoWBHf8vpLOuehzbGGw0EHxpZZ6lCpQ4fNJ8LE=
@@ -36,8 +36,8 @@ github.com/TecharoHQ/yeet v0.6.0 h1:RCBAjr7wIlllsgy0tpvWpLX7jsZgu2tiuBY3RrprcR0=
github.com/TecharoHQ/yeet v0.6.0/go.mod h1:bj2V4Fg8qKQXoiuPZa3HuawrE8g+LsOQv/9q2WyGSsA=
github.com/a-h/parse v0.0.0-20250122154542-74294addb73e h1:HjVbSQHy+dnlS6C3XajZ69NYAb5jbGNfHanvm1+iYlo=
github.com/a-h/parse v0.0.0-20250122154542-74294addb73e/go.mod h1:3mnrkvGpurZ4ZrTDbYU84xhwXW2TjTKShSwjRi2ihfQ=
github.com/a-h/templ v0.3.898 h1:g9oxL/dmM6tvwRe2egJS8hBDQTncokbMoOFk1oJMX7s=
github.com/a-h/templ v0.3.898/go.mod h1:oLBbZVQ6//Q6zpvSMPTuBK0F3qOtBdFBcGRspcT+VNQ=
github.com/a-h/templ v0.3.906 h1:ZUThc8Q9n04UATaCwaG60pB1AqbulLmYEAMnWV63svg=
github.com/a-h/templ v0.3.906/go.mod h1:FFAu4dI//ESmEN7PQkJ7E7QfnSEMdcnu7QrAY8Dn334=
github.com/andybalholm/brotli v1.1.0 h1:eLKJA0d02Lf0mVpIDgYnqXcUn0GqVmEFny3VuID1U3M=
github.com/andybalholm/brotli v1.1.0/go.mod h1:sms7XGricyQI9K10gOSf56VKKWS4oLer58Q+mhRPtnY=
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be h1:9AeTilPcZAjCFIImctFaOjnTIavg87rW78vTPkQqLI8=
@@ -69,6 +69,8 @@ github.com/cli/safeexec v1.0.0/go.mod h1:Z/D4tTN8Vs5gXYHDCbaM1S/anmEDnJb1iW0+EJ5
github.com/cli/shurcooL-graphql v0.0.1/go.mod h1:U7gCSuMZP/Qy7kbqkk5PrqXEeDgtfG5K+W+u8weorps=
github.com/cloudflare/circl v1.6.1 h1:zqIqSPIndyBh1bjLVVDHMPpVKqp8Su/V+6MeDzzQBQ0=
github.com/cloudflare/circl v1.6.1/go.mod h1:uddAzsPgqdMAYatqJ0lsjX1oECcQLIlRpzZh3pJrofs=
github.com/cpuguy83/go-md2man/v2 v2.0.6 h1:XJtiaUW6dEEqVuZiMTn1ldk455QWwEIsMIJlo5vtkx0=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
github.com/cyphar/filepath-securejoin v0.4.1 h1:JyxxyPEaktOD+GAnqIqTf9A8tHyAG22rowi7HkoSU1s=
@@ -150,9 +152,14 @@ github.com/google/cel-go v0.25.0 h1:jsFw9Fhn+3y2kBbltZR4VEz5xKkcIFRPDnuEzAGv5GY=
github.com/google/cel-go v0.25.0/go.mod h1:hjEb6r5SuOSlhCHmFoLzu8HGCERvIsDAbxDAyNU/MmI=
github.com/google/go-cmdtest v0.4.1-0.20220921163831-55ab3332a786 h1:rcv+Ippz6RAtvaGgKxc+8FQIpxHgsF+HBzPyYL2cyVU=
github.com/google/go-cmdtest v0.4.1-0.20220921163831-55ab3332a786/go.mod h1:apVn/GCasLZUVpAJ6oWAuyP7Ne7CEsQbTnc0plM3m+o=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-github/v70 v70.0.0 h1:/tqCp5KPrcvqCc7vIvYyFYTiCGrYvaWoYMGHSQbo55o=
github.com/google/go-github/v70 v70.0.0/go.mod h1:xBUZgo8MI3lUL/hwxl3hlceJW1U8MVnXP3zUyI+rhQY=
github.com/google/go-querystring v1.1.0 h1:AnCroh3fv4ZBgVIf1Iwtovgjaw/GiKJo8M8yD/fhyJ8=
github.com/google/go-querystring v1.1.0/go.mod h1:Kcdr2DB4koayq7X8pmAG4sNG59So17icRSOU623lUBU=
github.com/google/pprof v0.0.0-20230207041349-798e818bf904 h1:4/hN5RUoecvl+RmJRE2YxKWtnnQls6rQjjW5oV7qg2U=
github.com/google/pprof v0.0.0-20230207041349-798e818bf904/go.mod h1:uglQLonpP8qtYCYyzA+8c/9qtqgA3qsXGYqCPKARAFg=
github.com/google/renameio v0.1.0 h1:GOZbcHa3HfsPKPlmyPyN2KEohoMXOhdMbHrvbpl2QaA=
@@ -171,11 +178,13 @@ github.com/goreleaser/fileglob v1.3.0 h1:/X6J7U8lbDpQtBvGcwwPS6OpzkNVlVEsFUVRx9+
github.com/goreleaser/fileglob v1.3.0/go.mod h1:Jx6BoXv3mbYkEzwm9THo7xbr5egkAraxkGorbJb4RxU=
github.com/goreleaser/nfpm/v2 v2.42.1 h1:xu2pLRgQuz2ab+YZFoeIzwU/M5jjjCKDGwv1lRbVGvk=
github.com/goreleaser/nfpm/v2 v2.42.1/go.mod h1:dY53KWYKebkOocxgkmpM7SRX0Nv5hU+jEu2kIaM4/LI=
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.0.1 h1:qnpSQwGEnkcRpTqNOIR6bJbR0gAorgP9CSALpRcKoAA=
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.0.1/go.mod h1:lXGCsh6c22WGtjr+qGHj1otzZpV/1kwTMAqkwZsnWRU=
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.1.0 h1:pRhl55Yx1eC7BZ1N+BBWwnKaMyD8uC+34TLdndZMAKk=
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.1.0/go.mod h1:XKMd7iuf/RGPSMJ/U4HP0zS2Z9Fh8Ps9a+6X26m/tmI=
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.1.0 h1:QGLs/O40yoNK9vmy4rhUGBVyMf1lISBGtXRpsu/Qu/o=
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus v1.1.0/go.mod h1:hM2alZsMUni80N33RBe6J0e423LB+odMj7d3EMP9l20=
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.2 h1:sGm2vDRFUrQJO/Veii4h4zG2vvqG6uWNkBHSTqXOZk0=
github.com/grpc-ecosystem/go-grpc-middleware/v2 v2.3.2/go.mod h1:wd1YpapPLivG6nQgbf7ZkG1hhSOXDhhn4MLTknx2aAc=
github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI=
github.com/hashicorp/go-version v1.7.0 h1:5tqGy27NaOTB8yJKUZELlFAS/LTKJkrmONwQKeRZfjY=
github.com/hashicorp/go-version v1.7.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
github.com/henvic/httpretty v0.0.6/go.mod h1:X38wLjWXHkXT7r2+uK8LjCMne9rsuNaBLJ+5cU2/Pmo=
github.com/huandu/xstrings v1.5.0 h1:2ag3IFq9ZDANvthTwTiqSSZLjDc+BedvHPAp5tJy2TI=
github.com/huandu/xstrings v1.5.0/go.mod h1:y5/lhBue+AyNmUVz9RLU9xbLR0o4KIIExikq4ovT0aE=
@@ -208,8 +217,9 @@ github.com/matryer/is v1.4.0 h1:sosSmIWwkYITGrxZ25ULNDeKiMNzFSr4V/eqBQP0PeE=
github.com/matryer/is v1.4.0/go.mod h1:8I/i5uYgLzgsgEloJE1U6xx5HkBQpAZvepWuujKwMRU=
github.com/mattn/go-colorable v0.1.8/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
@@ -227,6 +237,8 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8m
github.com/natefinch/atomic v1.0.1 h1:ZPYKxkqQOx3KZ+RsbnP/YsgvxWQPGxjC0oBt2AhwV0A=
github.com/natefinch/atomic v1.0.1/go.mod h1:N/D/ELrljoqDyT3rZrsUmtsuzvHkeB/wWjHV22AZRbM=
github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32/go.mod h1:9wM+0iRr9ahx58uYLpLIr5fm8diHn0JbqRycJi6w0Ms=
github.com/nicksnyder/go-i18n/v2 v2.6.0 h1:C/m2NNWNiTB6SK4Ao8df5EWm3JETSTIGNXBpMJTxzxQ=
github.com/nicksnyder/go-i18n/v2 v2.6.0/go.mod h1:88sRqr0C6OPyJn0/KRNaEz1uWorjxIKP7rUUcvycecE=
github.com/onsi/gomega v1.35.1 h1:Cwbd75ZBPxFSuZ6T+rN/WCb/gOc6YgFBXLlZLhC7Ds4=
github.com/onsi/gomega v1.35.1/go.mod h1:PvZbdDc8J6XJEpDK4HCuRBm8a6Fzp9/DmhC9C7yFlog=
github.com/pjbgf/sha1cd v0.3.2 h1:a9wb0bp1oC2TGwStyn0Umc/IGKQnEgF0vVaZ8QF8eo4=
@@ -249,6 +261,8 @@ github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoG
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sassoftware/go-rpmutils v0.4.0 h1:ojND82NYBxgwrV+mX1CWsd5QJvvEZTKddtCdFLPWhpg=
github.com/sassoftware/go-rpmutils v0.4.0/go.mod h1:3goNWi7PGAT3/dlql2lv3+MSN5jNYPjT5mVcQcIsYzI=
github.com/sebest/xff v0.0.0-20210106013422-671bd2870b3a h1:iLcLb5Fwwz7g/DLK89F+uQBDeAhHhwdzB5fSlVdhGcM=
@@ -258,12 +272,16 @@ github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG
github.com/shopspring/decimal v1.4.0 h1:bxl37RwXBklmTi0C79JfXCEBD1cqqHt0bbgBAGFp81k=
github.com/shopspring/decimal v1.4.0/go.mod h1:gawqmDU56v4yIKSwfBSFip1HdCCXN8/+DMd9qYNcwME=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/skeema/knownhosts v1.3.1 h1:X2osQ+RAjK76shCbvhHHHVl3ZlgDm8apHEHFqRjnBY8=
github.com/skeema/knownhosts v1.3.1/go.mod h1:r7KTdC8l4uxWRyK2TpQZ/1o5HaSzh06ePQNxPwTcfiY=
github.com/smarty/assertions v1.15.0 h1:cR//PqUBUiQRakZWqBiFFQ9wb8emQGDb0HeGdqGByCY=
github.com/smarty/assertions v1.15.0/go.mod h1:yABtdzeQs6l1brC900WlRNwj6ZR55d7B+E8C6HtKdec=
github.com/smartystreets/goconvey v1.8.1 h1:qGjIddxOk4grTu9JPOU31tVfq3cNdBlNa5sSznIX1xY=
github.com/smartystreets/goconvey v1.8.1/go.mod h1:+/u4qLyY6x1jReYOp7GOM2FSt8aP9CzCZL03bI28W60=
github.com/spf13/afero v1.14.0 h1:9tH6MapGnn/j0eb0yIXiLjERO8RB6xIVZRDCX7PtqWA=
github.com/spf13/afero v1.14.0/go.mod h1:acJQ8t0ohCGuMN3O+Pv0V0hgMxNYDlvdk+VTfyZmbYo=
github.com/spf13/cast v1.7.1 h1:cuNEagBQEHWN1FnbGEjCXL2szYEXqfJPbP2HNUaca9Y=
github.com/spf13/cast v1.7.1/go.mod h1:ancEpBxwJDODSW/UG4rDrAqiKolqNNh2DX3mk86cAdo=
github.com/stoewer/go-strcase v1.3.0 h1:g0eASXYtp+yvN9fK8sH94oCIk0fau9uV1/ZdJ0AVEzs=
@@ -279,28 +297,42 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/suzuki-shunsuke/logrus-error v0.1.4 h1:nWo98uba1fANHdZ9Y5pJ2RKs/PpVjrLzRp5m+mRb9KE=
github.com/suzuki-shunsuke/logrus-error v0.1.4/go.mod h1:WsVvvw6SKSt08/fB2qbnsKIMJA4K1MYCUprqsBJbMiM=
github.com/suzuki-shunsuke/pinact v1.6.0 h1:2QvSzREOquwLwKXhF9Hj0AInE/Rl63SZz9dKkHFC6so=
github.com/suzuki-shunsuke/pinact v1.6.0/go.mod h1:FDUMck0mmL0mcnNZ23Vjh/aOR5cIdZhF1IIpGksT4dQ=
github.com/suzuki-shunsuke/urfave-cli-help-all v0.0.4 h1:YGHgrVjGTYHY98II6zijXUHP+OyvrzSCvd8m9iUcaK8=
github.com/suzuki-shunsuke/urfave-cli-help-all v0.0.4/go.mod h1:sSi6xaUaHfaqu32ECLeyE7NTMv+ZM5dW0JikhllaalY=
github.com/thlib/go-timezone-local v0.0.0-20210907160436-ef149e42d28e/go.mod h1:/Tnicc6m/lsJE0irFMA0LfIwTBo4QP7A8IfyIv4zZKI=
github.com/ulikunitz/xz v0.5.12 h1:37Nm15o69RwBkXM0J6A5OlE67RZTfzUxTj8fB3dfcsc=
github.com/ulikunitz/xz v0.5.12/go.mod h1:nbz6k7qbPmH4IRqmfOplQw/tblSgqTqBwxkY0oWt/14=
github.com/urfave/cli/v2 v2.27.6 h1:VdRdS98FNhKZ8/Az8B7MTyGQmpIr36O1EHybx/LaZ4g=
github.com/urfave/cli/v2 v2.27.6/go.mod h1:3Sevf16NykTbInEnD0yKkjDAeZDS0A6bzhBH5hrMvTQ=
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8 h1:nIPpBwaJSVYIxUFsDv3M8ofmx9yWTog9BfvIu0q41lo=
github.com/xi2/xz v0.0.0-20171230120015-48954b6210f8/go.mod h1:HUYIGzjTL3rfEspMxjDjgmT5uz5wzYJKVo23qUhYTos=
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1 h1:gEOO8jv9F4OT7lGCjxCBTO/36wtF6j2nSip77qHd4x4=
github.com/xrash/smetrics v0.0.0-20240521201337-686a1a2994c1/go.mod h1:Ohn+xnUBiLI6FVj/9LpzZWtj1/D6lUovWYBkxHVV3aM=
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
gitlab.com/digitalxero/go-conventional-commit v1.0.7 h1:8/dO6WWG+98PMhlZowt/YjuiKhqhGlOCwlIV8SqqGh8=
gitlab.com/digitalxero/go-conventional-commit v1.0.7/go.mod h1:05Xc2BFsSyC5tKhK0y+P3bs0AwUtNuTp+mTpbCU/DZ0=
go.opentelemetry.io/auto/sdk v1.1.0 h1:cH53jehLUN6UFLY71z+NDOiNJqDdPRaXzTel0sJySYA=
go.opentelemetry.io/auto/sdk v1.1.0/go.mod h1:3wSPjt5PWp2RhlCcmmOial7AvC4DQqZb7a7wCow3W8A=
go.opentelemetry.io/otel v1.34.0 h1:zRLXxLCgL1WyKsPVrgbSdMN4c0FMkDAskSTQP+0hdUY=
go.opentelemetry.io/otel v1.34.0/go.mod h1:OWFPOQ+h4G8xpyjgqo4SxJYdDQ/qmRH+wivy7zzx9oI=
go.opentelemetry.io/otel/metric v1.34.0 h1:+eTR3U0MyfWjRDhmFMxe2SsW64QrZ84AOhvqS7Y+PoQ=
go.opentelemetry.io/otel/metric v1.34.0/go.mod h1:CEDrp0fy2D0MvkXE+dPV7cMi8tWZwX3dmaIhwPOaqHE=
go.opentelemetry.io/otel/sdk v1.34.0 h1:95zS4k/2GOy069d321O8jWgYsW3MzVV+KuSPKp7Wr1A=
go.opentelemetry.io/otel/sdk v1.34.0/go.mod h1:0e/pNiaMAqaykJGKbi+tSjWfNNHMTxoC9qANsCzbyxU=
go.opentelemetry.io/otel/sdk/metric v1.34.0 h1:5CeK9ujjbFVL5c1PhLuStg1wxA7vQv7ce1EK0Gyvahk=
go.opentelemetry.io/otel/sdk/metric v1.34.0/go.mod h1:jQ/r8Ze28zRKoNRdkjCZxfs6YvBTG1+YIqyFVFYec5w=
go.opentelemetry.io/otel/trace v1.34.0 h1:+ouXS2V8Rd4hp4580a8q23bg0azF2nI8cqLYnC8mh/k=
go.opentelemetry.io/otel/trace v1.34.0/go.mod h1:Svm7lSjQD7kG7KJ/MUHPVXSDGz2OX4h0M2jHBhmSfRE=
go.opentelemetry.io/otel v1.35.0 h1:xKWKPxrxB6OtMCbmMY021CqC45J+3Onta9MqjhnusiQ=
go.opentelemetry.io/otel v1.35.0/go.mod h1:UEqy8Zp11hpkUrL73gSlELM0DupHoiq72dR+Zqel/+Y=
go.opentelemetry.io/otel/metric v1.35.0 h1:0znxYu2SNyuMSQT4Y9WDWej0VpcsxkuklLa4/siN90M=
go.opentelemetry.io/otel/metric v1.35.0/go.mod h1:nKVFgxBZ2fReX6IlyW28MgZojkoAkJGaE8CpgeAU3oE=
go.opentelemetry.io/otel/sdk v1.35.0 h1:iPctf8iprVySXSKJffSS79eOjl9pvxV9ZqOWT0QejKY=
go.opentelemetry.io/otel/sdk v1.35.0/go.mod h1:+ga1bZliga3DxJ3CQGg3updiaAJoNECOgJREo9KHGQg=
go.opentelemetry.io/otel/sdk/metric v1.35.0 h1:1RriWBmCKgkeHEhM7a2uMjMUfP7MsOF5JpUCaEqEI9o=
go.opentelemetry.io/otel/sdk/metric v1.35.0/go.mod h1:is6XYCUMpcKi+ZsOvfluY5YstFnhW0BidkR+gL+qN+w=
go.opentelemetry.io/otel/trace v1.35.0 h1:dPpEfJu1sDIqruz7BHFG3c7528f6ddfSWfFDVt/xgMs=
go.opentelemetry.io/otel/trace v1.35.0/go.mod h1:WUk7DtFp1Aw2MkvqGdwiXYDZZNvA/1J8o6xRXLrIkyc=
go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
go.yaml.in/yaml/v3 v3.0.3 h1:bXOww4E/J3f66rav3pX3m8w6jDE4knZjGOw8b5Y6iNE=
go.yaml.in/yaml/v3 v3.0.3/go.mod h1:tBHosrYAkRZjRAOREWbDnBXUf08JOwYq++0QNwQiWzI=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
@@ -325,6 +357,8 @@ golang.org/x/net v0.6.0/go.mod h1:2Tu9+aMcznHK/AK1HMvgo6xiTLG5rD5rZLDS+rp2Bjs=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.41.0 h1:vBTly1HeNPEn3wtREYfy4GZ/NECgw2Cnl+nK6Nz3uvw=
golang.org/x/net v0.41.0/go.mod h1:B/K4NNqkfmg07DQYrbwvSluqCJOOXwUjeb/5lOisjbA=
golang.org/x/oauth2 v0.28.0 h1:CrgCKl8PPAVtLnU3c+EDw6x11699EWlsDeWNWKdIOkc=
golang.org/x/oauth2 v0.28.0/go.mod h1:onh5ek6nERTohokkhCD/y2cV4Do3fxFHFuAejCkRWT8=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -376,21 +410,22 @@ golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGm
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
golang.org/x/tools v0.6.0/go.mod h1:Xwgl3UAJ/d3gWutnCtw505GrjyAbvKui8lOU390QaIU=
golang.org/x/tools v0.33.0 h1:4qz2S3zmRxbGIhDIAgjxvFutSvH5EfnsYrRBj0UI0bc=
golang.org/x/tools v0.33.0/go.mod h1:CIJMaWEY88juyUfo7UbgPqbC8rU2OqfAV1h2Qp0oMYI=
golang.org/x/tools v0.34.0 h1:qIpSLOxeCYGg9TrcJokLBG4KFA6d795g0xkBkiESGlo=
golang.org/x/tools v0.34.0/go.mod h1:pAP9OwEaY1CAW3HOmg3hLZC5Z0CCmzjAF2UQMSqNARg=
golang.org/x/vuln v1.1.4 h1:Ju8QsuyhX3Hk8ma3CesTbO8vfJD9EvUBgHvkxHBzj0I=
golang.org/x/vuln v1.1.4/go.mod h1:F+45wmU18ym/ca5PLTPLsSzr2KppzswxPP603ldA67s=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20220609144429-65e65417b02f/go.mod h1:K8+ghG5WaK9qNqU5K3HdILfMLy1f3aNYFI/wnl100a8=
golang.org/x/xerrors v0.0.0-20240716161551-93cc26a95ae9 h1:LLhsEBxRTBLuKlQxFBYUOU8xyFgXv6cOTp2HASDlsDk=
golang.org/x/xerrors v0.0.0-20240716161551-93cc26a95ae9/go.mod h1:NDW/Ps6MPRej6fsCIbMTohpP40sJ/P/vI1MoTEGwX90=
google.golang.org/genproto/googleapis/api v0.0.0-20250218202821-56aae31c358a h1:nwKuGPlUAt+aR+pcrkfFRrTU1BVrSmYyYMxYbUIVHr0=
google.golang.org/genproto/googleapis/api v0.0.0-20250218202821-56aae31c358a/go.mod h1:3kWAYMk1I75K4vykHtKt2ycnOgpA6974V7bREqbsenU=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a h1:51aaUVRocpvUOSQKM6Q7VuoaktNIaMCLuhZB6DKksq4=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250218202821-56aae31c358a/go.mod h1:uRxBH1mhmO8PGhU89cMcHaXKZqO+OfakD8QQO0oYwlQ=
google.golang.org/grpc v1.72.2 h1:TdbGzwb82ty4OusHWepvFWGLgIbNo1/SUynEN0ssqv8=
google.golang.org/grpc v1.72.2/go.mod h1:wH5Aktxcg25y1I3w7H69nHfXdOG3UiadoBtjh3izSDM=
google.golang.org/genproto/googleapis/api v0.0.0-20250324211829-b45e905df463 h1:hE3bRWtU6uceqlh4fhrSnUyjKHMKB9KrTLLG+bc0ddM=
google.golang.org/genproto/googleapis/api v0.0.0-20250324211829-b45e905df463/go.mod h1:U90ffi8eUL9MwPcrJylN5+Mk2v3vuPDptd5yyNUiRR8=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250324211829-b45e905df463 h1:e0AIkUUhxyBKh6ssZNrAMeqhA7RKUj42346d1y02i2g=
google.golang.org/genproto/googleapis/rpc v0.0.0-20250324211829-b45e905df463/go.mod h1:qQ0YXyHHx3XkvlzUtpXDkS29lDSafHMZBAZDc03LQ3A=
google.golang.org/grpc v1.73.0 h1:VIWSmpI2MegBtTuFt5/JWy2oXxtjJ/e89Z70ImfD2ok=
google.golang.org/grpc v1.73.0/go.mod h1:50sbHOUqWoCQGI8V2HQLJM0B+LMlIUjNSZmow7EVBQc=
google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY=
google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
@@ -409,8 +444,8 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
honnef.co/go/tools v0.6.1 h1:R094WgE8K4JirYjBaOpz/AvTyUu/3wbmAoskKN/pxTI=
honnef.co/go/tools v0.6.1/go.mod h1:3puzxxljPCe8RGJX7BIy1plGbxEOZni5mR2aXe3/uk4=
k8s.io/apimachinery v0.33.1 h1:mzqXWV8tW9Rw4VeW9rEkqvnxj59k1ezDUl20tFK/oM4=
k8s.io/apimachinery v0.33.1/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM=
k8s.io/apimachinery v0.33.2 h1:IHFVhqg59mb8PJWTLi8m1mAoepkUNYmptHsV+Z1m5jY=
k8s.io/apimachinery v0.33.2/go.mod h1:BHW0YOu7n22fFv/JkYOEfkUYNRN0fj0BlvMFWA7b+SM=
mvdan.cc/sh/v3 v3.11.0 h1:q5h+XMDRfUGUedCqFFsjoFjrhwf2Mvtt1rkMvVz0blw=
mvdan.cc/sh/v3 v3.11.0/go.mod h1:LRM+1NjoYCzuq/WZ6y44x14YNAI0NK7FLPeQSaFagGg=
pault.ag/go/debian v0.18.0 h1:nr0iiyOU5QlG1VPnhZLNhnCcHx58kukvBJp+dvaM6CQ=
@@ -419,5 +454,5 @@ pault.ag/go/topsort v0.1.1 h1:L0QnhUly6LmTv0e3DEzbN2q6/FGgAcQvaEw65S53Bg4=
pault.ag/go/topsort v0.1.1/go.mod h1:r1kc/L0/FZ3HhjezBIPaNVhkqv8L0UJ9bxRuHRVZ0q4=
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3 h1:/Rv+M11QRah1itp8VhT6HoVx1Ray9eB4DBr+K+/sCJ8=
sigs.k8s.io/json v0.0.0-20241010143419-9aa6b5e7a4b3/go.mod h1:18nIHnGi6636UCz6m8i4DhaJ65T6EruyzmoQqI2BVDo=
sigs.k8s.io/yaml v1.4.0 h1:Mk1wCc2gy/F0THH0TAp1QYyJNzRm2KCLy3o5ASXVI5E=
sigs.k8s.io/yaml v1.4.0/go.mod h1:Ejl7/uTz7PSA4eKMyQCUTnhZYNmLIl+5c2lQPGR2BPY=
sigs.k8s.io/yaml v1.5.0 h1:M10b2U7aEUY6hRtU870n2VTPgR5RZiL/I6Lcc2F4NUQ=
sigs.k8s.io/yaml v1.5.0/go.mod h1:wZs27Rbxoai4C0f8/9urLZtZtF3avA3gKvGyPdDqTO4=

View File

@@ -13,6 +13,10 @@ func (c *OGTagCache) GetOGTags(url *url.URL, originalHost string) (map[string]st
return nil, errors.New("nil URL provided, cannot fetch OG tags")
}
if len(c.ogOverride) != 0 {
return c.ogOverride, nil
}
target := c.getTarget(url)
cacheKey := c.generateCacheKey(target, originalHost)

View File

@@ -7,10 +7,49 @@ import (
"reflect"
"testing"
"time"
"github.com/TecharoHQ/anubis/lib/policy/config"
)
func TestCacheReturnsDefault(t *testing.T) {
want := map[string]string{
"og:title": "Foo bar",
"og:description": "The best website ever made!!!1!",
}
cache := NewOGTagCache("", config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: false,
Override: want,
})
u, err := url.Parse("https://anubis.techaro.lol")
if err != nil {
t.Fatal(err)
}
result, err := cache.GetOGTags(u, "anubis.techaro.lol")
if err != nil {
t.Fatal(err)
}
for k, v := range want {
t.Run(k, func(t *testing.T) {
if got := result[k]; got != v {
t.Logf("want: tags[%q] = %q", k, v)
t.Logf("got: tags[%q] = %q", k, got)
t.Error("invalid result from function")
}
})
}
}
func TestCheckCache(t *testing.T) {
cache := NewOGTagCache("http://example.com", true, time.Minute, false)
cache := NewOGTagCache("http://example.com", config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: false,
})
// Set up test data
urlStr := "http://example.com/page"
@@ -69,7 +108,11 @@ func TestGetOGTags(t *testing.T) {
defer ts.Close()
// Create an instance of OGTagCache with a short TTL for testing
cache := NewOGTagCache(ts.URL, true, 1*time.Minute, false)
cache := NewOGTagCache(ts.URL, config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: false,
})
// Parse the test server URL
parsedURL, err := url.Parse(ts.URL)
@@ -216,7 +259,11 @@ func TestGetOGTagsWithHostConsideration(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
loadCount = 0 // Reset load count for each test case
cache := NewOGTagCache(ts.URL, true, 1*time.Minute, tc.ogCacheConsiderHost)
cache := NewOGTagCache(ts.URL, config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: tc.ogCacheConsiderHost,
})
for i, req := range tc.requests {
ogTags, err := cache.GetOGTags(parsedURL, req.host)

View File

@@ -10,6 +10,7 @@ import (
"testing"
"time"
"github.com/TecharoHQ/anubis/lib/policy/config"
"golang.org/x/net/html"
)
@@ -80,7 +81,11 @@ func TestFetchHTMLDocument(t *testing.T) {
}))
defer ts.Close()
cache := NewOGTagCache("", true, time.Minute, false)
cache := NewOGTagCache("", config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: false,
})
doc, err := cache.fetchHTMLDocument(ts.URL, "anything")
if tt.expectError {
@@ -107,7 +112,11 @@ func TestFetchHTMLDocumentInvalidURL(t *testing.T) {
t.Skip("test requires theoretical network egress")
}
cache := NewOGTagCache("", true, time.Minute, false)
cache := NewOGTagCache("", config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: false,
})
doc, err := cache.fetchHTMLDocument("http://invalid.url.that.doesnt.exist.example", "anything")

View File

@@ -6,6 +6,8 @@ import (
"net/url"
"testing"
"time"
"github.com/TecharoHQ/anubis/lib/policy/config"
)
func TestIntegrationGetOGTags(t *testing.T) {
@@ -104,7 +106,11 @@ func TestIntegrationGetOGTags(t *testing.T) {
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
// Create cache instance
cache := NewOGTagCache(ts.URL, true, 1*time.Minute, false)
cache := NewOGTagCache(ts.URL, config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: false,
})
// Create URL for test
testURL, _ := url.Parse(ts.URL)

View File

@@ -6,6 +6,7 @@ import (
"strings"
"testing"
"github.com/TecharoHQ/anubis/lib/policy/config"
"golang.org/x/net/html"
)
@@ -29,7 +30,7 @@ func BenchmarkGetTarget(b *testing.B) {
for _, tt := range tests {
b.Run(tt.name, func(b *testing.B) {
cache := NewOGTagCache(tt.target, false, 0, false)
cache := NewOGTagCache(tt.target, config.OpenGraph{})
urls := make([]*url.URL, len(tt.paths))
for i, path := range tt.paths {
u, _ := url.Parse(path)
@@ -65,7 +66,7 @@ func BenchmarkExtractOGTags(b *testing.B) {
</head><body><div><p>Content</p></div></body></html>`,
}
cache := NewOGTagCache("http://example.com", false, 0, false)
cache := NewOGTagCache("http://example.com", config.OpenGraph{})
docs := make([]*html.Node, len(htmlSamples))
for i, sample := range htmlSamples {
@@ -83,7 +84,7 @@ func BenchmarkExtractOGTags(b *testing.B) {
// Memory usage test
func TestMemoryUsage(t *testing.T) {
cache := NewOGTagCache("http://example.com", false, 0, false)
cache := NewOGTagCache("http://example.com", config.OpenGraph{})
// Force GC and wait for it to complete
runtime.GC()

View File

@@ -10,6 +10,7 @@ import (
"time"
"github.com/TecharoHQ/anubis/decaymap"
"github.com/TecharoHQ/anubis/lib/policy/config"
)
const (
@@ -32,9 +33,10 @@ type OGTagCache struct {
ogTimeToLive time.Duration
ogCacheConsiderHost bool
ogPassthrough bool
ogOverride map[string]string
}
func NewOGTagCache(target string, ogPassthrough bool, ogTimeToLive time.Duration, ogTagsConsiderHost bool) *OGTagCache {
func NewOGTagCache(target string, conf config.OpenGraph) *OGTagCache {
// Predefined approved tags and prefixes
defaultApprovedTags := []string{"description", "keywords", "author"}
defaultApprovedPrefixes := []string{"og:", "twitter:", "fediverse:"}
@@ -77,9 +79,10 @@ func NewOGTagCache(target string, ogPassthrough bool, ogTimeToLive time.Duration
return &OGTagCache{
cache: decaymap.New[string, map[string]string](),
targetURL: parsedTargetURL,
ogPassthrough: ogPassthrough,
ogTimeToLive: ogTimeToLive,
ogCacheConsiderHost: ogTagsConsiderHost,
ogPassthrough: conf.Enabled,
ogTimeToLive: conf.TimeToLive,
ogCacheConsiderHost: conf.ConsiderHost,
ogOverride: conf.Override,
approvedTags: defaultApprovedTags,
approvedPrefixes: defaultApprovedPrefixes,
client: client,

View File

@@ -6,6 +6,7 @@ import (
"testing"
"unicode/utf8"
"github.com/TecharoHQ/anubis/lib/policy/config"
"golang.org/x/net/html"
)
@@ -45,7 +46,7 @@ func FuzzGetTarget(f *testing.F) {
}
// Create cache - should not panic
cache := NewOGTagCache(target, false, 0, false)
cache := NewOGTagCache(target, config.OpenGraph{})
// Create URL
u := &url.URL{
@@ -129,7 +130,7 @@ func FuzzExtractOGTags(f *testing.F) {
return
}
cache := NewOGTagCache("http://example.com", false, 0, false)
cache := NewOGTagCache("http://example.com", config.OpenGraph{})
// Should not panic
tags := cache.extractOGTags(doc)
@@ -185,7 +186,7 @@ func FuzzGetTargetRoundTrip(f *testing.F) {
t.Skip()
}
cache := NewOGTagCache(target, false, 0, false)
cache := NewOGTagCache(target, config.OpenGraph{})
u := &url.URL{Path: path, RawQuery: query}
result := cache.getTarget(u)
@@ -242,7 +243,7 @@ func FuzzExtractMetaTagInfo(f *testing.F) {
},
}
cache := NewOGTagCache("http://example.com", false, 0, false)
cache := NewOGTagCache("http://example.com", config.OpenGraph{})
// Should not panic
property, content := cache.extractMetaTagInfo(node)
@@ -295,7 +296,7 @@ func BenchmarkFuzzedGetTarget(b *testing.B) {
for _, input := range inputs {
b.Run(input.name, func(b *testing.B) {
cache := NewOGTagCache(input.target, false, 0, false)
cache := NewOGTagCache(input.target, config.OpenGraph{})
u := &url.URL{Path: input.path, RawQuery: input.query}
b.ResetTimer()

View File

@@ -13,6 +13,8 @@ import (
"strings"
"testing"
"time"
"github.com/TecharoHQ/anubis/lib/policy/config"
)
func TestNewOGTagCache(t *testing.T) {
@@ -38,7 +40,11 @@ func TestNewOGTagCache(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cache := NewOGTagCache(tt.target, tt.ogPassthrough, tt.ogTimeToLive, false)
cache := NewOGTagCache(tt.target, config.OpenGraph{
Enabled: tt.ogPassthrough,
TimeToLive: tt.ogTimeToLive,
ConsiderHost: false,
})
if cache == nil {
t.Fatal("expected non-nil cache, got nil")
@@ -74,7 +80,11 @@ func TestNewOGTagCache_UnixSocket(t *testing.T) {
socketPath := filepath.Join(tempDir, "test.sock")
target := "unix://" + socketPath
cache := NewOGTagCache(target, true, 5*time.Minute, false)
cache := NewOGTagCache(target, config.OpenGraph{
Enabled: true,
TimeToLive: 5 * time.Minute,
ConsiderHost: false,
})
if cache == nil {
t.Fatal("expected non-nil cache, got nil")
@@ -155,7 +165,11 @@ func TestGetTarget(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
cache := NewOGTagCache(tt.target, false, time.Minute, false)
cache := NewOGTagCache(tt.target, config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: false,
})
u := &url.URL{
Path: tt.path,
@@ -175,7 +189,9 @@ func TestGetTarget(t *testing.T) {
func TestIntegrationGetOGTags_UnixSocket(t *testing.T) {
tempDir := t.TempDir()
socketPath := filepath.Join(tempDir, "anubis-test.sock")
// XXX(Xe): if this is named longer, macOS fails with `bind: invalid argument`
// because the unix socket path is too long. I love computers.
socketPath := filepath.Join(tempDir, "t")
// Ensure the socket does not exist initially
_ = os.Remove(socketPath)
@@ -222,7 +238,11 @@ func TestIntegrationGetOGTags_UnixSocket(t *testing.T) {
// Create cache instance pointing to the Unix socket
targetURL := "unix://" + socketPath
cache := NewOGTagCache(targetURL, true, 1*time.Minute, false)
cache := NewOGTagCache(targetURL, config.OpenGraph{
Enabled: true,
TimeToLive: time.Minute,
ConsiderHost: false,
})
// Create a dummy URL for the request (path and query matter)
testReqURL, _ := url.Parse("/some/page?query=1")

View File

@@ -6,13 +6,18 @@ import (
"testing"
"time"
"github.com/TecharoHQ/anubis/lib/policy/config"
"golang.org/x/net/html"
)
// TestExtractOGTags updated with correct expectations based on filtering logic
func TestExtractOGTags(t *testing.T) {
// Use a cache instance that reflects the default approved lists
testCache := NewOGTagCache("", false, time.Minute, false)
testCache := NewOGTagCache("", config.OpenGraph{
Enabled: false,
ConsiderHost: false,
TimeToLive: time.Minute,
})
// Manually set approved tags/prefixes based on the user request for clarity
testCache.approvedTags = []string{"description"}
testCache.approvedPrefixes = []string{"og:"}
@@ -189,7 +194,11 @@ func TestIsOGMetaTag(t *testing.T) {
func TestExtractMetaTagInfo(t *testing.T) {
// Use a cache instance that reflects the default approved lists
testCache := NewOGTagCache("", false, time.Minute, false)
testCache := NewOGTagCache("", config.OpenGraph{
Enabled: false,
ConsiderHost: false,
TimeToLive: time.Minute,
})
testCache.approvedTags = []string{"description"}
testCache.approvedPrefixes = []string{"og:"}

View File

@@ -15,6 +15,7 @@ import (
"time"
"github.com/golang-jwt/jwt/v5"
"github.com/google/cel-go/common/types"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
@@ -25,6 +26,7 @@ import (
"github.com/TecharoHQ/anubis/internal/dnsbl"
"github.com/TecharoHQ/anubis/internal/ogtags"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/localization"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/lib/policy/checker"
"github.com/TecharoHQ/anubis/lib/policy/config"
@@ -62,28 +64,39 @@ var (
)
type Server struct {
next http.Handler
mux *http.ServeMux
policy *policy.ParsedConfig
DNSBLCache *decaymap.Impl[string, dnsbl.DroneBLResponse]
OGTags *ogtags.OGTagCache
cookieName string
priv ed25519.PrivateKey
pub ed25519.PublicKey
opts Options
next http.Handler
mux *http.ServeMux
policy *policy.ParsedConfig
DNSBLCache *decaymap.Impl[string, dnsbl.DroneBLResponse]
OGTags *ogtags.OGTagCache
ed25519Priv ed25519.PrivateKey
hs512Secret []byte
opts Options
}
func (s *Server) getTokenKeyfunc() jwt.Keyfunc {
// return ED25519 key if HS512 is not set
if len(s.hs512Secret) == 0 {
return func(token *jwt.Token) (interface{}, error) {
return s.ed25519Priv.Public().(ed25519.PublicKey), nil
}
} else {
return func(token *jwt.Token) (interface{}, error) {
return s.hs512Secret, nil
}
}
}
func (s *Server) challengeFor(r *http.Request, difficulty int) string {
fp := sha256.Sum256(s.pub[:])
acceptLanguage := r.Header.Get("Accept-Language")
if len(acceptLanguage) > 5 {
acceptLanguage = acceptLanguage[:5]
var fp [32]byte
if len(s.hs512Secret) == 0 {
fp = sha256.Sum256(s.ed25519Priv.Public().(ed25519.PublicKey)[:])
} else {
fp = sha256.Sum256(s.hs512Secret)
}
challengeData := fmt.Sprintf(
"Accept-Language=%s,X-Real-IP=%s,User-Agent=%s,WeekTime=%s,Fingerprint=%x,Difficulty=%d",
acceptLanguage,
"X-Real-IP=%s,User-Agent=%s,WeekTime=%s,Fingerprint=%x,Difficulty=%d",
r.Header.Get("X-Real-Ip"),
r.UserAgent(),
time.Now().UTC().Round(24*7*time.Hour).Format(time.RFC3339),
@@ -113,7 +126,8 @@ func (s *Server) maybeReverseProxy(w http.ResponseWriter, r *http.Request, httpS
cr, rule, err := s.check(r)
if err != nil {
lg.Error("check failed", "err", err)
s.respondWithError(w, r, "Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to look for the logs around \"maybeReverseProxy\"")
localizer := localization.GetLocalizer(r)
s.respondWithError(w, r, fmt.Sprintf("%s \"maybeReverseProxy\"", localizer.T("internal_server_error")))
return
}
@@ -132,35 +146,33 @@ func (s *Server) maybeReverseProxy(w http.ResponseWriter, r *http.Request, httpS
return
}
ckie, err := r.Cookie(s.cookieName)
ckie, err := r.Cookie(anubis.CookieName)
if err != nil {
lg.Debug("cookie not found", "path", r.URL.Path)
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
if err := ckie.Valid(); err != nil {
lg.Debug("cookie is invalid", "err", err)
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
if time.Now().After(ckie.Expires) && !ckie.Expires.IsZero() {
lg.Debug("cookie expired", "path", r.URL.Path)
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
token, err := jwt.ParseWithClaims(ckie.Value, jwt.MapClaims{}, func(token *jwt.Token) (interface{}, error) {
return s.pub, nil
}, jwt.WithExpirationRequired(), jwt.WithStrictDecoding())
token, err := jwt.ParseWithClaims(ckie.Value, jwt.MapClaims{}, s.getTokenKeyfunc(), jwt.WithExpirationRequired(), jwt.WithStrictDecoding())
if err != nil || !token.Valid {
lg.Debug("invalid token", "path", r.URL.Path, "err", err)
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
@@ -168,7 +180,7 @@ func (s *Server) maybeReverseProxy(w http.ResponseWriter, r *http.Request, httpS
claims, ok := token.Claims.(jwt.MapClaims)
if !ok {
lg.Debug("invalid token claims type", "path", r.URL.Path)
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
@@ -176,14 +188,14 @@ func (s *Server) maybeReverseProxy(w http.ResponseWriter, r *http.Request, httpS
policyRule, ok := claims["policyRule"].(string)
if !ok {
lg.Debug("policyRule claim is not a string")
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
if policyRule != rule.Hash() {
lg.Debug("user originally passed with a different rule, issuing new challenge", "old", policyRule, "new", rule.Name)
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.RenderIndex(w, r, rule, httpStatusOnly)
return
}
@@ -199,23 +211,25 @@ func (s *Server) checkRules(w http.ResponseWriter, r *http.Request, cr policy.Ch
cookiePath = strings.TrimSuffix(anubis.BasePrefix, "/") + "/"
}
localizer := localization.GetLocalizer(r)
switch cr.Rule {
case config.RuleAllow:
lg.Debug("allowing traffic to origin (explicit)")
s.ServeHTTPNext(w, r)
return true
case config.RuleDeny:
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
lg.Info("explicit deny")
if rule == nil {
lg.Error("rule is nil, cannot calculate checksum")
s.respondWithError(w, r, "Internal Server Error: Please contact the administrator and ask them to look for the logs around \"maybeReverseProxy.RuleDeny\"")
s.respondWithError(w, r, fmt.Sprintf("%s \"maybeReverseProxy.RuleDeny\"", localizer.T("internal_server_error")))
return true
}
hash := rule.Hash()
lg.Debug("rule hash", "hash", hash)
s.respondWithStatus(w, r, fmt.Sprintf("Access Denied: error code %s", hash), s.policy.StatusCodes.Deny)
s.respondWithStatus(w, r, fmt.Sprintf("%s %s", localizer.T("access_denied"), hash), s.policy.StatusCodes.Deny)
return true
case config.RuleChallenge:
lg.Debug("challenge requested")
@@ -224,9 +238,9 @@ func (s *Server) checkRules(w http.ResponseWriter, r *http.Request, cr policy.Ch
s.RenderBench(w, r)
return true
default:
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
slog.Error("CONFIG ERROR: unknown rule", "rule", cr.Rule)
s.respondWithError(w, r, "Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to look for the logs around \"maybeReverseProxy.Rules\"")
s.respondWithError(w, r, fmt.Sprintf("%s \"maybeReverseProxy.Rules\"", localizer.T("internal_server_error")))
return true
}
return false
@@ -247,7 +261,12 @@ func (s *Server) handleDNSBL(w http.ResponseWriter, r *http.Request, ip string,
if resp != dnsbl.AllGood {
lg.Info("DNSBL hit", "status", resp.String())
s.respondWithStatus(w, r, fmt.Sprintf("DroneBL reported an entry: %s, see https://dronebl.org/lookup?ip=%s", resp.String(), ip), s.policy.StatusCodes.Deny)
localizer := localization.GetLocalizer(r)
s.respondWithStatus(w, r, fmt.Sprintf("%s: %s, %s https://dronebl.org/lookup?ip=%s",
localizer.T("dronebl_entry"),
resp.String(),
localizer.T("see_dronebl_lookup"),
ip), s.policy.StatusCodes.Deny)
return true
}
}
@@ -256,6 +275,7 @@ func (s *Server) handleDNSBL(w http.ResponseWriter, r *http.Request, ip string,
func (s *Server) MakeChallenge(w http.ResponseWriter, r *http.Request) {
lg := internal.GetRequestLogger(r)
localizer := localization.GetLocalizer(r)
redir := r.FormValue("redir")
if redir == "" {
@@ -265,7 +285,7 @@ func (s *Server) MakeChallenge(w http.ResponseWriter, r *http.Request) {
encoder.Encode(struct {
Error string `json:"error"`
}{
Error: "Invalid invocation of MakeChallenge",
Error: localizer.T("invalid_invocation"),
})
return
}
@@ -280,7 +300,7 @@ func (s *Server) MakeChallenge(w http.ResponseWriter, r *http.Request) {
err := encoder.Encode(struct {
Error string `json:"error"`
}{
Error: "Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to look for the logs around \"makeChallenge\"",
Error: fmt.Sprintf("%s \"makeChallenge\"", localizer.T("internal_server_error")),
})
if err != nil {
lg.Error("failed to encode error response", "err", err)
@@ -291,7 +311,7 @@ func (s *Server) MakeChallenge(w http.ResponseWriter, r *http.Request) {
lg = lg.With("check_result", cr)
chal := s.challengeFor(r, rule.Challenge.Difficulty)
s.SetCookie(w, anubis.TestCookieName, chal, "/")
s.SetCookie(w, CookieOpts{Host: r.Host, Name: anubis.TestCookieName, Value: chal})
err = encoder.Encode(struct {
Rules *config.ChallengeRules `json:"rules"`
@@ -311,6 +331,7 @@ func (s *Server) MakeChallenge(w http.ResponseWriter, r *http.Request) {
func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
lg := internal.GetRequestLogger(r)
localizer := localization.GetLocalizer(r)
// Adjust cookie path if base prefix is not empty
cookiePath := "/"
@@ -319,20 +340,20 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
}
if _, err := r.Cookie(anubis.TestCookieName); errors.Is(err, http.ErrNoCookie) {
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, anubis.TestCookieName, "/")
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.ClearCookie(w, CookieOpts{Name: anubis.TestCookieName, Host: r.Host})
lg.Warn("user has cookies disabled, this is not an anubis bug")
s.respondWithError(w, r, "Your browser is configured to disable cookies. Anubis requires cookies for the legitimate interest of making sure you are a valid client. Please enable cookies for this domain")
s.respondWithError(w, r, localizer.T("cookies_disabled"))
return
}
s.ClearCookie(w, anubis.TestCookieName, "/")
s.ClearCookie(w, CookieOpts{Name: anubis.TestCookieName, Host: r.Host})
redir := r.FormValue("redir")
redirURL, err := url.ParseRequestURI(redir)
if err != nil {
lg.Error("invalid redirect", "err", err)
s.respondWithError(w, r, "Invalid redirect")
s.respondWithError(w, r, localizer.T("invalid_redirect"))
return
}
// used by the path checker rule
@@ -340,18 +361,18 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
urlParsed, err := r.URL.Parse(redir)
if err != nil {
s.respondWithError(w, r, "Redirect URL not parseable")
s.respondWithError(w, r, localizer.T("redirect_not_parseable"))
return
}
if (len(urlParsed.Host) > 0 && len(s.opts.RedirectDomains) != 0 && !slices.Contains(s.opts.RedirectDomains, urlParsed.Host)) || urlParsed.Host != r.URL.Host {
s.respondWithError(w, r, "Redirect domain not allowed")
s.respondWithError(w, r, localizer.T("redirect_domain_not_allowed"))
return
}
cr, rule, err := s.check(r)
if err != nil {
lg.Error("check failed", "err", err)
s.respondWithError(w, r, "Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to look for the logs around \"passChallenge\"")
s.respondWithError(w, r, fmt.Sprintf("%s \"passChallenge\"", localizer.T("internal_server_error")))
return
}
lg = lg.With("check_result", cr)
@@ -359,7 +380,7 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
impl, ok := challenge.Get(rule.Challenge.Algorithm)
if !ok {
lg.Error("check failed", "err", err)
s.respondWithError(w, r, fmt.Sprintf("Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to file a bug as Anubis is trying to use challenge method %s but it does not exist in the challenge registry", rule.Challenge.Algorithm))
s.respondWithError(w, r, fmt.Sprintf("%s: %s", localizer.T("internal_server_error"), rule.Challenge.Algorithm))
return
}
@@ -368,7 +389,7 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
if err := impl.Validate(r, lg, rule, challengeStr); err != nil {
failedValidations.WithLabelValues(rule.Challenge.Algorithm).Inc()
var cerr *challenge.Error
s.ClearCookie(w, s.cookieName, cookiePath)
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
lg.Debug("challenge validate call failed", "err", err)
switch {
@@ -391,12 +412,12 @@ func (s *Server) PassChallenge(w http.ResponseWriter, r *http.Request) {
})
if err != nil {
lg.Error("failed to sign JWT", "err", err)
s.ClearCookie(w, s.cookieName, cookiePath)
s.respondWithError(w, r, "failed to sign JWT")
s.ClearCookie(w, CookieOpts{Path: cookiePath, Host: r.Host})
s.respondWithError(w, r, localizer.T("failed_to_sign_jwt"))
return
}
s.SetCookie(w, s.cookieName, tokenString, cookiePath)
s.SetCookie(w, CookieOpts{Path: cookiePath, Host: r.Host, Value: tokenString})
challengesValidated.WithLabelValues(rule.Challenge.Algorithm).Inc()
lg.Debug("challenge passed, redirecting to app")
@@ -411,12 +432,6 @@ func cr(name string, rule config.Rule, weight int) policy.CheckResult {
}
}
var (
weightOkayStatic = policy.NewStaticHashChecker("weight/okay")
weightMildSusStatic = policy.NewStaticHashChecker("weight/mild-suspicion")
weightVerySusStatic = policy.NewStaticHashChecker("weight/extreme-suspicion")
)
// Check evaluates the list of rules, and returns the result
func (s *Server) check(r *http.Request) (policy.CheckResult, *policy.Bot, error) {
host := r.Header.Get("X-Real-Ip")
@@ -448,34 +463,25 @@ func (s *Server) check(r *http.Request) (policy.CheckResult, *policy.Bot, error)
}
}
switch {
case weight <= 0:
return cr("weight/okay", config.RuleAllow, weight), &policy.Bot{
Challenge: &config.ChallengeRules{
Difficulty: s.policy.DefaultDifficulty,
ReportAs: s.policy.DefaultDifficulty,
Algorithm: config.DefaultAlgorithm,
},
Rules: weightOkayStatic,
}, nil
case weight > 0 && weight < 10:
return cr("weight/mild-suspicion", config.RuleChallenge, weight), &policy.Bot{
Challenge: &config.ChallengeRules{
Difficulty: s.policy.DefaultDifficulty,
ReportAs: s.policy.DefaultDifficulty,
Algorithm: "metarefresh",
},
Rules: weightMildSusStatic,
}, nil
case weight >= 10:
return cr("weight/extreme-suspicion", config.RuleChallenge, weight), &policy.Bot{
Challenge: &config.ChallengeRules{
Difficulty: s.policy.DefaultDifficulty,
ReportAs: s.policy.DefaultDifficulty,
Algorithm: "fast",
},
Rules: weightVerySusStatic,
}, nil
for _, t := range s.policy.Thresholds {
result, _, err := t.Program.ContextEval(r.Context(), &policy.ThresholdRequest{Weight: weight})
if err != nil {
slog.Error("error when evaluating threshold expression", "expression", t.Expression.String(), "err", err)
continue
}
var matches bool
if val, ok := result.(types.Bool); ok {
matches = bool(val)
}
if matches {
return cr("threshold/"+t.Name, t.Action, weight), &policy.Bot{
Challenge: t.Challenge,
Rules: &checker.List{},
}, nil
}
}
return cr("default/allow", config.RuleAllow, weight), &policy.Bot{

View File

@@ -24,12 +24,16 @@ func init() {
internal.InitSlog("debug")
}
func loadPolicies(t *testing.T, fname string) *policy.ParsedConfig {
func loadPolicies(t *testing.T, fname string, difficulty int) *policy.ParsedConfig {
t.Helper()
ctx := thothmock.WithMockThoth(t)
anubisPolicy, err := LoadPoliciesOrDefault(ctx, fname, anubis.DefaultDifficulty)
if fname == "" {
fname = "./testdata/test_config.yaml"
}
anubisPolicy, err := LoadPoliciesOrDefault(ctx, fname, difficulty)
if err != nil {
t.Fatal(err)
}
@@ -40,6 +44,10 @@ func loadPolicies(t *testing.T, fname string) *policy.ParsedConfig {
func spawnAnubis(t *testing.T, opts Options) *Server {
t.Helper()
if opts.Policy == nil {
opts.Policy = loadPolicies(t, "", 4)
}
s, err := New(opts)
if err != nil {
t.Fatalf("can't construct libanubis.Server: %v", err)
@@ -176,14 +184,11 @@ func TestLoadPolicies(t *testing.T) {
// Regression test for CVE-2025-24369
func TestCVE2025_24369(t *testing.T) {
pol := loadPolicies(t, "")
pol.DefaultDifficulty = 4
pol := loadPolicies(t, "", anubis.DefaultDifficulty)
srv := spawnAnubis(t, Options{
Next: http.NewServeMux(),
Policy: pol,
CookieName: t.Name(),
})
ts := httptest.NewServer(internal.RemoteXRealIP(true, "tcp", srv))
@@ -200,8 +205,7 @@ func TestCVE2025_24369(t *testing.T) {
}
func TestCookieCustomExpiration(t *testing.T) {
pol := loadPolicies(t, "")
pol.DefaultDifficulty = 0
pol := loadPolicies(t, "", 0)
ckieExpiration := 10 * time.Minute
srv := spawnAnubis(t, Options{
@@ -229,13 +233,13 @@ func TestCookieCustomExpiration(t *testing.T) {
var ckie *http.Cookie
for _, cookie := range resp.Cookies() {
t.Logf("%#v", cookie)
if cookie.Name == srv.cookieName {
if cookie.Name == anubis.CookieName {
ckie = cookie
break
}
}
if ckie == nil {
t.Errorf("Cookie %q not found", srv.cookieName)
t.Errorf("Cookie %q not found", anubis.CookieName)
return
}
@@ -250,8 +254,7 @@ func TestCookieCustomExpiration(t *testing.T) {
}
func TestCookieSettings(t *testing.T) {
pol := loadPolicies(t, "")
pol.DefaultDifficulty = 0
pol := loadPolicies(t, "", 0)
srv := spawnAnubis(t, Options{
Next: http.NewServeMux(),
@@ -259,7 +262,7 @@ func TestCookieSettings(t *testing.T) {
CookieDomain: "127.0.0.1",
CookiePartitioned: true,
CookieName: t.Name(),
CookieSecure: true,
CookieExpiration: anubis.CookieDefaultExpirationTime,
})
@@ -281,13 +284,13 @@ func TestCookieSettings(t *testing.T) {
var ckie *http.Cookie
for _, cookie := range resp.Cookies() {
t.Logf("%#v", cookie)
if cookie.Name == srv.cookieName {
if cookie.Name == anubis.CookieName {
ckie = cookie
break
}
}
if ckie == nil {
t.Errorf("Cookie %q not found", srv.cookieName)
t.Errorf("Cookie %q not found", anubis.CookieName)
return
}
@@ -307,6 +310,10 @@ func TestCookieSettings(t *testing.T) {
if ckie.Partitioned != srv.opts.CookiePartitioned {
t.Errorf("wanted partitioned flag %v, got: %v", srv.opts.CookiePartitioned, ckie.Partitioned)
}
if ckie.Secure != srv.opts.CookieSecure {
t.Errorf("wanted secure flag %v, got: %v", srv.opts.CookieSecure, ckie.Secure)
}
}
func TestCheckDefaultDifficultyMatchesPolicy(t *testing.T) {
@@ -316,10 +323,7 @@ func TestCheckDefaultDifficultyMatchesPolicy(t *testing.T) {
for i := 1; i < 10; i++ {
t.Run(fmt.Sprint(i), func(t *testing.T) {
anubisPolicy, err := LoadPoliciesOrDefault(t.Context(), "", i)
if err != nil {
t.Fatal(err)
}
anubisPolicy := loadPolicies(t, "", i)
s, err := New(Options{
Next: h,
@@ -337,11 +341,13 @@ func TestCheckDefaultDifficultyMatchesPolicy(t *testing.T) {
req.Header.Add("X-Real-Ip", "127.0.0.1")
_, bot, err := s.check(req)
cr, bot, err := s.check(req)
if err != nil {
t.Fatal(err)
}
t.Log(cr.Name)
if bot.Challenge.Difficulty != i {
t.Errorf("Challenge.Difficulty is wrong, wanted %d, got: %d", i, bot.Challenge.Difficulty)
}
@@ -389,8 +395,7 @@ func TestBasePrefix(t *testing.T) {
// Reset the global BasePrefix before each test
anubis.BasePrefix = ""
pol := loadPolicies(t, "")
pol.DefaultDifficulty = 4
pol := loadPolicies(t, "", 4)
srv := spawnAnubis(t, Options{
Next: h,
@@ -518,8 +523,7 @@ func TestCustomStatusCodes(t *testing.T) {
"DENY": 403,
}
pol := loadPolicies(t, "./testdata/aggressive_403.yaml")
pol.DefaultDifficulty = 4
pol := loadPolicies(t, "./testdata/aggressive_403.yaml", 4)
srv := spawnAnubis(t, Options{
Next: h,
@@ -553,7 +557,7 @@ func TestCustomStatusCodes(t *testing.T) {
func TestCloudflareWorkersRule(t *testing.T) {
for _, variant := range []string{"cel", "header"} {
t.Run(variant, func(t *testing.T) {
pol := loadPolicies(t, "./testdata/cloudflare-workers-"+variant+".yaml")
pol := loadPolicies(t, "./testdata/cloudflare-workers-"+variant+".yaml", 0)
h := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "OK")
@@ -609,8 +613,7 @@ func TestCloudflareWorkersRule(t *testing.T) {
}
func TestRuleChange(t *testing.T) {
pol := loadPolicies(t, "testdata/rule_change.yaml")
pol.DefaultDifficulty = 0
pol := loadPolicies(t, "testdata/rule_change.yaml", 0)
ckieExpiration := 10 * time.Minute
srv := spawnAnubis(t, Options{
@@ -618,7 +621,6 @@ func TestRuleChange(t *testing.T) {
Policy: pol,
CookieDomain: "127.0.0.1",
CookieName: t.Name(),
CookieExpiration: ckieExpiration,
})

View File

@@ -7,6 +7,7 @@ import (
"sync"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/lib/policy/config"
"github.com/a-h/templ"
)
@@ -40,12 +41,19 @@ func Methods() []string {
return result
}
type IssueInput struct {
Impressum *config.Impressum
Rule *policy.Bot
Challenge string
OGTags map[string]string
}
type Impl interface {
// Setup registers any additional routes with the Impl for assets or API routes.
Setup(mux *http.ServeMux)
// Issue a new challenge to the user, called by the Anubis.
Issue(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string, ogTags map[string]string) (templ.Component, error)
Issue(r *http.Request, lg *slog.Logger, in *IssueInput) (templ.Component, error)
// Validate a challenge, making sure that it passes muster.
Validate(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string) error

View File

@@ -8,6 +8,7 @@ import (
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/localization"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/web"
"github.com/a-h/templ"
@@ -23,7 +24,7 @@ type Impl struct{}
func (i *Impl) Setup(mux *http.ServeMux) {}
func (i *Impl) Issue(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string, ogTags map[string]string) (templ.Component, error) {
func (i *Impl) Issue(r *http.Request, lg *slog.Logger, in *challenge.IssueInput) (templ.Component, error) {
u, err := r.URL.Parse(anubis.BasePrefix + "/.within.website/x/cmd/anubis/api/pass-challenge")
if err != nil {
return nil, fmt.Errorf("can't render page: %w", err)
@@ -31,10 +32,12 @@ func (i *Impl) Issue(r *http.Request, lg *slog.Logger, rule *policy.Bot, challen
q := u.Query()
q.Set("redir", r.URL.String())
q.Set("challenge", challenge)
q.Set("challenge", in.Challenge)
u.RawQuery = q.Encode()
component, err := web.BaseWithChallengeAndOGTags("Making sure you're not a bot!", page(challenge, u.String(), rule.Challenge.Difficulty), challenge, rule.Challenge, ogTags)
loc := localization.GetLocalizer(r)
component, err := web.BaseWithChallengeAndOGTags(loc.T("making_sure_not_bot"), page(in.Challenge, u.String(), in.Rule.Challenge.Difficulty, loc), in.Impressum, in.Challenge, in.Rule.Challenge, in.OGTags, loc)
if err != nil {
return nil, fmt.Errorf("can't render page: %w", err)
}

View File

@@ -4,14 +4,15 @@ import (
"fmt"
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/lib/localization"
)
templ page(challenge, redir string, difficulty int) {
templ page(challenge, redir string, difficulty int, loc *localization.SimpleLocalizer) {
<div class="centered-div">
<img id="image" style="width:100%;max-width:256px;" src={ anubis.BasePrefix + "/.within.website/x/cmd/anubis/static/img/pensive.webp?cacheBuster=" + anubis.Version }/>
<img style="display:none;" style="width:100%;max-width:256px;" src={ anubis.BasePrefix + "/.within.website/x/cmd/anubis/static/img/happy.webp?cacheBuster=" + anubis.Version }/>
<p id="status">Loading...</p>
<p>Please wait a moment while we ensure the security of your connection.</p>
<p id="status">{ loc.T("loading") }</p>
<p>{ loc.T("connection_security") }</p>
<meta http-equiv="refresh" content={ fmt.Sprintf("%d; url=%s", difficulty, redir) }/>
</div>
}

View File

@@ -1,6 +1,6 @@
// Code generated by templ - DO NOT EDIT.
// templ: version: v0.3.898
// templ: version: v0.3.906
package metarefresh
//lint:file-ignore SA4006 This context is only used if a nested component is present.
@@ -12,9 +12,10 @@ import (
"fmt"
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/lib/localization"
)
func page(challenge, redir string, difficulty int) templ.Component {
func page(challenge, redir string, difficulty int, loc *localization.SimpleLocalizer) templ.Component {
return templruntime.GeneratedTemplate(func(templ_7745c5c3_Input templruntime.GeneratedComponentInput) (templ_7745c5c3_Err error) {
templ_7745c5c3_W, ctx := templ_7745c5c3_Input.Writer, templ_7745c5c3_Input.Context
if templ_7745c5c3_CtxErr := ctx.Err(); templ_7745c5c3_CtxErr != nil {
@@ -42,7 +43,7 @@ func page(challenge, redir string, difficulty int) templ.Component {
var templ_7745c5c3_Var2 string
templ_7745c5c3_Var2, templ_7745c5c3_Err = templ.JoinStringErrs(anubis.BasePrefix + "/.within.website/x/cmd/anubis/static/img/pensive.webp?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 11, Col: 165}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 12, Col: 165}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var2))
if templ_7745c5c3_Err != nil {
@@ -55,26 +56,52 @@ func page(challenge, redir string, difficulty int) templ.Component {
var templ_7745c5c3_Var3 string
templ_7745c5c3_Var3, templ_7745c5c3_Err = templ.JoinStringErrs(anubis.BasePrefix + "/.within.website/x/cmd/anubis/static/img/happy.webp?cacheBuster=" + anubis.Version)
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 12, Col: 174}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 13, Col: 174}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var3))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "\"><p id=\"status\">Loading...</p><p>Please wait a moment while we ensure the security of your connection.</p><meta http-equiv=\"refresh\" content=\"")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 3, "\"><p id=\"status\">")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var4 string
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d; url=%s", difficulty, redir))
templ_7745c5c3_Var4, templ_7745c5c3_Err = templ.JoinStringErrs(loc.T("loading"))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 15, Col: 83}
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 14, Col: 35}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var4))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "\"></div>")
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 4, "</p><p>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var5 string
templ_7745c5c3_Var5, templ_7745c5c3_Err = templ.JoinStringErrs(loc.T("connection_security"))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 15, Col: 35}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var5))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 5, "</p><meta http-equiv=\"refresh\" content=\"")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
var templ_7745c5c3_Var6 string
templ_7745c5c3_Var6, templ_7745c5c3_Err = templ.JoinStringErrs(fmt.Sprintf("%d; url=%s", difficulty, redir))
if templ_7745c5c3_Err != nil {
return templ.Error{Err: templ_7745c5c3_Err, FileName: `metarefresh.templ`, Line: 16, Col: 83}
}
_, templ_7745c5c3_Err = templ_7745c5c3_Buffer.WriteString(templ.EscapeString(templ_7745c5c3_Var6))
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}
templ_7745c5c3_Err = templruntime.WriteString(templ_7745c5c3_Buffer, 6, "\"></div>")
if templ_7745c5c3_Err != nil {
return templ_7745c5c3_Err
}

View File

@@ -10,6 +10,7 @@ import (
"github.com/TecharoHQ/anubis/internal"
chall "github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/localization"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/web"
"github.com/a-h/templ"
@@ -28,8 +29,9 @@ func (i *Impl) Setup(mux *http.ServeMux) {
/* no implementation required */
}
func (i *Impl) Issue(r *http.Request, lg *slog.Logger, rule *policy.Bot, challenge string, ogTags map[string]string) (templ.Component, error) {
component, err := web.BaseWithChallengeAndOGTags("Making sure you're not a bot!", web.Index(), challenge, rule.Challenge, ogTags)
func (i *Impl) Issue(r *http.Request, lg *slog.Logger, in *chall.IssueInput) (templ.Component, error) {
loc := localization.GetLocalizer(r)
component, err := web.BaseWithChallengeAndOGTags(loc.T("making_sure_not_bot"), web.Index(loc), in.Impressum, in.Challenge, in.Rule.Challenge, in.OGTags, loc)
if err != nil {
return nil, fmt.Errorf("can't render page: %w", err)
}

View File

@@ -124,7 +124,12 @@ func TestBasic(t *testing.T) {
t.Run(cs.name, func(t *testing.T) {
lg := slog.With()
if _, err := i.Issue(cs.req, lg, bot, cs.challengeStr, nil); err != nil {
inp := &challenge.IssueInput{
Rule: bot,
Challenge: cs.challengeStr,
}
if _, err := i.Issue(cs.req, lg, inp); err != nil {
t.Errorf("can't issue challenge: %v", err)
}

View File

@@ -20,28 +20,31 @@ import (
"github.com/TecharoHQ/anubis/internal/dnsbl"
"github.com/TecharoHQ/anubis/internal/ogtags"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/localization"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/lib/policy/config"
"github.com/TecharoHQ/anubis/web"
"github.com/TecharoHQ/anubis/xess"
"github.com/a-h/templ"
)
type Options struct {
Next http.Handler
Policy *policy.ParsedConfig
Target string
CookieDomain string
CookieName string
BasePrefix string
WebmasterEmail string
RedirectDomains []string
PrivateKey ed25519.PrivateKey
CookieExpiration time.Duration
OGTimeToLive time.Duration
StripBasePrefix bool
OGCacheConsidersHost bool
OGPassthrough bool
CookiePartitioned bool
ServeRobotsTXT bool
Next http.Handler
Policy *policy.ParsedConfig
Target string
CookieDynamicDomain bool
CookieDomain string
CookieExpiration time.Duration
CookiePartitioned bool
BasePrefix string
WebmasterEmail string
RedirectDomains []string
ED25519PrivateKey ed25519.PrivateKey
HS512Secret []byte
StripBasePrefix bool
OpenGraph config.OpenGraph
ServeRobotsTXT bool
CookieSecure bool
}
func LoadPoliciesOrDefault(ctx context.Context, fname string, defaultDifficulty int) (*policy.ParsedConfig, error) {
@@ -88,32 +91,25 @@ func LoadPoliciesOrDefault(ctx context.Context, fname string, defaultDifficulty
}
func New(opts Options) (*Server, error) {
if opts.PrivateKey == nil {
if opts.ED25519PrivateKey == nil && opts.HS512Secret == nil {
slog.Debug("opts.PrivateKey not set, generating a new one")
_, priv, err := ed25519.GenerateKey(rand.Reader)
if err != nil {
return nil, fmt.Errorf("lib: can't generate private key: %v", err)
}
opts.PrivateKey = priv
opts.ED25519PrivateKey = priv
}
anubis.BasePrefix = opts.BasePrefix
cookieName := anubis.CookieName
if opts.CookieDomain != "" {
cookieName = anubis.WithDomainCookieName + opts.CookieDomain
}
result := &Server{
next: opts.Next,
priv: opts.PrivateKey,
pub: opts.PrivateKey.Public().(ed25519.PublicKey),
policy: opts.Policy,
opts: opts,
DNSBLCache: decaymap.New[string, dnsbl.DroneBLResponse](),
OGTags: ogtags.NewOGTagCache(opts.Target, opts.OGPassthrough, opts.OGTimeToLive, opts.OGCacheConsidersHost),
cookieName: cookieName,
next: opts.Next,
ed25519Priv: opts.ED25519PrivateKey,
hs512Secret: opts.HS512Secret,
policy: opts.Policy,
opts: opts,
DNSBLCache: decaymap.New[string, dnsbl.DroneBLResponse](),
OGTags: ogtags.NewOGTagCache(opts.Target, opts.Policy.OpenGraph),
}
mux := http.NewServeMux()
@@ -150,6 +146,14 @@ func New(opts Options) (*Server, error) {
}), "GET")
}
if opts.Policy.Impressum != nil {
registerWithPrefix(anubis.APIPrefix+"imprint", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
templ.Handler(
web.Base(opts.Policy.Impressum.Page.Title, opts.Policy.Impressum.Page, opts.Policy.Impressum, localization.GetLocalizer(r)),
).ServeHTTP(w, r)
}), "GET")
}
registerWithPrefix(anubis.APIPrefix+"pass-challenge", http.HandlerFunc(result.PassChallenge), "GET")
registerWithPrefix(anubis.APIPrefix+"check", http.HandlerFunc(result.maybeReverseProxyHttpStatusOnly), "")
registerWithPrefix("/", http.HandlerFunc(result.maybeReverseProxyOrPage), "")

View File

@@ -26,7 +26,7 @@ func TestBadConfigs(t *testing.T) {
for _, st := range finfos {
st := st
t.Run(st.Name(), func(t *testing.T) {
if _, err := LoadPoliciesOrDefault(t.Context(), filepath.Join("policy", "config", "testdata", "good", st.Name()), anubis.DefaultDifficulty); err == nil {
if _, err := LoadPoliciesOrDefault(t.Context(), filepath.Join("policy", "config", "testdata", "bad", st.Name()), anubis.DefaultDifficulty); err == nil {
t.Fatal(err)
} else {
t.Log(err)

View File

@@ -4,6 +4,7 @@ import (
"fmt"
"math/rand"
"net/http"
"regexp"
"slices"
"strings"
"time"
@@ -11,33 +12,81 @@ import (
"github.com/TecharoHQ/anubis"
"github.com/TecharoHQ/anubis/internal"
"github.com/TecharoHQ/anubis/lib/challenge"
"github.com/TecharoHQ/anubis/lib/localization"
"github.com/TecharoHQ/anubis/lib/policy"
"github.com/TecharoHQ/anubis/web"
"github.com/a-h/templ"
"github.com/golang-jwt/jwt/v5"
"golang.org/x/net/publicsuffix"
)
func (s *Server) SetCookie(w http.ResponseWriter, name, value, path string) {
var domainMatchRegexp = regexp.MustCompile(`^((xn--)?[a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,}$`)
type CookieOpts struct {
Value string
Host string
Path string
Name string
Expiry time.Duration
}
func (s *Server) SetCookie(w http.ResponseWriter, cookieOpts CookieOpts) {
var domain = s.opts.CookieDomain
var name = anubis.CookieName
var path = "/"
if cookieOpts.Name != "" {
name = cookieOpts.Name
}
if cookieOpts.Path != "" {
path = cookieOpts.Path
}
if s.opts.CookieDynamicDomain && domainMatchRegexp.MatchString(cookieOpts.Host) {
if etld, err := publicsuffix.EffectiveTLDPlusOne(cookieOpts.Host); err == nil {
domain = etld
}
}
if cookieOpts.Expiry == 0 {
cookieOpts.Expiry = s.opts.CookieExpiration
}
http.SetCookie(w, &http.Cookie{
Name: name,
Value: value,
Expires: time.Now().Add(s.opts.CookieExpiration),
SameSite: http.SameSiteLaxMode,
Domain: s.opts.CookieDomain,
Value: cookieOpts.Value,
Expires: time.Now().Add(cookieOpts.Expiry),
SameSite: http.SameSiteNoneMode,
Domain: domain,
Secure: s.opts.CookieSecure,
Partitioned: s.opts.CookiePartitioned,
Path: path,
})
}
func (s *Server) ClearCookie(w http.ResponseWriter, name, path string) {
func (s *Server) ClearCookie(w http.ResponseWriter, cookieOpts CookieOpts) {
var domain = s.opts.CookieDomain
var name = anubis.CookieName
var path = "/"
if cookieOpts.Name != "" {
name = cookieOpts.Name
}
if cookieOpts.Path != "" {
path = cookieOpts.Path
}
if s.opts.CookieDynamicDomain && domainMatchRegexp.MatchString(cookieOpts.Host) {
if etld, err := publicsuffix.EffectiveTLDPlusOne(cookieOpts.Host); err == nil {
domain = etld
}
}
http.SetCookie(w, &http.Cookie{
Name: name,
Value: "",
MaxAge: -1,
Expires: time.Now().Add(-1 * time.Minute),
SameSite: http.SameSiteLaxMode,
SameSite: http.SameSiteNoneMode,
Partitioned: s.opts.CookiePartitioned,
Domain: s.opts.CookieDomain,
Domain: domain,
Secure: s.opts.CookieSecure,
Path: path,
})
}
@@ -63,9 +112,11 @@ func randomChance(n int) bool {
}
func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, rule *policy.Bot, returnHTTPStatusOnly bool) {
localizer := localization.GetLocalizer(r)
if returnHTTPStatusOnly {
w.WriteHeader(http.StatusUnauthorized)
w.Write([]byte("Authorization required"))
w.Write([]byte(localizer.T("authorization_required")))
return
}
@@ -73,14 +124,14 @@ func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, rule *polic
if !strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") && randomChance(64) {
lg.Error("client was given a challenge but does not in fact support gzip compression")
s.respondWithError(w, r, "Client Error: Please ensure your browser is up to date and try again later.")
s.respondWithError(w, r, localizer.T("client_error_browser"))
}
challengesIssued.WithLabelValues("embedded").Add(1)
challengeStr := s.challengeFor(r, rule.Challenge.Difficulty)
var ogTags map[string]string = nil
if s.opts.OGPassthrough {
if s.opts.OpenGraph.Enabled {
var err error
ogTags, err = s.OGTags.GetOGTags(r.URL, r.Host)
if err != nil {
@@ -88,24 +139,32 @@ func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, rule *polic
}
}
http.SetCookie(w, &http.Cookie{
Name: anubis.TestCookieName,
Value: challengeStr,
Expires: time.Now().Add(30 * time.Minute),
Path: "/",
s.SetCookie(w, CookieOpts{
Value: challengeStr,
Host: r.Host,
Path: "/",
Name: anubis.TestCookieName,
Expiry: 30 * time.Minute,
})
impl, ok := challenge.Get(rule.Challenge.Algorithm)
if !ok {
lg.Error("check failed", "err", "can't get algorithm", "algorithm", rule.Challenge.Algorithm)
s.respondWithError(w, r, fmt.Sprintf("Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to file a bug as Anubis is trying to use challenge method %s but it does not exist in the challenge registry", rule.Challenge.Algorithm))
s.respondWithError(w, r, fmt.Sprintf("%s: %s", localizer.T("internal_server_error"), rule.Challenge.Algorithm))
return
}
component, err := impl.Issue(r, lg, rule, challengeStr, ogTags)
in := &challenge.IssueInput{
Impressum: s.policy.Impressum,
Rule: rule,
Challenge: challengeStr,
OGTags: ogTags,
}
component, err := impl.Issue(r, lg, in)
if err != nil {
lg.Error("[unexpected] render failed, please open an issue", "err", err) // This is likely a bug in the template. Should never be triggered as CI tests for this.
s.respondWithError(w, r, "Internal Server Error: please contact the administrator and ask them to look for the logs around \"RenderIndex\"")
s.respondWithError(w, r, fmt.Sprintf("%s \"RenderIndex\"", localizer.T("internal_server_error")))
return
}
@@ -117,8 +176,10 @@ func (s *Server) RenderIndex(w http.ResponseWriter, r *http.Request, rule *polic
}
func (s *Server) RenderBench(w http.ResponseWriter, r *http.Request) {
localizer := localization.GetLocalizer(r)
templ.Handler(
web.Base("Benchmarking Anubis!", web.Bench()),
web.Base(localizer.T("benchmarking_anubis"), web.Bench(localizer), s.policy.Impressum, localizer),
).ServeHTTP(w, r)
}
@@ -127,7 +188,9 @@ func (s *Server) respondWithError(w http.ResponseWriter, r *http.Request, messag
}
func (s *Server) respondWithStatus(w http.ResponseWriter, r *http.Request, msg string, status int) {
templ.Handler(web.Base("Oh noes!", web.ErrorPage(msg, s.opts.WebmasterEmail)), templ.WithStatus(status)).ServeHTTP(w, r)
localizer := localization.GetLocalizer(r)
templ.Handler(web.Base(localizer.T("oh_noes"), web.ErrorPage(msg, s.opts.WebmasterEmail, localizer), s.policy.Impressum, localizer), templ.WithStatus(status)).ServeHTTP(w, r)
}
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
@@ -162,15 +225,17 @@ func (s *Server) stripBasePrefixFromRequest(r *http.Request) *http.Request {
func (s *Server) ServeHTTPNext(w http.ResponseWriter, r *http.Request) {
if s.next == nil {
localizer := localization.GetLocalizer(r)
redir := r.FormValue("redir")
urlParsed, err := r.URL.Parse(redir)
if err != nil {
s.respondWithStatus(w, r, "Redirect URL not parseable", http.StatusBadRequest)
s.respondWithStatus(w, r, localizer.T("redirect_not_parseable"), http.StatusBadRequest)
return
}
if (len(urlParsed.Host) > 0 && len(s.opts.RedirectDomains) != 0 && !slices.Contains(s.opts.RedirectDomains, urlParsed.Host)) || urlParsed.Host != r.URL.Host {
s.respondWithStatus(w, r, "Redirect domain not allowed", http.StatusBadRequest)
s.respondWithStatus(w, r, localizer.T("redirect_domain_not_allowed"), http.StatusBadRequest)
return
}
@@ -180,7 +245,7 @@ func (s *Server) ServeHTTPNext(w http.ResponseWriter, r *http.Request) {
}
templ.Handler(
web.Base("You are not a bot!", web.StaticHappy()),
web.Base(localizer.T("you_are_not_a_bot"), web.StaticHappy(localizer), s.policy.Impressum, localizer),
).ServeHTTP(w, r)
} else {
requestsProxied.WithLabelValues(r.Host).Inc()
@@ -194,5 +259,9 @@ func (s *Server) signJWT(claims jwt.MapClaims) (string, error) {
claims["nbf"] = time.Now().Add(-1 * time.Minute).Unix()
claims["exp"] = time.Now().Add(s.opts.CookieExpiration).Unix()
return jwt.NewWithClaims(jwt.SigningMethodEdDSA, claims).SignedString(s.priv)
if len(s.hs512Secret) == 0 {
return jwt.NewWithClaims(jwt.SigningMethodEdDSA, claims).SignedString(s.ed25519Priv)
} else {
return jwt.NewWithClaims(jwt.SigningMethodHS512, claims).SignedString(s.hs512Secret)
}
}

View File

@@ -7,11 +7,55 @@ import (
"github.com/TecharoHQ/anubis"
)
func TestSetCookie(t *testing.T) {
for _, tt := range []struct {
name string
options Options
host string
cookieName string
}{
{
name: "basic",
options: Options{},
host: "",
cookieName: anubis.CookieName,
},
{
name: "domain techaro.lol",
options: Options{CookieDomain: "techaro.lol"},
host: "",
cookieName: anubis.CookieName,
},
{
name: "dynamic cookie domain",
options: Options{CookieDynamicDomain: true},
host: "techaro.lol",
cookieName: anubis.CookieName,
},
} {
t.Run(tt.name, func(t *testing.T) {
srv := spawnAnubis(t, tt.options)
rw := httptest.NewRecorder()
srv.SetCookie(rw, CookieOpts{Value: "test", Host: tt.host})
resp := rw.Result()
cookies := resp.Cookies()
ckie := cookies[0]
if ckie.Name != tt.cookieName {
t.Errorf("wanted cookie named %q, got cookie named %q", tt.cookieName, ckie.Name)
}
})
}
}
func TestClearCookie(t *testing.T) {
srv := spawnAnubis(t, Options{})
rw := httptest.NewRecorder()
srv.ClearCookie(rw, srv.cookieName, "/")
srv.ClearCookie(rw, CookieOpts{Host: "localhost"})
resp := rw.Result()
@@ -36,7 +80,7 @@ func TestClearCookieWithDomain(t *testing.T) {
srv := spawnAnubis(t, Options{CookieDomain: "techaro.lol"})
rw := httptest.NewRecorder()
srv.ClearCookie(rw, srv.cookieName, "/")
srv.ClearCookie(rw, CookieOpts{Host: "localhost"})
resp := rw.Result()
@@ -48,8 +92,37 @@ func TestClearCookieWithDomain(t *testing.T) {
ckie := cookies[0]
if ckie.Name != srv.cookieName {
t.Errorf("wanted cookie named %q, got cookie named %q", srv.cookieName, ckie.Name)
if ckie.Name != anubis.CookieName {
t.Errorf("wanted cookie named %q, got cookie named %q", anubis.CookieName, ckie.Name)
}
if ckie.MaxAge != -1 {
t.Errorf("wanted cookie max age of -1, got: %d", ckie.MaxAge)
}
}
func TestClearCookieWithDynamicDomain(t *testing.T) {
srv := spawnAnubis(t, Options{CookieDynamicDomain: true})
rw := httptest.NewRecorder()
srv.ClearCookie(rw, CookieOpts{Host: "subdomain.xeiaso.net"})
resp := rw.Result()
cookies := resp.Cookies()
if len(cookies) != 1 {
t.Errorf("wanted 1 cookie, got %d cookies", len(cookies))
}
ckie := cookies[0]
if ckie.Name != anubis.CookieName {
t.Errorf("wanted cookie named %q, got cookie named %q", anubis.CookieName, ckie.Name)
}
if ckie.Domain != "xeiaso.net" {
t.Errorf("wanted cookie domain %q, got cookie domain %q", "xeiaso.net", ckie.Domain)
}
if ckie.MaxAge != -1 {

View File

@@ -0,0 +1,63 @@
{
"loading": "Loading...",
"why_am_i_seeing": "Why am I seeing this?",
"protected_by": "Protected by",
"made_with": "Made with ❤️ in 🇨🇦",
"mascot_design": "Mascot design by",
"ai_companies_explanation": "You are seeing this because the administrator of this website has set up Anubis to protect the server against the scourge of AI companies aggressively scraping websites. This can and does cause downtime for the websites, which makes their resources inaccessible for everyone.",
"anubis_compromise": "Anubis is a compromise. Anubis uses a Proof-of-Work scheme in the vein of Hashcash, a proposed proof-of-work scheme for reducing email spam. The idea is that at individual scales the additional load is ignorable, but at mass scraper levels it adds up and makes scraping much more expensive.",
"hack_purpose": "Ultimately, this is a hack whose real purpose is to give a \"good enough\" placeholder solution so that more time can be spent on fingerprinting and identifying headless browsers (EG: via how they do font rendering) so that the challenge proof of work page doesn't need to be presented to users that are much more likely to be legitimate.",
"jshelter_note": "Please note that Anubis requires the use of modern JavaScript features that plugins like JShelter will disable. Please disable JShelter or other such plugins for this domain.",
"version_info": "This website is running Anubis version",
"try_again": "Try again",
"go_home": "Go home",
"contact_webmaster": "or if you believe you should not be blocked, please contact the webmaster at",
"connection_security": "Please wait a moment while we ensure the security of your connection.",
"javascript_required": "Sadly, you must enable JavaScript to get past this challenge. This is required because AI companies have changed the social contract around how website hosting works. A no-JS solution is a work-in-progress.",
"benchmark_requires_js": "Running the benchmark tool requires JavaScript to be enabled.",
"difficulty": "Difficulty:",
"algorithm": "Algorithm:",
"compare": "Compare:",
"time": "Time",
"iters": "Iters",
"time_a": "Time A",
"iters_a": "Iters A",
"time_b": "Time B",
"iters_b": "Iters B",
"static_check_endpoint": "This is just a check endpoint for your reverse proxy to use.",
"authorization_required": "Authorization required",
"cookies_disabled": "Your browser is configured to disable cookies. Anubis requires cookies for the legitimate interest of making sure you are a valid client. Please enable cookies for this domain",
"access_denied": "Access Denied: error code",
"dronebl_entry": "DroneBL reported an entry",
"see_dronebl_lookup": "see",
"internal_server_error": "Internal Server Error: administrator has misconfigured Anubis. Please contact the administrator and ask them to look for the logs around",
"invalid_redirect": "Invalid redirect",
"redirect_not_parseable": "Redirect URL not parseable",
"redirect_domain_not_allowed": "Redirect domain not allowed",
"failed_to_sign_jwt": "failed to sign JWT",
"invalid_invocation": "Invalid invocation of MakeChallenge",
"client_error_browser": "Client Error: Please ensure your browser is up to date and try again later.",
"oh_noes": "Oh noes!",
"benchmarking_anubis": "Benchmarking Anubis!",
"you_are_not_a_bot": "You are not a bot!",
"making_sure_not_bot": "Making sure you're not a bot!",
"celphase": "CELPHASE",
"js_web_crypto_error": "Your browser doesn't have a functioning web.crypto element. Are you viewing this over a secure context?",
"js_web_workers_error": "Your browser doesn't support web workers (Anubis uses this to avoid freezing your browser). Do you have a plugin like JShelter installed?",
"js_cookies_error": "Your browser doesn't store cookies. Anubis uses cookies to determine which clients have passed challenges by storing a signed token in a cookie. Please enable storing cookies for this domain. The names of the cookies Anubis stores may vary without notice. Cookie names and values are not part of the public API.",
"js_context_not_secure": "Your context is not secure!",
"js_context_not_secure_msg": "Try connecting over HTTPS or let the admin know to set up HTTPS. For more information, see <a href=\"https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts#when_is_a_context_considered_secure\">MDN</a>.",
"js_calculating": "Calculating...",
"js_missing_feature": "Missing feature",
"js_challenge_error": "Challenge error!",
"js_challenge_error_msg": "Failed to resolve check algorithm. You may want to reload the page.",
"js_calculating_difficulty": "Calculating...<br/>Difficulty:",
"js_speed": "Speed:",
"js_verification_longer": "Verification is taking longer than expected. Please do not refresh the page.",
"js_success": "Success!",
"js_done_took": "Done! Took",
"js_iterations": "iterations",
"js_finished_reading": "I've finished reading, continue →",
"js_calculation_error": "Calculation error!",
"js_calculation_error_msg": "Failed to calculate challenge:"
}

View File

@@ -0,0 +1,63 @@
{
"loading": "Cargando...",
"why_am_i_seeing": "¿Por qué veo esto?",
"protected_by": "Protegido por",
"made_with": "Hecho con ❤️ en 🇨🇦",
"mascot_design": "Diseño de la mascota por",
"ai_companies_explanation": "Ves esto porque el administrador de este sitio web ha configurado Anubis para proteger el servidor contra la plaga de empresas de IA que rastrean agresivamente los sitios web. Esto puede y causa tiempo de inactividad para los sitios web, haciendo que sus recursos sean inaccesibles para todos.",
"anubis_compromise": "Anubis es un compromiso. Anubis utiliza un esquema de Prueba de Trabajo en la línea de Hashcash, un esquema de prueba de trabajo propuesto para reducir el spam por correo electrónico. La idea es que a escala individual, la carga adicional es insignificante, pero a escala de raspadores masivos, se acumula y hace que el raspado sea mucho más costoso.",
"hack_purpose": "En última instancia, esto es un hack cuyo verdadero propósito es dar una solución alternativa \"suficientemente buena\" para que se pueda dedicar más tiempo a la huella digital e identificación de navegadores sin cabeza (por ejemplo: a través de cómo renderizan las fuentes) para que la página de desafío de prueba de trabajo no necesite ser presentada a usuarios que son mucho más propensos a ser legítimos.",
"jshelter_note": "Ten en cuenta que Anubis requiere el uso de características modernas de JavaScript que plugins como JShelter deshabilitarán. Por favor, deshabilita JShelter u otros plugins similares para este dominio.",
"version_info": "Este sitio web utiliza Anubis versión",
"try_again": "Intentar de nuevo",
"go_home": "Inicio",
"contact_webmaster": "o si crees que no deberías estar bloqueado, por favor contacta al webmaster en",
"connection_security": "Espere un momento mientras garantizamos la seguridad de su conexión.",
"javascript_required": "Desafortunadamente, necesitas habilitar JavaScript para pasar este desafío. Esto es requerido porque las empresas de IA han cambiado el contrato social sobre cómo funciona el alojamiento de sitios web. Una solución sin JS está en desarrollo.",
"benchmark_requires_js": "Ejecutar la herramienta de benchmark requiere que JavaScript esté habilitado.",
"difficulty": "Dificultad:",
"algorithm": "Algoritmo:",
"compare": "Comparar:",
"time": "Tiempo",
"iters": "Iteraciones",
"time_a": "Tiempo A",
"iters_a": "Iter. A",
"time_b": "Tiempo B",
"iters_b": "Iter. B",
"static_check_endpoint": "Este es solo un endpoint de verificación para que tu proxy inverso lo use.",
"authorization_required": "Autorización requerida",
"cookies_disabled": "Tu navegador está configurado para deshabilitar las cookies. Anubis requiere cookies para el interés legítimo de asegurar que eres un cliente válido. Por favor habilita las cookies para este dominio",
"access_denied": "Acceso denegado: código de error",
"dronebl_entry": "DroneBL reportó una entrada",
"see_dronebl_lookup": "ver",
"internal_server_error": "Error interno del servidor: el administrador ha configurado mal Anubis. Por favor contacta al administrador y pídele que revise los logs alrededor de",
"invalid_redirect": "Redirección inválida",
"redirect_not_parseable": "URL de redirección no analizable",
"redirect_domain_not_allowed": "Dominio de redirección no permitido",
"failed_to_sign_jwt": "falló al firmar JWT",
"invalid_invocation": "Invocación inválida de MakeChallenge",
"client_error_browser": "Error del cliente: Por favor asegúrate de que tu navegador esté actualizado e inténtalo de nuevo más tarde.",
"oh_noes": "¡Oh no!",
"benchmarking_anubis": "¡Benchmarking de Anubis!",
"you_are_not_a_bot": "¡No eres un robot!",
"making_sure_not_bot": "¡Asegurándonos de que no eres un robot!",
"celphase": "CELPHASE",
"js_web_crypto_error": "Tu navegador no tiene un elemento web.crypto funcional. ¿Estás viendo esta página en un contexto seguro?",
"js_web_workers_error": "Tu navegador no soporta web workers (Anubis los usa para evitar bloquear tu navegador). ¿Tienes un plugin como JShelter instalado?",
"js_cookies_error": "Tu navegador no almacena cookies. Anubis usa cookies para determinar qué clientes han pasado los desafíos almacenando un token firmado en una cookie. Por favor habilita el almacenamiento de cookies para este dominio. Los nombres de las cookies que Anubis almacena pueden variar sin previo aviso. Los nombres y valores de las cookies no son parte de la API pública.",
"js_context_not_secure": "¡Tu contexto no es seguro!",
"js_context_not_secure_msg": "Intenta conectarte a través de HTTPS o informa al administrador para configurar HTTPS. Para más información, consulta <a href=\"https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts#when_is_a_context_considered_secure\">MDN</a>.",
"js_calculating": "Calculando...",
"js_missing_feature": "Característica faltante",
"js_challenge_error": "¡Error de desafío!",
"js_challenge_error_msg": "Falló al resolver el algoritmo de verificación. Puedes intentar recargar la página.",
"js_calculating_difficulty": "Calculando...<br/>Dificultad:",
"js_speed": "Velocidad:",
"js_verification_longer": "La verificación está tomando más tiempo del esperado. Por favor no actualices la página.",
"js_success": "¡Éxito!",
"js_done_took": "¡Terminado! Tomó",
"js_iterations": "iteraciones",
"js_finished_reading": "He terminado de leer, continuar →",
"js_calculation_error": "¡Error de cálculo!",
"js_calculation_error_msg": "Falló al calcular el desafío:"
}

View File

@@ -0,0 +1,63 @@
{
"loading": "Chargement...",
"why_am_i_seeing": "Pourquoi je vois ceci ?",
"protected_by": "Protégé par",
"made_with": "Fait avec ❤️ au 🇨🇦",
"mascot_design": "Design de la mascotte par",
"ai_companies_explanation": "Vous voyez ceci car l'administrateur de ce site web a configuré Anubis pour protéger le serveur contre le fléau des entreprises d'IA qui scrapent agressivement les sites web. Cela peut et cause des temps d'arrêt pour les sites web, ce qui rend leurs ressources inaccessibles pour tout le monde.",
"anubis_compromise": "Anubis est un compromis. Anubis utilise un schéma de Preuve de Travail dans la veine de Hashcash, un schéma de preuve de travail proposé pour réduire le spam par email. L'idée est qu'à l'échelle individuelle, la charge supplémentaire est négligeable, mais à l'échelle des scrapers de masse, cela s'accumule et rend le scraping beaucoup plus coûteux.",
"hack_purpose": "En fin de compte, c'est un hack dont le véritable objectif est de donner une solution de substitution \"assez bonne\" pour que plus de temps puisse être consacré à l'empreinte digitale et à l'identification des navigateurs sans tête (par exemple : via la façon dont ils font le rendu des polices) afin que la page de défi de preuve de travail n'ait pas besoin d'être présentée aux utilisateurs qui sont beaucoup plus susceptibles d'être légitimes.",
"jshelter_note": "Veuillez noter qu'Anubis nécessite l'utilisation de fonctionnalités JavaScript modernes que des plugins comme JShelter désactiveront. Veuillez désactiver JShelter ou d'autres plugins similaires pour ce domaine.",
"version_info": "Ce site web utilise Anubis version",
"try_again": "Réessayer",
"go_home": "Accueil",
"contact_webmaster": "ou si vous pensez que vous ne devriez pas être bloqué, veuillez contacter le webmaster à",
"connection_security": "Veuillez patienter un instant pendant que nous assurons la sécurité de votre connexion.",
"javascript_required": "Malheureusement, vous devez activer JavaScript pour passer ce défi. Ceci est requis car les entreprises d'IA ont changé le contrat social autour du fonctionnement de l'hébergement de sites web. Une solution sans JS est en cours de développement.",
"benchmark_requires_js": "L'exécution de l'outil de benchmark nécessite l'activation de JavaScript.",
"difficulty": "Difficulté :",
"algorithm": "Algorithme :",
"compare": "Comparer :",
"time": "Temps",
"iters": "Itérations",
"time_a": "Temps A",
"iters_a": "Itér. A",
"time_b": "Temps B",
"iters_b": "Itér. B",
"static_check_endpoint": "Ceci est juste un point de terminaison de vérification pour votre proxy inverse à utiliser.",
"authorization_required": "Autorisation requise",
"cookies_disabled": "Votre navigateur est configuré pour désactiver les cookies. Anubis nécessite des cookies pour l'intérêt légitime de s'assurer que vous êtes un client valide. Veuillez activer les cookies pour ce domaine",
"access_denied": "Accès refusé : code d'erreur",
"dronebl_entry": "DroneBL a signalé une entrée",
"see_dronebl_lookup": "voir",
"internal_server_error": "Erreur interne du serveur : l'administrateur a mal configuré Anubis. Veuillez contacter l'administrateur et lui demander de consulter les logs autour de",
"invalid_redirect": "Redirection invalide",
"redirect_not_parseable": "URL de redirection non analysable",
"redirect_domain_not_allowed": "Domaine de redirection non autorisé",
"failed_to_sign_jwt": "échec de la signature JWT",
"invalid_invocation": "Invocation invalide de MakeChallenge",
"client_error_browser": "Erreur client : Veuillez vous assurer que votre navigateur est à jour et réessayez plus tard.",
"oh_noes": "Oh non !",
"benchmarking_anubis": "Test de performance d'Anubis !",
"you_are_not_a_bot": "Vous n'êtes pas un robot !",
"making_sure_not_bot": "Vérification que vous n'êtes pas un robot !",
"celphase": "PHASE de CEL",
"js_web_crypto_error": "Votre navigateur n'a pas d'élément web.crypto fonctionnel. Consultez-vous cette page dans un contexte sécurisé ?",
"js_web_workers_error": "Votre navigateur ne prend pas en charge les web workers (Anubis les utilise pour éviter de bloquer votre navigateur). Avez-vous un plugin comme JShelter installé ?",
"js_cookies_error": "Votre navigateur ne stocke pas les cookies. Anubis utilise des cookies pour déterminer quels clients ont réussi les défis en stockant un jeton signé dans un cookie. Veuillez activer le stockage des cookies pour ce domaine. Les noms des cookies qu'Anubis stocke peuvent varier sans préavis. Les noms et valeurs des cookies ne font pas partie de l'API publique.",
"js_context_not_secure": "Votre contexte n'est pas sécurisé !",
"js_context_not_secure_msg": "Essayez de vous connecter via HTTPS ou informez l'administrateur de configurer HTTPS. Pour plus d'informations, voir <a href=\"https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts#when_is_a_context_considered_secure\">MDN</a>.",
"js_calculating": "Calcul en cours...",
"js_missing_feature": "Fonctionnalité manquante",
"js_challenge_error": "Erreur de défi !",
"js_challenge_error_msg": "Échec de la résolution de l'algorithme de vérification. Vous pouvez essayer de recharger la page.",
"js_calculating_difficulty": "Calcul en cours...<br/>Difficulté :",
"js_speed": "Vitesse :",
"js_verification_longer": "La vérification prend plus de temps que prévu. Veuillez ne pas actualiser la page.",
"js_success": "Succès !",
"js_done_took": "Terminé ! A pris",
"js_iterations": "itérations",
"js_finished_reading": "J'ai fini de lire, continuer →",
"js_calculation_error": "Erreur de calcul !",
"js_calculation_error_msg": "Échec du calcul du défi :"
}

View File

@@ -0,0 +1,3 @@
{
"supportedLanguages": ["en", "fr", "es", "pt-BR"]
}

View File

@@ -0,0 +1,63 @@
{
"loading": "Carregando...",
"why_am_i_seeing": "Por que estou vendo isso?",
"protected_by": "Protegido por",
"made_with": "Feito com ❤️ no 🇨🇦",
"mascot_design": "Design do mascote por",
"ai_companies_explanation": "Você está vendo isso porque o administrador deste site configurou Anubis para proteger o servidor contra a praga de empresas de IA que realizam scraping agressivo em sites. Isso pode causar, e de fato causa, inoperância nos sites, o que torna seus recursos inacessíveis para todos.",
"anubis_compromise": "O Anubis é um meio-termo. Ele utiliza um esquema de Prova de Trabalho (Proof-of-Work) semelhante ao Hashcash, um esquema de Prova de Trabalho proposto para reduzir spam de e-mail. A ideia é que, em escalas individuais, a carga adicional seja insignificante, mas em níveis em massa de scrapers, ela se acumula e torna o scraping muito mais custoso.",
"hack_purpose": "Em última análise, este é um hack cujo propósito real é fornecer uma solução \"boa o suficiente\" para que mais tempo possa ser gasto na identificação de navegadores sem interface (por exemplo: por meio de como eles fazem a renderização de fontes), para que a página do desafio da prova de trabalho não precise ser apresentada a usuários que têm muito mais probabilidade de serem legítimos.",
"jshelter_note": "Observe que o Anubis requer o uso de recursos JavaScript modernos que plugins como o JShelter desabilitarão. Desabilite o JShelter ou outros plugins semelhantes para este domínio.",
"version_info": "Este site está usando o Anubis versão",
"try_again": "Tente novamente",
"go_home": "Início",
"contact_webmaster": "ou se você acredita que não deveria estar bloqueado, contate o webmaster em",
"connection_security": "Por favor, aguarde um momento enquanto nós garantimos a segurança de sua conexão.",
"javascript_required": "Infelizmente, você deve habilitar JavaScript para passar por este desafio. Isso é necessário porque empresas de IA alteraram o contrato social sobre como a hospedagem de sites funciona. Uma solução que não use JavaScript ainda está sendo desenvolvida.",
"benchmark_requires_js": "Para executar a ferramenta de benchmark, é necessário que o JavaScript esteja habilitado.",
"difficulty": "Dificuldade:",
"algorithm": "Algoritmo:",
"compare": "Comparar:",
"time": "Tempo",
"iters": "Iteração",
"time_a": "Tempo A",
"iters_a": "Iteração A",
"time_b": "Tempo B",
"iters_b": "Iteração B",
"static_check_endpoint": "Este é apenas um ponto de verificação para seu proxy reverso usar.",
"authorization_required": "Autorização necessária",
"cookies_disabled": "Seu navegador está configurado para desabilitar cookies. O Anubis requer cookies para o interesse legítimo de garantir que você seja um cliente válido. Habilite os cookies para este domínio.",
"access_denied": "Acesso negado: código de erro",
"dronebl_entry": "DroneBL relatou uma entrada",
"see_dronebl_lookup": "consulte",
"internal_server_error": "Erro interno do servidor: o administrador configurou incorretamente o Anubis. Entre em contato com o administrador e peça para analisar os logs relacionados.",
"invalid_redirect": "Redirecionamento inválido",
"redirect_not_parseable": "URL de redirecionamento não analisável",
"redirect_domain_not_allowed": "Domínio de redirecionamento não permitido",
"failed_to_sign_jwt": "falha ao assinar JWT",
"invalid_invocation": "Invocação inválida de MakeChallenge",
"client_error_browser": "Erro do cliente: verifique se seu navegador está atualizado e tente novamente mais tarde..",
"oh_noes": "Ah, não!",
"benchmarking_anubis": "Fazendo benchmark do Anubis!",
"you_are_not_a_bot": "Você não é um bot!",
"making_sure_not_bot": "Certificando de que você não é um bot!",
"celphase": "CELPHASE",
"js_web_crypto_error": "Seu navegador não possui um elemento web.crypto funcional. Você está visualizando isso em um contexto seguro?",
"js_web_workers_error": "Seu navegador não oferece suporte a web workers (o Anubis usa isso para evitar que seu navegador trave). Você tem um plugin como o JShelter instalado?",
"js_cookies_error": "Seu navegador não armazena cookies. O Anubis usa cookies para determinar quais clientes passaram nos desafios, armazenando um token assinado em um cookie. Habilite o armazenamento de cookies para este domínio. Os nomes dos cookies armazenados pelo Anubis podem variar sem aviso prévio. Os nomes e valores dos cookies não fazem parte da API pública.",
"js_context_not_secure": "Seu contexto não é seguro!",
"js_context_not_secure_msg": "Tente conectar-se via HTTPS ou avise o administrador para configurar o HTTPS. Para mais informações, consulte o <a href=\"https://developer.mozilla.org/en-US/docs/Web/Security/Secure_Contexts#when_is_a_context_considered_secure\">MDN</a>.",
"js_calculating": "Calculando...",
"js_missing_feature": "Faltando recurso",
"js_challenge_error": "Erro no desafio!",
"js_challenge_error_msg": "Falha ao resolver o algoritmo de verificação. Talvez seja necessário recarregar a página.",
"js_calculating_difficulty": "Calculando...<br/>Dificuldade:",
"js_speed": "Velocidade:",
"js_verification_longer": "A verificação está demorando mais do que o esperado. Não atualize a página.",
"js_success": "Sucesso!",
"js_done_took": "Feito! Levou",
"js_iterations": "iterações",
"js_finished_reading": "Terminei de ler, continue →",
"js_calculation_error": "Erro de cálculo!",
"js_calculation_error_msg": "Falha ao calcular o desafio:"
}

View File

@@ -0,0 +1,106 @@
package localization
import (
"embed"
"encoding/json"
"github.com/TecharoHQ/anubis"
"net/http"
"strings"
"sync"
"github.com/nicksnyder/go-i18n/v2/i18n"
"golang.org/x/text/language"
)
//go:embed locales/*.json
var localeFS embed.FS
type LocalizationService struct {
bundle *i18n.Bundle
}
var (
globalService *LocalizationService
once sync.Once
)
func NewLocalizationService() *LocalizationService {
once.Do(func() {
bundle := i18n.NewBundle(language.English)
bundle.RegisterUnmarshalFunc("json", json.Unmarshal)
// Read all JSON files from the locales directory
entries, err := localeFS.ReadDir("locales")
if err != nil {
// Try fallback - create a minimal service with default messages
globalService = &LocalizationService{bundle: bundle}
return
}
loadedAny := false
for _, entry := range entries {
if !entry.IsDir() && strings.HasSuffix(entry.Name(), ".json") {
filePath := "locales/" + entry.Name()
_, err := bundle.LoadMessageFileFS(localeFS, filePath)
if err != nil {
// Log error but continue with other files
continue
}
loadedAny = true
}
}
if !loadedAny {
// If no files were loaded successfully, create minimal service
globalService = &LocalizationService{bundle: bundle}
return
}
globalService = &LocalizationService{bundle: bundle}
})
// Safety check - if globalService is still nil, create a minimal one
if globalService == nil {
bundle := i18n.NewBundle(language.English)
bundle.RegisterUnmarshalFunc("json", json.Unmarshal)
globalService = &LocalizationService{bundle: bundle}
}
return globalService
}
func (ls *LocalizationService) GetLocalizer(lang string) *i18n.Localizer {
return i18n.NewLocalizer(ls.bundle, lang)
}
func (ls *LocalizationService) GetLocalizerFromRequest(r *http.Request) *i18n.Localizer {
if ls == nil || ls.bundle == nil {
// Fallback to a basic bundle if service is not properly initialized
bundle := i18n.NewBundle(language.English)
bundle.RegisterUnmarshalFunc("json", json.Unmarshal)
return i18n.NewLocalizer(bundle, "en")
}
acceptLanguage := r.Header.Get("Accept-Language")
return i18n.NewLocalizer(ls.bundle, acceptLanguage, "en")
}
// SimpleLocalizer wraps i18n.Localizer with a more convenient API
type SimpleLocalizer struct {
Localizer *i18n.Localizer
}
// T provides a concise way to localize messages
func (sl *SimpleLocalizer) T(messageID string) string {
return sl.Localizer.MustLocalize(&i18n.LocalizeConfig{MessageID: messageID})
}
// GetLocalizer creates a localizer based on the request's Accept-Language header or forcedLanguage option
func GetLocalizer(r *http.Request) *SimpleLocalizer {
var localizer *i18n.Localizer
if anubis.ForcedLanguage == "" {
localizer = NewLocalizationService().GetLocalizerFromRequest(r)
} else {
localizer = NewLocalizationService().GetLocalizer(anubis.ForcedLanguage)
}
return &SimpleLocalizer{Localizer: localizer}
}

View File

@@ -0,0 +1,116 @@
package localization
import (
"encoding/json"
"sort"
"testing"
"github.com/nicksnyder/go-i18n/v2/i18n"
)
func TestLocalizationService(t *testing.T) {
service := NewLocalizationService()
t.Run("English localization", func(t *testing.T) {
localizer := service.GetLocalizer("en")
result := localizer.MustLocalize(&i18n.LocalizeConfig{MessageID: "loading"})
if result != "Loading..." {
t.Errorf("Expected 'Loading...', got '%s'", result)
}
})
t.Run("French localization", func(t *testing.T) {
localizer := service.GetLocalizer("fr")
result := localizer.MustLocalize(&i18n.LocalizeConfig{MessageID: "loading"})
if result != "Chargement..." {
t.Errorf("Expected 'Chargement...', got '%s'", result)
}
})
t.Run("All required keys exist in English", func(t *testing.T) {
localizer := service.GetLocalizer("en")
requiredKeys := []string{
"loading", "why_am_i_seeing", "protected_by", "made_with",
"mascot_design", "try_again", "go_home", "javascript_required",
}
for _, key := range requiredKeys {
result := localizer.MustLocalize(&i18n.LocalizeConfig{MessageID: key})
if result == "" {
t.Errorf("Key '%s' returned empty string", key)
}
}
})
t.Run("All required keys exist in French", func(t *testing.T) {
localizer := service.GetLocalizer("fr")
requiredKeys := []string{
"loading", "why_am_i_seeing", "protected_by", "made_with",
"mascot_design", "try_again", "go_home", "javascript_required",
}
for _, key := range requiredKeys {
result := localizer.MustLocalize(&i18n.LocalizeConfig{MessageID: key})
if result == "" {
t.Errorf("Key '%s' returned empty string", key)
}
}
})
}
type manifest struct {
SupportedLanguages []string `json:"supported_languages"`
}
func loadManifest(t *testing.T) manifest {
t.Helper()
fin, err := localeFS.Open("locales/manifest.json")
if err != nil {
t.Fatal(err)
}
defer fin.Close()
var result manifest
if err := json.NewDecoder(fin).Decode(&result); err != nil {
t.Fatal(err)
}
return result
}
func TestComprehensiveTranslations(t *testing.T) {
service := NewLocalizationService()
var translations = map[string]any{}
fin, err := localeFS.Open("locales/en.json")
if err != nil {
t.Fatal(err)
}
defer fin.Close()
if err := json.NewDecoder(fin).Decode(&translations); err != nil {
t.Fatal(err)
}
var keys []string
for k := range translations {
keys = append(keys, k)
}
sort.Strings(keys)
for _, lang := range loadManifest(t).SupportedLanguages {
t.Run(lang, func(t *testing.T) {
loc := service.GetLocalizer(lang)
sl := SimpleLocalizer{Localizer: loc}
for _, key := range keys {
t.Run(key, func(t *testing.T) {
if result := sl.T(key); result == "" {
t.Error("key not defined")
}
})
}
})
}
}

View File

@@ -17,47 +17,18 @@ type CELChecker struct {
}
func NewCELChecker(cfg *config.ExpressionOrList) (*CELChecker, error) {
env, err := expressions.NewEnvironment()
env, err := expressions.BotEnvironment()
if err != nil {
return nil, err
}
var src string
var ast *cel.Ast
if cfg.Expression != "" {
src = cfg.Expression
var iss *cel.Issues
intermediate, iss := env.Compile(src)
if iss != nil {
return nil, iss.Err()
}
ast, iss = env.Check(intermediate)
if iss != nil {
return nil, iss.Err()
}
}
if len(cfg.All) != 0 {
ast, err = expressions.Join(env, expressions.JoinAnd, cfg.All...)
}
if len(cfg.Any) != 0 {
ast, err = expressions.Join(env, expressions.JoinOr, cfg.Any...)
}
if err != nil {
return nil, err
}
program, err := expressions.Compile(env, ast)
program, err := expressions.Compile(env, cfg.String())
if err != nil {
return nil, fmt.Errorf("can't compile CEL program: %w", err)
}
return &CELChecker{
src: src,
src: cfg.String(),
program: program,
}, nil
}

View File

@@ -17,20 +17,6 @@ var (
ErrMisconfiguration = errors.New("[unexpected] policy: administrator misconfiguration")
)
type staticHashChecker struct {
hash string
}
func (staticHashChecker) Check(r *http.Request) (bool, error) {
return true, nil
}
func (s staticHashChecker) Hash() string { return s.hash }
func NewStaticHashChecker(hashable string) checker.Impl {
return staticHashChecker{hash: internal.FastHash(hashable)}
}
type RemoteAddrChecker struct {
prefixTable *bart.Lite
hash string

View File

@@ -0,0 +1,55 @@
package config
import (
"errors"
"fmt"
"testing"
)
func TestASNsValid(t *testing.T) {
for _, tt := range []struct {
name string
input *ASNs
err error
}{
{
name: "basic valid",
input: &ASNs{
Match: []uint32{13335}, // Cloudflare
},
},
{
name: "private ASN",
input: &ASNs{
Match: []uint32{64513, 4206942069}, // 16 and 32 bit private ASN
},
err: ErrPrivateASN,
},
} {
t.Run(tt.name, func(t *testing.T) {
if err := tt.input.Valid(); !errors.Is(err, tt.err) {
t.Logf("want: %v", tt.err)
t.Logf("got: %v", err)
t.Error("got wrong validation error")
}
})
}
}
func TestIsPrivateASN(t *testing.T) {
for _, tt := range []struct {
input uint32
output bool
}{
{13335, false}, // Cloudflare
{64513, true}, // 16 bit private ASN
{4206942069, true}, // 32 bit private ASN
} {
t.Run(fmt.Sprint(tt.input, "->", tt.output), func(t *testing.T) {
result := isPrivateASN(tt.input)
if result != tt.output {
t.Errorf("wanted isPrivateASN(%d) == %v, got: %v", tt.input, tt.output, result)
}
})
}
}

View File

@@ -10,6 +10,7 @@ import (
"os"
"regexp"
"strings"
"time"
"github.com/TecharoHQ/anubis/data"
"k8s.io/apimachinery/pkg/util/yaml"
@@ -43,6 +44,15 @@ const (
RuleBenchmark Rule = "DEBUG_BENCHMARK"
)
func (r Rule) Valid() error {
switch r {
case RuleAllow, RuleDeny, RuleChallenge, RuleWeigh, RuleBenchmark:
return nil
default:
return ErrUnknownAction
}
}
const DefaultAlgorithm = "fast"
type BotConfig struct {
@@ -184,13 +194,18 @@ type ChallengeRules struct {
}
var (
ErrChallengeDifficultyTooLow = errors.New("config.Bot.ChallengeRules: difficulty is too low (must be >= 1)")
ErrChallengeDifficultyTooHigh = errors.New("config.Bot.ChallengeRules: difficulty is too high (must be <= 64)")
ErrChallengeDifficultyTooLow = errors.New("config.ChallengeRules: difficulty is too low (must be >= 1)")
ErrChallengeDifficultyTooHigh = errors.New("config.ChallengeRules: difficulty is too high (must be <= 64)")
ErrChallengeMustHaveAlgorithm = errors.New("config.ChallengeRules: must have algorithm name set")
)
func (cr ChallengeRules) Valid() error {
var errs []error
if cr.Algorithm == "" {
errs = append(errs, ErrChallengeMustHaveAlgorithm)
}
if cr.Difficulty < 1 {
errs = append(errs, fmt.Errorf("%w, got: %d", ErrChallengeDifficultyTooLow, cr.Difficulty))
}
@@ -309,20 +324,29 @@ func (sc StatusCodes) Valid() error {
}
type fileConfig struct {
Bots []BotOrImport `json:"bots"`
DNSBL bool `json:"dnsbl"`
StatusCodes StatusCodes `json:"status_codes"`
Bots []BotOrImport `json:"bots"`
DNSBL bool `json:"dnsbl"`
OpenGraph openGraphFileConfig `json:"openGraph,omitempty"`
Impressum *Impressum `json:"impressum,omitempty"`
StatusCodes StatusCodes `json:"status_codes"`
Thresholds []Threshold `json:"thresholds"`
}
func (c fileConfig) Valid() error {
func (c *fileConfig) Valid() error {
var errs []error
if len(c.Bots) == 0 {
errs = append(errs, ErrNoBotRulesDefined)
}
for _, b := range c.Bots {
for i, b := range c.Bots {
if err := b.Valid(); err != nil {
errs = append(errs, fmt.Errorf("bot %d: %w", i, err))
}
}
if c.OpenGraph.Enabled {
if err := c.OpenGraph.Valid(); err != nil {
errs = append(errs, err)
}
}
@@ -331,6 +355,12 @@ func (c fileConfig) Valid() error {
errs = append(errs, err)
}
for i, t := range c.Thresholds {
if err := t.Valid(); err != nil {
errs = append(errs, fmt.Errorf("threshold %d: %w", i, err))
}
}
if len(errs) != 0 {
return fmt.Errorf("config is not valid:\n%w", errors.Join(errs...))
}
@@ -339,11 +369,13 @@ func (c fileConfig) Valid() error {
}
func Load(fin io.Reader, fname string) (*Config, error) {
var c fileConfig
c.StatusCodes = StatusCodes{
Challenge: http.StatusOK,
Deny: http.StatusOK,
c := &fileConfig{
StatusCodes: StatusCodes{
Challenge: http.StatusOK,
Deny: http.StatusOK,
},
}
if err := yaml.NewYAMLToJSONDecoder(fin).Decode(&c); err != nil {
return nil, fmt.Errorf("can't parse policy config YAML %s: %w", fname, err)
}
@@ -353,10 +385,21 @@ func Load(fin io.Reader, fname string) (*Config, error) {
}
result := &Config{
DNSBL: c.DNSBL,
DNSBL: c.DNSBL,
OpenGraph: OpenGraph{
Enabled: c.OpenGraph.Enabled,
ConsiderHost: c.OpenGraph.ConsiderHost,
Override: c.OpenGraph.Override,
},
StatusCodes: c.StatusCodes,
}
if c.OpenGraph.TimeToLive != "" {
// XXX(Xe): already validated in Valid()
ogTTL, _ := time.ParseDuration(c.OpenGraph.TimeToLive)
result.OpenGraph.TimeToLive = ogTTL
}
var validationErrs []error
for _, boi := range c.Bots {
@@ -379,6 +422,27 @@ func Load(fin io.Reader, fname string) (*Config, error) {
}
}
if c.Impressum != nil {
if err := c.Impressum.Valid(); err != nil {
validationErrs = append(validationErrs, err)
}
result.Impressum = c.Impressum
}
if len(c.Thresholds) == 0 {
c.Thresholds = DefaultThresholds
}
for _, t := range c.Thresholds {
if err := t.Valid(); err != nil {
validationErrs = append(validationErrs, err)
continue
}
result.Thresholds = append(result.Thresholds, t)
}
if len(validationErrs) > 0 {
return nil, fmt.Errorf("errors validating policy config %s: %w", fname, errors.Join(validationErrs...))
}
@@ -388,7 +452,10 @@ func Load(fin io.Reader, fname string) (*Config, error) {
type Config struct {
Bots []BotConfig
Thresholds []Threshold
DNSBL bool
Impressum *Impressum
OpenGraph OpenGraph
StatusCodes StatusCodes
}

View File

@@ -8,7 +8,6 @@ import (
"testing"
"github.com/TecharoHQ/anubis/data"
"k8s.io/apimachinery/pkg/util/yaml"
)
func p[V any](v V) *V { return &v }
@@ -313,12 +312,8 @@ func TestConfigValidBad(t *testing.T) {
}
defer fin.Close()
var c fileConfig
if err := yaml.NewYAMLToJSONDecoder(fin).Decode(&c); err != nil {
t.Fatalf("can't decode file: %v", err)
}
if err := c.Valid(); err == nil {
_, err = Load(fin, filepath.Join("testdata", "bad", st.Name()))
if err == nil {
t.Fatal("validation should have failed but didn't somehow")
} else {
t.Log(err)

View File

@@ -3,7 +3,9 @@ package config
import (
"encoding/json"
"errors"
"fmt"
"slices"
"strings"
)
var (
@@ -18,6 +20,32 @@ type ExpressionOrList struct {
Any []string `json:"any,omitempty" yaml:"any,omitempty"`
}
func (eol ExpressionOrList) String() string {
switch {
case len(eol.Expression) != 0:
return eol.Expression
case len(eol.All) != 0:
var sb strings.Builder
for i, pred := range eol.All {
if i != 0 {
fmt.Fprintf(&sb, " && ")
}
fmt.Fprintf(&sb, "( %s )", pred)
}
return sb.String()
case len(eol.Any) != 0:
var sb strings.Builder
for i, pred := range eol.Any {
if i != 0 {
fmt.Fprintf(&sb, " || ")
}
fmt.Fprintf(&sb, "( %s )", pred)
}
return sb.String()
}
panic("this should not happen")
}
func (eol ExpressionOrList) Equal(rhs *ExpressionOrList) bool {
if eol.Expression != rhs.Expression {
return false

View File

@@ -213,3 +213,54 @@ func TestExpressionOrListUnmarshalJSON(t *testing.T) {
})
}
}
func TestExpressionOrListString(t *testing.T) {
for _, tt := range []struct {
name string
in ExpressionOrList
out string
}{
{
name: "single expression",
in: ExpressionOrList{
Expression: "true",
},
out: "true",
},
{
name: "all",
in: ExpressionOrList{
All: []string{"true"},
},
out: "( true )",
},
{
name: "all with &&",
in: ExpressionOrList{
All: []string{"true", "true"},
},
out: "( true ) && ( true )",
},
{
name: "any",
in: ExpressionOrList{
All: []string{"true"},
},
out: "( true )",
},
{
name: "any with ||",
in: ExpressionOrList{
Any: []string{"true", "true"},
},
out: "( true ) || ( true )",
},
} {
t.Run(tt.name, func(t *testing.T) {
result := tt.in.String()
if result != tt.out {
t.Errorf("wanted %q, got: %q", tt.out, result)
}
})
}
}

View File

@@ -8,7 +8,7 @@ import (
)
var (
countryCodeRegexp = regexp.MustCompile(`^\w{2}$`)
countryCodeRegexp = regexp.MustCompile(`^[a-zA-Z]{2}$`)
ErrNotCountryCode = errors.New("config.Bot: invalid country code")
)

View File

@@ -0,0 +1,36 @@
package config
import (
"errors"
"testing"
)
func TestGeoIPValid(t *testing.T) {
for _, tt := range []struct {
name string
input *GeoIP
err error
}{
{
name: "basic valid",
input: &GeoIP{
Countries: []string{"CA"},
},
},
{
name: "invalid country",
input: &GeoIP{
Countries: []string{"XOB"},
},
err: ErrNotCountryCode,
},
} {
t.Run(tt.name, func(t *testing.T) {
if err := tt.input.Valid(); !errors.Is(err, tt.err) {
t.Logf("want: %v", tt.err)
t.Logf("got: %v", err)
t.Error("got wrong validation error")
}
})
}
}

View File

@@ -0,0 +1,71 @@
package config
import (
"context"
"errors"
"fmt"
"io"
)
var ErrMissingValue = errors.New("config: missing value")
type Impressum struct {
Footer string `json:"footer" yaml:"footer"`
Page ImpressumPage `json:"page" yaml:"page"`
}
func (i Impressum) Render(_ context.Context, w io.Writer) error {
if _, err := fmt.Fprint(w, i.Footer); err != nil {
return err
}
return nil
}
func (i Impressum) Valid() error {
var errs []error
if len(i.Footer) == 0 {
errs = append(errs, fmt.Errorf("%w: impressum footer must be defined", ErrMissingValue))
}
if err := i.Page.Valid(); err != nil {
errs = append(errs, err)
}
if len(errs) != 0 {
return errors.Join(errs...)
}
return nil
}
type ImpressumPage struct {
Title string `json:"title" yaml:"title"`
Body string `json:"body" yaml:"body"`
}
func (ip ImpressumPage) Render(_ context.Context, w io.Writer) error {
if _, err := fmt.Fprint(w, ip.Body); err != nil {
return err
}
return nil
}
func (ip ImpressumPage) Valid() error {
var errs []error
if len(ip.Title) == 0 {
errs = append(errs, fmt.Errorf("%w: impressum page title must be defined", ErrMissingValue))
}
if len(ip.Body) == 0 {
errs = append(errs, fmt.Errorf("%w: impressum body title must be defined", ErrMissingValue))
}
if len(errs) != 0 {
return errors.Join(errs...)
}
return nil
}

View File

@@ -0,0 +1,62 @@
package config
import (
"bytes"
"errors"
"testing"
)
func TestImpressumValid(t *testing.T) {
for _, cs := range []struct {
name string
inp Impressum
err error
}{
{
name: "basic happy path",
inp: Impressum{
Footer: "<p>Website hosted by Techaro.<p>",
Page: ImpressumPage{
Title: "Techaro Imprint",
Body: "<p>This is an imprint page.</p>",
},
},
err: nil,
},
{
name: "no footer",
inp: Impressum{
Footer: "",
Page: ImpressumPage{
Title: "Techaro Imprint",
Body: "<p>This is an imprint page.</p>",
},
},
err: ErrMissingValue,
},
{
name: "page not valid",
inp: Impressum{
Footer: "test page please ignore",
},
err: ErrMissingValue,
},
} {
t.Run(cs.name, func(t *testing.T) {
if err := cs.inp.Valid(); !errors.Is(err, cs.err) {
t.Logf("want: %v", cs.err)
t.Logf("got: %v", err)
t.Error("validation failed")
}
var buf bytes.Buffer
if err := cs.inp.Render(t.Context(), &buf); err != nil {
t.Errorf("can't render footer: %v", err)
}
if err := cs.inp.Page.Render(t.Context(), &buf); err != nil {
t.Errorf("can't render page: %v", err)
}
})
}
}

View File

@@ -0,0 +1,51 @@
package config
import (
"errors"
"fmt"
"time"
)
var (
ErrInvalidOpenGraphConfig = errors.New("config.OpenGraph: invalid OpenGraph configuration")
ErrOpenGraphTTLDoesNotParse = errors.New("config.OpenGraph: ttl does not parse as a Duration, see https://pkg.go.dev/time#ParseDuration (formatted like 5m -> 5 minutes, 2h -> 2 hours, etc)")
ErrOpenGraphMissingProperty = errors.New("config.OpenGraph: default opengraph tags missing a property")
)
type openGraphFileConfig struct {
Enabled bool `json:"enabled" yaml:"enabled"`
ConsiderHost bool `json:"considerHost" yaml:"enabled"`
TimeToLive string `json:"ttl" yaml:"ttl"`
Override map[string]string `json:"override,omitempty" yaml:"override,omitempty"`
}
type OpenGraph struct {
Enabled bool `json:"enabled" yaml:"enabled"`
ConsiderHost bool `json:"considerHost" yaml:"enabled"`
Override map[string]string `json:"override,omitempty" yaml:"override,omitempty"`
TimeToLive time.Duration `json:"ttl" yaml:"ttl"`
}
func (og *openGraphFileConfig) Valid() error {
var errs []error
if _, err := time.ParseDuration(og.TimeToLive); err != nil {
errs = append(errs, fmt.Errorf("%w: ParseDuration(%q) returned: %w", ErrOpenGraphTTLDoesNotParse, og.TimeToLive, err))
}
if len(og.Override) != 0 {
for _, tag := range []string{
"og:title",
} {
if _, ok := og.Override[tag]; !ok {
errs = append(errs, fmt.Errorf("%w: %s", ErrOpenGraphMissingProperty, tag))
}
}
}
if len(errs) != 0 {
return errors.Join(ErrInvalidOpenGraphConfig, errors.Join(errs...))
}
return nil
}

View File

@@ -0,0 +1,67 @@
package config
import (
"errors"
"testing"
)
func TestOpenGraphFileConfigValid(t *testing.T) {
for _, tt := range []struct {
name string
input *openGraphFileConfig
err error
}{
{
name: "basic happy path",
input: &openGraphFileConfig{
Enabled: true,
ConsiderHost: false,
TimeToLive: "1h",
Override: map[string]string{},
},
err: nil,
},
{
name: "basic happy path with default",
input: &openGraphFileConfig{
Enabled: true,
ConsiderHost: false,
TimeToLive: "1h",
Override: map[string]string{
"og:title": "foobar",
},
},
err: nil,
},
{
name: "invalid time duration",
input: &openGraphFileConfig{
Enabled: true,
ConsiderHost: false,
TimeToLive: "taco",
Override: map[string]string{},
},
err: ErrOpenGraphTTLDoesNotParse,
},
{
name: "missing og:title in defaults",
input: &openGraphFileConfig{
Enabled: true,
ConsiderHost: false,
TimeToLive: "1h",
Override: map[string]string{
"description": "foobar",
},
},
err: ErrOpenGraphMissingProperty,
},
} {
t.Run(tt.name, func(t *testing.T) {
if err := tt.input.Valid(); !errors.Is(err, tt.err) {
t.Logf("wanted error: %v", tt.err)
t.Logf("got error: %v", err)
t.Error("validation failed")
}
})
}
}

View File

@@ -0,0 +1,11 @@
bots:
- name: simple-weight-adjust
action: WEIGH
user_agent_regex: Mozilla
weight:
adjust: 5
impressum:
page:
title: Test
body: <p>This is a test</p>

View File

@@ -0,0 +1,10 @@
bots:
- name: simple-weight-adjust
action: WEIGH
user_agent_regex: Mozilla
weight:
adjust: 5
impressum:
footer: "Hi there these are WORDS on the INTERNET."
page: {}

View File

@@ -0,0 +1,12 @@
bots:
- name: everything
user_agent_regex: .*
action: DENY
openGraph:
enabled: true
considerHost: false
ttl: taco
default:
"og:title": "Xe's magic land of fun"
"og:description": "We're no strangers to love, you know the rules and so do I"

View File

@@ -0,0 +1,11 @@
bots:
- name: simple-weight-adjust
action: WEIGH
user_agent_regex: Mozilla
weight:
adjust: 5
thresholds:
- name: extreme-suspicion
expression: "true"
action: WEIGH

View File

@@ -0,0 +1,15 @@
bots:
- name: simple-weight-adjust
action: WEIGH
user_agent_regex: Mozilla
weight:
adjust: 5
thresholds:
- name: extreme-suspicion
expression: "true"
action: WEIGH
challenge:
algorithm: fast
difficulty: 4
report_as: 4

View File

@@ -0,0 +1 @@
}

View File

@@ -0,0 +1 @@
}

View File

@@ -0,0 +1,10 @@
bots:
- name: simple
action: CHALLENGE
user_agent_regex: Mozilla
impressum:
footer: "Hi these are WORDS on the INTERNET."
page:
title: Test
body: <p>This is a test</p>

View File

@@ -0,0 +1,8 @@
bots:
- name: simple-weight-adjust
action: WEIGH
user_agent_regex: Mozilla
weight:
adjust: 5
thresholds: []

View File

@@ -0,0 +1,12 @@
bots:
- name: everything
user_agent_regex: .*
action: DENY
openGraph:
enabled: true
considerHost: false
ttl: 1h
default:
"og:title": "Xe's magic land of fun"
"og:description": "We're no strangers to love, you know the rules and so do I"

View File

@@ -0,0 +1,38 @@
bots:
- name: simple-weight-adjust
action: WEIGH
user_agent_regex: Mozilla
weight:
adjust: 5
thresholds:
- name: minimal-suspicion
expression: weight < 0
action: ALLOW
- name: mild-suspicion
expression:
all:
- weight >= 0
- weight < 10
action: CHALLENGE
challenge:
algorithm: metarefresh
difficulty: 1
report_as: 1
- name: moderate-suspicion
expression:
all:
- weight >= 10
- weight < 20
action: CHALLENGE
challenge:
algorithm: fast
difficulty: 2
report_as: 2
- name: extreme-suspicion
expression: weight >= 20
action: CHALLENGE
challenge:
algorithm: fast
difficulty: 4
report_as: 4

View File

@@ -0,0 +1,80 @@
package config
import (
"errors"
"fmt"
"github.com/TecharoHQ/anubis"
)
var (
ErrNoThresholdRulesDefined = errors.New("config: no thresholds defined")
ErrThresholdMustHaveName = errors.New("config.Threshold: must set name")
ErrThresholdMustHaveExpression = errors.New("config.Threshold: must set expression")
ErrThresholdChallengeMustHaveChallenge = errors.New("config.Threshold: a threshold with the CHALLENGE action must have challenge set")
ErrThresholdCannotHaveWeighAction = errors.New("config.Threshold: a threshold cannot have the WEIGH action")
DefaultThresholds = []Threshold{
{
Name: "legacy-anubis-behaviour",
Expression: &ExpressionOrList{
Expression: "weight > 0",
},
Action: RuleChallenge,
Challenge: &ChallengeRules{
Algorithm: "fast",
Difficulty: anubis.DefaultDifficulty,
ReportAs: anubis.DefaultDifficulty,
},
},
}
)
type Threshold struct {
Name string `json:"name" yaml:"name"`
Expression *ExpressionOrList `json:"expression" yaml:"expression"`
Action Rule `json:"action" yaml:"action"`
Challenge *ChallengeRules `json:"challenge" yaml:"challenge"`
}
func (t Threshold) Valid() error {
var errs []error
if len(t.Name) == 0 {
errs = append(errs, ErrThresholdMustHaveName)
}
if t.Expression == nil {
errs = append(errs, ErrThresholdMustHaveExpression)
}
if t.Expression != nil {
if err := t.Expression.Valid(); err != nil {
errs = append(errs, err)
}
}
if err := t.Action.Valid(); err != nil {
errs = append(errs, err)
}
if t.Action == RuleWeigh {
errs = append(errs, ErrThresholdCannotHaveWeighAction)
}
if t.Action == RuleChallenge && t.Challenge == nil {
errs = append(errs, ErrThresholdChallengeMustHaveChallenge)
}
if t.Challenge != nil {
if err := t.Challenge.Valid(); err != nil {
errs = append(errs, err)
}
}
if len(errs) != 0 {
return fmt.Errorf("config: threshold entry for %q is not valid:\n%w", t.Name, errors.Join(errs...))
}
return nil
}

Some files were not shown because too many files have changed in this diff Show More