feat(lib): add log filtering rules

Closes #942

Signed-off-by: Xe Iaso <me@xeiaso.net>
This commit is contained in:
Xe Iaso
2025-08-18 10:51:14 +00:00
parent 53516738c1
commit a7a5e0d5c7
13 changed files with 128 additions and 142 deletions

View File

@@ -14,6 +14,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
<!-- This changes the project to: -->
- Fix lock convoy problem in decaymap ([#1103](https://github.com/TecharoHQ/anubis/issues/1103))
- [Log filtering](./admin/configuration/expressions.mdx#log-filtering) rules have been added. This allows users to write custom log filtering logic.
- Document missing environment variables in installation guide: `SLOG_LEVEL`, `COOKIE_PREFIX`, `FORCED_LANGUAGE`, and `TARGET_DISABLE_KEEPALIVE` ([#1086](https://github.com/TecharoHQ/anubis/pull/1086))
- Add validation warning when persistent storage is used without setting signing keys
- Fixed `robots2policy` to properly group consecutive user agents into `any:` instead of only processing the last one ([#925](https://github.com/TecharoHQ/anubis/pull/925))

View File

@@ -99,6 +99,10 @@ For this rule, if a request comes in matching [the signature of the `go get` com
Anubis exposes the following variables to expressions:
### Bot expressions
Bot expressions are used for evaluating [bot rules](../policies.mdx#bot-policies).
| Name | Type | Explanation | Example |
| :-------------- | :-------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------- |
| `headers` | `map[string, string]` | The [headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers) of the request being processed. | `{"User-Agent": "Mozilla/5.0 Gecko/20100101 Firefox/137.0"}` |
@@ -182,6 +186,17 @@ Something to keep in mind about system load average is that it is not aware of t
Also keep in mind that this does not account for other kinds of latency like I/O latency. A system can have its web applications unresponsive due to high latency from a MySQL server but still have that web application server report a load near or at zero.
### Log filtering
Log filters are run on every time Anubis logs data. These are high throughput filters and should be written with care.
| Name | Type | Explanation | Example |
| :------ | :-------------------- | :----------------------------------------------------------------------------------------------------- | --------------------------------------- |
| `time` | Timestamp | The time that the log line was emitted. | `2025-08-18T06:45:38-04:00` |
| `msg` | `string` | The text-based message for the given log line. | `"invalid response"` |
| `level` | `string` | The [log level](https://pkg.go.dev/log/slog#Level) for the log message. | `"INFO"` |
| `attrs` | `map[string, string]` | The key -> value attributes for the given log line. Note that this is an expensive variable to access. | `{"err": "internal: the sun exploded"}` |
## Functions exposed to Anubis expressions
Anubis expressions can be augmented with the following functions:

View File

@@ -123,6 +123,10 @@ remote_addresses:
Anubis has support for showing imprint / impressum information. This is defined in the `impressum` block of your configuration. See [Imprint / Impressum configuration](./configuration/impressum.mdx) for more information.
## Logging
Anubis has support for configuring log filtering using expressions. See the [log filters](./configuration/expressions.mdx#log-filters) of the [expression](./configuration/expressions.mdx) documentation for more information.
## Storage backends
Anubis needs to store temporary data in order to determine if a user is legitimate or not. Administrators should choose a storage backend based on their infrastructure needs. Each backend has its own advantages and disadvantages.