diff --git a/docs/docs/admin/environments/kubernetes.mdx b/docs/docs/admin/environments/kubernetes.mdx index 6b81313e..e400e5bf 100644 --- a/docs/docs/admin/environments/kubernetes.mdx +++ b/docs/docs/admin/environments/kubernetes.mdx @@ -131,11 +131,27 @@ Then point your Ingress to the Anubis port: name: anubis ``` +## Storage + +By default, Anubis stores all of its data in memory. This memory is not shared between pods. If you have multiple instances of Anubis without the data being [stored outside of memory](../policies.mdx#storage-backends) and a [shared cookie key](../installation.mdx#key-generation), you will run into [unexpected behaviour](https://github.com/TecharoHQ/anubis/issues/1602) when user traffic traverses between pods. + +Based on the deployment of your Kubernetes cluster, here are the preferable storage backends to pick from: + +| Backend | Pro | Con | +| :------- | :-------------------------------------------------------------- | :------------------------------------------------------------------------------------------- | +| `bbolt` | Only requires a ReadWriteOnce PVC. | Does not support more than one Anubis pod. | +| `memory` | Requires no configuration. | Process memory is not shared between pods. | +| `s3api` | Great if your cluster includes Rook/Ceph to use RADOS directly. | Potentially higher latency unless you use a store like [Tigris](https://www.tigrisdata.com). | +| `valkey` | Trivial to configure in your cluster. | If your Redis/Valkey server is down, Anubis is going to have issues. | + +Pick your poison accordingly. Many production deployments use the `s3api` and `valkey` backends without issue. Single node deployments can get away with either `memory` or `bbolt` depending on the facts and circumstances of the deployment. + ## Envoy Gateway If you are using envoy-gateway, the `X-Real-Ip` header is not set by default, but Anubis does require it. You can resolve this by adding the header, either on the specific `HTTPRoute` where Anubis is listening, or on the `ClientTrafficPolicy` to apply it to any number of Gateways: HTTPRoute: + ```yaml apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute @@ -160,6 +176,7 @@ spec: ``` Applying to any number of Gateways: + ```yaml apiVersion: gateway.envoyproxy.io/v1alpha1 kind: ClientTrafficPolicy diff --git a/docs/docs/admin/policies.mdx b/docs/docs/admin/policies.mdx index 53119013..9e35bdfd 100644 --- a/docs/docs/admin/policies.mdx +++ b/docs/docs/admin/policies.mdx @@ -219,8 +219,11 @@ Anubis offers the following storage backends: - [`memory`](#memory) -- A simple in-memory hashmap - [`bbolt`](#bbolt) -- An on-disk key/value store backed by [bbolt](https://github.com/etcd-io/bbolt), an embedded key/value database for Go programs +- [`s3api`](#s3api) -- Amazon S3 based storage or another compatible object store - [`valkey`](#valkey) -- A remote in-memory key/value database backed by [Valkey](https://valkey.io/) (or another database compatible with the [RESP](https://redis.io/docs/latest/develop/reference/protocol-spec/) protocol) +:::warning + If no storage backend is set in the policy file, Anubis will use the [`memory`](#memory) backend by default. This is equivalent to the following in the policy file: ```yaml @@ -229,6 +232,10 @@ store: parameters: {} ``` +This means that all session data that is required for the challenge mechanism to work is stored **IN PROCESS MEMORY** that is **NOT** shared between instances of Anubis. If you set up Anubis with multiple instances using the `memory` storage backend, your users will sometimes get "Administrator has misconfigured Anubis" error messages when it cannot look up the aforementioned session data. + +::: + ### `memory` The memory backend is an in-memory cache. This backend works best if you don't use multiple instances of Anubis or don't have mutable storage in the environment you're running Anubis in.