Compare commits

..

5 Commits

Author SHA1 Message Date
Xe Iaso e9969ba22a chore: update spelling
check-spelling run (pull_request) for Xe/ci-multiple-go-versions

Signed-off-by: check-spelling-bot <check-spelling-bot@users.noreply.github.com>
on-behalf-of: @check-spelling <check-spelling-bot@check-spelling.dev>
2026-01-16 11:21:06 -05:00
Xe Iaso 7db2c9ebb5 ci: test against multiple go versions
Several Linux distributions and other open source package managers
build Anubis against Go oldstable. Eventually this will have to expand
to at least what FreeBSD supports. I hope they can bump the versions
of Go as soon as upstream makes new code available.

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-01-16 11:18:31 -05:00
Timon de Groot 57c0b2b22c Add IP mapped Perplexity user agents (#1393)
Perplexity has some proper documentation available for their crawlers,
with published IP addresses: https://docs.perplexity.ai/guides/bots.

Signed-off-by: Timon de Groot <timon.degroot@team.blue>
2026-01-15 19:57:31 -05:00
Thomas Arrow 186ffeb744 docs: clarify botstopper kubernetes instructions (#1404)
This makes it clear that when generating a kubernetes secret to pull the bot stopper image that:
- no email is required
- a user is required but the actual value of the username is not checked
- the GH token needs to be pasted in

Signed-off-by: Thomas Arrow <tarrow@users.noreply.github.com>
2026-01-15 11:13:10 +00:00
Xe Iaso ff87aac4e7 fix(web): include base prefix in generated URLs (#1403)
* fix(web): include base prefix in generated URLs

Forgot to add the base prefix to these URLs. Committed a fix for this
and added a test to ensure this does not repeat. Oops!

Closes: #1402
Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: update CHANGELOG

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
2026-01-14 23:47:44 +00:00
9 changed files with 48 additions and 10 deletions
+2 -1
View File
@@ -64,7 +64,6 @@ ckie
cloudflare
Codespaces
confd
connnection
containerbuild
containerregistry
coreutils
@@ -243,6 +242,7 @@ oci
OCOB
ogtag
oklch
oldstable
omgili
omgilibot
openai
@@ -257,6 +257,7 @@ Pangu
parseable
passthrough
Patreon
perplexitybot
pgrep
phrik
pidfile
+8 -2
View File
@@ -12,6 +12,11 @@ permissions:
jobs:
go_tests:
strategy:
matrix:
go_version:
- oldstable
- stable
#runs-on: alrest-techarohq
runs-on: ubuntu-24.04
steps:
@@ -26,10 +31,11 @@ jobs:
- uses: actions/setup-node@395ad3262231945c25e8478fd5baf05154b1d79f # v6.1.0
with:
node-version: '24.11.0'
node-version: "latest"
- uses: actions/setup-go@4dc6199c7b1a012772edbd06daecab0f50c9053c # v6.1.0
with:
go-version: '1.25.4'
go-version: ${{ matrix.go_version }}
- name: Cache playwright binaries
uses: actions/cache@9255dc7a253b0ccc959486e2bca901246202afeb # v5.0.1
+1 -1
View File
@@ -4,5 +4,5 @@
# - Claude-User: No published IP allowlist
- name: "ai-clients"
user_agent_regex: >-
ChatGPT-User|Claude-User|MistralAI-User
ChatGPT-User|Claude-User|MistralAI-User|Perplexity-User
action: DENY
+12
View File
@@ -0,0 +1,12 @@
# Acts on behalf of user requests
# https://docs.perplexity.ai/guides/bots
- name: perplexity-user
user_agent_regex: Perplexity-User/.+; \+https\://perplexity\.ai/perplexity-user
action: ALLOW
# https://www.perplexity.com/perplexity-user.json
remote_addresses: [
"44.208.221.197/32",
"34.193.163.52/32",
"18.97.21.0/30",
"18.97.43.80/29",
]
+1 -1
View File
@@ -4,5 +4,5 @@
# - Claude-SearchBot: No published IP allowlist
- name: "ai-crawlers-search"
user_agent_regex: >-
OAI-SearchBot|Claude-SearchBot
OAI-SearchBot|Claude-SearchBot|PerplexityBot
action: DENY
+16
View File
@@ -0,0 +1,16 @@
# Indexing for search, does not collect training data
# https://docs.perplexity.ai/guides/bots
- name: perplexitybot
user_agent_regex: PerplexityBot/.+; \+https\://perplexity\.ai/perplexitybot
action: ALLOW
# https://www.perplexity.com/perplexitybot.json
remote_addresses: [
"107.20.236.150/32",
"3.224.62.45/32",
"18.210.92.235/32",
"3.222.232.239/32",
"3.211.124.183/32",
"3.231.139.107/32",
"18.97.1.228/30",
"18.97.9.96/29",
]
+3 -1
View File
@@ -3,5 +3,7 @@
- import: (data)/bots/ai-catchall.yaml
- import: (data)/crawlers/ai-training.yaml
- import: (data)/crawlers/openai-searchbot.yaml
- import: (data)/crawlers/perplexitybot.yaml
- import: (data)/clients/openai-chatgpt-user.yaml
- import: (data)/clients/mistral-mistralai-user.yaml
- import: (data)/clients/mistral-mistralai-user.yaml
- import: (data)/clients/perplexity-user.yaml
+3 -1
View File
@@ -2,5 +2,7 @@
- import: (data)/bots/ai-catchall.yaml
- import: (data)/crawlers/openai-searchbot.yaml
- import: (data)/crawlers/openai-gptbot.yaml
- import: (data)/crawlers/perplexitybot.yaml
- import: (data)/clients/openai-chatgpt-user.yaml
- import: (data)/clients/mistral-mistralai-user.yaml
- import: (data)/clients/mistral-mistralai-user.yaml
- import: (data)/clients/perplexity-user.yaml
+2 -3
View File
@@ -51,9 +51,8 @@ If you are using Kubernetes, you will need to create an image pull secret:
kubectl create secret docker-registry \
techarohq-botstopper \
--docker-server ghcr.io \
--docker-username your-username \
--docker-password your-access-token \
--docker-email your@email.address
--docker-username any-username \
--docker-password <your-access-token> \
```
Then attach it to your Deployment: