arian/anubis-mirror
1
0
Fork 0
mirror of https://github.com/TecharoHQ/anubis.git synced 2026-04-11 19:18:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: iplist2rule utility command (#1373)

Browse Source 
* feat: iplist2rule utility command

Assisted-By: GLM 4.7 via Claude Code
Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: update CHANGELOG

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: fix spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: fix spelling again

Signed-off-by: Xe Iaso <me@xeiaso.net>

* feat(iplist2rule): add comment describing how rule was generated

Signed-off-by: Xe Iaso <me@xeiaso.net>

* docs: add iplist2rule docs

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: fix spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>
This commit is contained in:
Xe Iaso
2025-12-29 12:10:17 -05:00
committed by GitHub
parent 1d8e98c5ec
commit 359613f35a
6 changed files with 223 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
docs/docs/admin/robots2policy.mdx
View File

@@ -12,6 +12,7 @@ Install directly with Go:
```bash
go install github.com/TecharoHQ/anubis/cmd/robots2policy@latest
```
## Usage
Basic conversion from URL:
@@ -35,8 +36,8 @@ robots2policy -input robots.txt -action DENY -format json
## Options
| Flag | Description | Default |
|-----------------------|--------------------------------------------------------------------|---------------------|
| `-input` | robots.txt file path or URL (use `-` for stdin) | *required* |
| --------------------- | ------------------------------------------------------------------ | ------------------- |
| `-input` | robots.txt file path or URL (use `-` for stdin) | _required_ |
| `-output` | Output file (use `-` for stdout) | stdout |
| `-format` | Output format: `yaml` or `json` | `yaml` |
| `-action` | Action for disallowed paths: `ALLOW`, `DENY`, `CHALLENGE`, `WEIGH` | `CHALLENGE` |
@@ -47,6 +48,7 @@ robots2policy -input robots.txt -action DENY -format json
## Example
Input robots.txt:
```txt
User-agent: *
Disallow: /admin/
@@ -57,6 +59,7 @@ Disallow: /
```
Generated policy:
```yaml
- name: robots-txt-policy-disallow-1
action: CHALLENGE
@@ -77,8 +80,8 @@ Generated policy:
Save the output and import it in your main policy file:
```yaml
import:
- path: "./robots-policy.yaml"
bots:
- import: "./robots-policy.yaml"
```
The tool handles wildcard patterns, user-agent specific rules, and blacklisted bots automatically.