docs: add iplist2rule docs

Signed-off-by: Xe Iaso <me@xeiaso.net>
This commit is contained in:
Xe Iaso
2025-12-29 10:58:11 -05:00
parent ad41bc7a25
commit c84a414570
3 changed files with 63 additions and 6 deletions
+50
View File
@@ -0,0 +1,50 @@
---
title: iplist2rule CLI tool
---
The `iplist2rule` tool converts IP blocklists into Anubis challenge policies. It reads common IP block list formats and generates the appropriate Anubis policy file for IP address filtering.
## Installation
Install directly with Go
```bash
go install github.com/TecharoHQ/anubis/utils/cmd/iplist2rule@latest
```
## Usage
Basic conversion from URL:
```bash
iplist2rule https://raw.githubusercontent.com/7c/torfilter/refs/heads/main/lists/txt/torfilter-1m-flat.txt filter-tor.yaml
```
Explicitly allow every IP address on a list:
```bash
iplist2rule --action ALLOW https://raw.githubusercontent.com/7c/torfilter/refs/heads/main/lists/txt/torfilter-1m-flat.txt filter-tor.yaml
```
Add weight to requests matching IP addresses on a list:
```bash
iplist2rule --action WEIGH --weight 20 https://raw.githubusercontent.com/7c/torfilter/refs/heads/main/lists/txt/torfilter-1m-flat.txt filter-tor.yaml
```
## Options
| Flag | Description | Default |
| :------------ | :----------------------------------------------------------------------------------------------- | :-------------------------------- |
| `--action` | The Anubis action to take for the IP address in question, must be in ALL CAPS. | `DENY` (forbids traffic) |
| `--rule-name` | The name for the generated Anubis rule, should be in kebab-case. | (not set, inferred from filename) |
| `--weight` | When `--action=WEIGH`, how many weight points should be added or removed from matching requests? | 0 (not set) |
## Using the Generated Policy
Save the output and import it in your main policy file:
```yaml
bots:
- import: "./filter-tor.yaml"
```
+7 -4
View File
@@ -12,6 +12,7 @@ Install directly with Go:
```bash ```bash
go install github.com/TecharoHQ/anubis/cmd/robots2policy@latest go install github.com/TecharoHQ/anubis/cmd/robots2policy@latest
``` ```
## Usage ## Usage
Basic conversion from URL: Basic conversion from URL:
@@ -35,8 +36,8 @@ robots2policy -input robots.txt -action DENY -format json
## Options ## Options
| Flag | Description | Default | | Flag | Description | Default |
|-----------------------|--------------------------------------------------------------------|---------------------| | --------------------- | ------------------------------------------------------------------ | ------------------- |
| `-input` | robots.txt file path or URL (use `-` for stdin) | *required* | | `-input` | robots.txt file path or URL (use `-` for stdin) | _required_ |
| `-output` | Output file (use `-` for stdout) | stdout | | `-output` | Output file (use `-` for stdout) | stdout |
| `-format` | Output format: `yaml` or `json` | `yaml` | | `-format` | Output format: `yaml` or `json` | `yaml` |
| `-action` | Action for disallowed paths: `ALLOW`, `DENY`, `CHALLENGE`, `WEIGH` | `CHALLENGE` | | `-action` | Action for disallowed paths: `ALLOW`, `DENY`, `CHALLENGE`, `WEIGH` | `CHALLENGE` |
@@ -47,6 +48,7 @@ robots2policy -input robots.txt -action DENY -format json
## Example ## Example
Input robots.txt: Input robots.txt:
```txt ```txt
User-agent: * User-agent: *
Disallow: /admin/ Disallow: /admin/
@@ -57,6 +59,7 @@ Disallow: /
``` ```
Generated policy: Generated policy:
```yaml ```yaml
- name: robots-txt-policy-disallow-1 - name: robots-txt-policy-disallow-1
action: CHALLENGE action: CHALLENGE
@@ -77,8 +80,8 @@ Generated policy:
Save the output and import it in your main policy file: Save the output and import it in your main policy file:
```yaml ```yaml
import: bots:
- path: "./robots-policy.yaml" - import: "./robots-policy.yaml"
``` ```
The tool handles wildcard patterns, user-agent specific rules, and blacklisted bots automatically. The tool handles wildcard patterns, user-agent specific rules, and blacklisted bots automatically.
+6 -2
View File
@@ -25,15 +25,19 @@ func init() {
flag.Usage = func() { flag.Usage = func() {
fmt.Printf(`Usage of %[1]s: fmt.Printf(`Usage of %[1]s:
%[1]s [--action=DENY|ALLOW] [--rule-name=] <blocklist-url> <filename> %[1]s [flags] <blocklist-url> <filename>
Grabs the contents of the blocklist, converts it to an Anubis ruleset, and writes it to filename. Grabs the contents of the blocklist, converts it to an Anubis ruleset, and writes it to filename.
Flags:
`, filepath.Base(os.Args[0])) `, filepath.Base(os.Args[0]))
flag.PrintDefaults()
} }
} }
var ( var (
action = flag.String("action", "DENY", "Anubis action to take (ALLOW / DENY)") action = flag.String("action", "DENY", "Anubis action to take (ALLOW / DENY / WEIGH)")
manualRuleName = flag.String("rule-name", "", "If set, prefer this name over inferring from filename") manualRuleName = flag.String("rule-name", "", "If set, prefer this name over inferring from filename")
weight = flag.Int("weight", 0, "If set to any number, add/subtract this many weight points when --action=WEIGH") weight = flag.Int("weight", 0, "If set to any number, add/subtract this many weight points when --action=WEIGH")
) )