feat(data): add Meta's web indexer used for AI purposes (#1573)

This indexer is documented in
https://developers.facebook.com/docs/sharing/webmasters/web-crawlers. I
saw it parsing the entirety of my Forgejo instance, so I suggest to
widely block it.

Signed-off-by: Benjamin Bouvier <benjamin@bouvier.cc>

data/crawlers/ai-search.yaml

@@ -4,5 +4,5 @@
 #  - Claude-SearchBot: No published IP allowlist
 - name: "ai-crawlers-search"
   user_agent_regex: >-
-    OAI-SearchBot|Claude-SearchBot|PerplexityBot
+    OAI-SearchBot|Claude-SearchBot|PerplexityBot|meta-webindexer
   action: DENY

web/static/robots.txt

@@ -54,6 +54,7 @@ User-agent: meta-externalagent
 User-agent: Meta-ExternalAgent
 User-agent: meta-externalfetcher
 User-agent: Meta-ExternalFetcher
+User-agent: meta-webindexer
 User-agent: MistralAI-User
 User-agent: MistralAI-User/1.0
 User-agent: MyCentralAIScraperBot