docs(cscli): document cscli bot

crowdsec-docs/docs/appsec/bot_detection/customization.md

@@ -221,6 +221,15 @@ Example file:
 
 The reverse-DNS confirmation used by `rdns` goes through the engine's DNS cache; see [`dns_cache`](/configuration/crowdsec_configuration.md#dns_cache) if you need to tune its TTL or size.
 
+To check a definition without sending real traffic, use [`cscli bot match`](/cscli/cscli_bot_match.md). It runs the same matching against the `bots` files declared by your installed appsec-configs, or only by the ones you name, and tells you which definition matched and how:
+
+```bash
+$ sudo cscli bot match --ip 66.249.66.1 --user-agent "Googlebot/2.1" mycorp/appsec-bot-challenge-known-bot
+66.249.66.1 matches bot 'googlebot' (rdns: crawl-66-249-66-1.googlebot.com) from legit_bots/mybot.json
+```
+
+Reverse DNS is resolved from the machine you run it on, so run it where CrowdSec runs.
+
 ### Block on a weak signal the default ignores
 
 The collection scores weak signals but weights them low on purpose, so none can cross the threshold alone. `accept_language` is worth 15 points against a default threshold of 75. It fires when the `Accept-Language` header disagrees with the browser's `navigator.language`: legitimate for some multilingual or proxied setups, unusual for a normal browser. If your traffic makes it worth enforcing, reject on it directly:

crowdsec-docs/docs/appsec/bot_detection/hooks.md

@@ -116,6 +116,8 @@ pre_eval:
 
 Once `ExemptFromChallenge(reason)` has flagged a request, `SendChallenge()` becomes a no-op for the rest of that request, so the exempted client is never challenged.
 
+[`cscli bot match`](/cscli/cscli_bot_match.md) runs the same check from the command line, to find out whether a given IP and User-Agent would be recognized.
+
 ### `ExemptFromChallenge` vs `GrantChallengeCookie`
 
 Both keep a client out of the challenge, but at different scopes:

crowdsec-docs/docs/appsec/bot_detection/whats_included.md

@@ -138,7 +138,7 @@ $ sudo cscli alerts inspect 2019 -d
 +------------------+--------------------------------------------------------------+
 ```
 
-`fingerprint_id` (the `fsid`) is stable per device profile, so it correlates submissions from the same client across IP changes.
+`fingerprint_id` (the `fsid`) is stable per device profile, so it correlates submissions from the same client across IP changes. [`cscli bot fsid`](/cscli/cscli_bot_fsid.md) decodes it back into what it encodes: the bot-signal bitmasks, screen geometry, CPU and memory, language and timezone. The other signals are only kept as truncated hashes.
 
 The same alert in the Console, where most people will actually read it. Expanding the context shows the same fields as tags, with `score_reasons` spelling out how the client reached 115:
 

crowdsec-docs/docs/cscli/cscli.md

@@ -35,6 +35,7 @@ cscli [flags]
 * [cscli allowlists](/cscli/cscli_allowlists.md)	 - Manage centralized allowlists
 * [cscli appsec-configs](/cscli/cscli_appsec-configs.md)	 - Manage hub appsec-configs
 * [cscli appsec-rules](/cscli/cscli_appsec-rules.md)	 - Manage hub appsec-rules
+* [cscli bot](/cscli/cscli_bot.md)	 - Inspect bot detection: known-bot matching and fingerprint ids
 * [cscli bouncers](/cscli/cscli_bouncers.md)	 - Manage bouncers [requires local API]
 * [cscli capi](/cscli/cscli_capi.md)	 - Manage interaction with Central API (CAPI)
 * [cscli collections](/cscli/cscli_collections.md)	 - Manage hub collections

crowdsec-docs/docs/cscli/cscli_bot.md

@@ -0,0 +1,37 @@
+---
+id: cscli_bot
+title: cscli bot
+---
+## cscli bot
+
+Inspect bot detection: known-bot matching and fingerprint ids
+
+```
+cscli bot <action> [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for bot
+```
+
+### Options inherited from parent commands
+
+```
+      --color string    Output color: yes, no, auto (default "auto")
+  -c, --config string   path to crowdsec config file (default "/etc/crowdsec/config.yaml")
+      --debug           Set logging to debug
+      --error           Set logging to error
+      --info            Set logging to info
+  -o, --output string   Output format: human, json, raw
+      --trace           Set logging to trace
+      --warning         Set logging to warning
+```
+
+### SEE ALSO
+
+* [cscli](/cscli/cscli.md)	 - cscli allows you to manage crowdsec
+* [cscli bot fsid](/cscli/cscli_bot_fsid.md)	 - Decode a fingerprint id (FSID) back into a fingerprint
+* [cscli bot match](/cscli/cscli_bot_match.md)	 - Check if a request would be recognized as a known bot
+

crowdsec-docs/docs/cscli/cscli_bot_fsid.md

@@ -0,0 +1,53 @@
+---
+id: cscli_bot_fsid
+title: cscli bot fsid
+---
+## cscli bot fsid
+
+Decode a fingerprint id (FSID) back into a fingerprint
+
+### Synopsis
+
+Decode the FSID a challenged browser reports and rebuild the fingerprint object
+from it. Only what the FSID encodes can be recovered: bitmasks, screen geometry,
+cpu/memory, language and timezone. The remaining signals were folded into
+truncated hashes and are shown as such.
+
+Use '-' to read the FSID from stdin. With -o json the reconstructed fingerprint
+is printed instead of the summary.
+
+```
+cscli bot fsid <fsid> [flags]
+```
+
+### Examples
+
+```
+cscli bot fsid FS1_000000000000000000000_00000h1f2e_1920x1080c8m8b1000h-3ab1_f...
+cscli bot fsid -o json FS1_...
+echo 'FS1_...' | cscli bot fsid -
+```
+
+### Options
+
+```
+  -h, --help   help for fsid
+```
+
+### Options inherited from parent commands
+
+```
+      --color string    Output color: yes, no, auto (default "auto")
+  -c, --config string   path to crowdsec config file (default "/etc/crowdsec/config.yaml")
+      --debug           Set logging to debug
+      --error           Set logging to error
+      --info            Set logging to info
+  -o, --output string   Output format: human, json, raw
+      --trace           Set logging to trace
+      --warning         Set logging to warning
+```
+
+### SEE ALSO
+
+* [cscli bot](/cscli/cscli_bot.md)	 - Inspect bot detection: known-bot matching and fingerprint ids
+

crowdsec-docs/docs/cscli/cscli_bot_match.md

@@ -0,0 +1,51 @@
+---
+id: cscli_bot_match
+title: cscli bot match
+---
+## cscli bot match
+
+Check if a request would be recognized as a known bot
+
+### Synopsis
+
+Run the MatchKnownBot() expr helper against the 'bots' data files declared by
+the given appsec-configs (all installed ones by default). Reverse DNS is resolved at
+runtime, so the result depends on what the local resolver answers.
+
+```
+cscli bot match [appsec-config]... [flags]
+```
+
+### Examples
+
+```
+cscli bot match --ip 66.249.66.1 --user-agent "Googlebot/2.1"
+cscli bot match --ip 1.2.3.4 --path /robots.txt crowdsecurity/appsec-default
+```
+
+### Options
+
+```
+  -h, --help                help for match
+      --ip string           Source IP address of the request
+      --path string         URI path of the request
+      --user-agent string   User-Agent header of the request
+```
+
+### Options inherited from parent commands
+
+```
+      --color string    Output color: yes, no, auto (default "auto")
+  -c, --config string   path to crowdsec config file (default "/etc/crowdsec/config.yaml")
+      --debug           Set logging to debug
+      --error           Set logging to error
+      --info            Set logging to info
+  -o, --output string   Output format: human, json, raw
+      --trace           Set logging to trace
+      --warning         Set logging to warning
+```
+
+### SEE ALSO
+
+* [cscli bot](/cscli/cscli_bot.md)	 - Inspect bot detection: known-bot matching and fingerprint ids
+

crowdsec-docs/sidebars.ts

@@ -392,6 +392,15 @@ const sidebarsConfig: SidebarConfig = {
 				"cscli/cscli_appsec-rules_upgrade",
 			],
 		},
+		{
+			type: "category",
+			label: "cscli bot",
+			link: {
+				type: "doc",
+				id: "cscli/cscli_bot",
+			},
+			items: ["cscli/cscli_bot_fsid", "cscli/cscli_bot_match"],
+		},
 		{
 			type: "category",
 			label: "cscli bouncers",

crowdsec-docs/versioned_docs/version-v1.8/appsec/bot_detection/customization.md

@@ -2,7 +2,7 @@
 id: customization
 title: "Bot Detection Customization & Recipes"
 sidebar_label: "Customization & recipes"
-description: "Ready-to-use appsec-config customization for CrowdSec bot detection: challenge specific routes, tune scoring, and allow verified crawlers."
+description: "Ready-to-use appsec-config overlays for CrowdSec bot detection: challenge specific routes, tune scoring, and allow verified crawlers without editing hub files."
 sidebar_position: 4
 ---
 
@@ -221,6 +221,15 @@ Example file:
 
 The reverse-DNS confirmation used by `rdns` goes through the engine's DNS cache; see [`dns_cache`](/configuration/crowdsec_configuration.md#dns_cache) if you need to tune its TTL or size.
 
+To check a definition without sending real traffic, use [`cscli bot match`](/cscli/cscli_bot_match.md). It runs the same matching against the `bots` files declared by your installed appsec-configs, or only by the ones you name, and tells you which definition matched and how:
+
+```bash
+$ sudo cscli bot match --ip 66.249.66.1 --user-agent "Googlebot/2.1" mycorp/appsec-bot-challenge-known-bot
+66.249.66.1 matches bot 'googlebot' (rdns: crawl-66-249-66-1.googlebot.com) from legit_bots/mybot.json
+```
+
+Reverse DNS is resolved from the machine you run it on, so run it where CrowdSec runs.
+
 ### Block on a weak signal the default ignores
 
 The collection scores weak signals but weights them low on purpose, so none can cross the threshold alone. `accept_language` is worth 15 points against a default threshold of 75. It fires when the `Accept-Language` header disagrees with the browser's `navigator.language`: legitimate for some multilingual or proxied setups, unusual for a normal browser. If your traffic makes it worth enforcing, reject on it directly:

crowdsec-docs/versioned_docs/version-v1.8/appsec/bot_detection/hooks.md

@@ -116,6 +116,8 @@ pre_eval:
 
 Once `ExemptFromChallenge(reason)` has flagged a request, `SendChallenge()` becomes a no-op for the rest of that request, so the exempted client is never challenged.
 
+[`cscli bot match`](/cscli/cscli_bot_match.md) runs the same check from the command line, to find out whether a given IP and User-Agent would be recognized.
+
 ### `ExemptFromChallenge` vs `GrantChallengeCookie`
 
 Both keep a client out of the challenge, but at different scopes:

crowdsec-docs/versioned_docs/version-v1.8/appsec/bot_detection/whats_included.md

@@ -2,7 +2,7 @@
 id: whats_included
 title: "Bot Detection Default Configuration"
 sidebar_label: "Default Configuration"
-description: "What the crowdsecurity/appsec-bot-challenge collection installs: the bot scoring and browser fingerprinting, with thresholds and the behavioral scenarios."
+description: "What the crowdsecurity/appsec-bot-challenge collection installs: the two scoring and threshold appsec-configs, plus the behavioral scenarios it ships with."
 sidebar_position: 3
 ---
 
@@ -138,7 +138,7 @@ $ sudo cscli alerts inspect 2019 -d
 +------------------+--------------------------------------------------------------+
 ```
 
-`fingerprint_id` (the `fsid`) is stable per device profile, so it correlates submissions from the same client across IP changes.
+`fingerprint_id` (the `fsid`) is stable per device profile, so it correlates submissions from the same client across IP changes. [`cscli bot fsid`](/cscli/cscli_bot_fsid.md) decodes it back into what it encodes: the bot-signal bitmasks, screen geometry, CPU and memory, language and timezone. The other signals are only kept as truncated hashes.
 
 The same alert in the Console, where most people will actually read it. Expanding the context shows the same fields as tags, with `score_reasons` spelling out how the client reached 115:
 

crowdsec-docs/versioned_docs/version-v1.8/cscli/cscli.md

@@ -35,6 +35,7 @@ cscli [flags]
 * [cscli allowlists](/cscli/cscli_allowlists.md)	 - Manage centralized allowlists
 * [cscli appsec-configs](/cscli/cscli_appsec-configs.md)	 - Manage hub appsec-configs
 * [cscli appsec-rules](/cscli/cscli_appsec-rules.md)	 - Manage hub appsec-rules
+* [cscli bot](/cscli/cscli_bot.md)	 - Inspect bot detection: known-bot matching and fingerprint ids
 * [cscli bouncers](/cscli/cscli_bouncers.md)	 - Manage bouncers [requires local API]
 * [cscli capi](/cscli/cscli_capi.md)	 - Manage interaction with Central API (CAPI)
 * [cscli collections](/cscli/cscli_collections.md)	 - Manage hub collections

crowdsec-docs/versioned_docs/version-v1.8/cscli/cscli_bot.md

@@ -0,0 +1,37 @@
+---
+id: cscli_bot
+title: cscli bot
+---
+## cscli bot
+
+Inspect bot detection: known-bot matching and fingerprint ids
+
+```
+cscli bot <action> [flags]
+```
+
+### Options
+
+```
+  -h, --help   help for bot
+```
+
+### Options inherited from parent commands
+
+```
+      --color string    Output color: yes, no, auto (default "auto")
+  -c, --config string   path to crowdsec config file (default "/etc/crowdsec/config.yaml")
+      --debug           Set logging to debug
+      --error           Set logging to error
+      --info            Set logging to info
+  -o, --output string   Output format: human, json, raw
+      --trace           Set logging to trace
+      --warning         Set logging to warning
+```
+
+### SEE ALSO
+
+* [cscli](/cscli/cscli.md)	 - cscli allows you to manage crowdsec
+* [cscli bot fsid](/cscli/cscli_bot_fsid.md)	 - Decode a fingerprint id (FSID) back into a fingerprint
+* [cscli bot match](/cscli/cscli_bot_match.md)	 - Check if a request would be recognized as a known bot
+

crowdsec-docs/versioned_docs/version-v1.8/cscli/cscli_bot_fsid.md

@@ -0,0 +1,53 @@
+---
+id: cscli_bot_fsid
+title: cscli bot fsid
+---
+## cscli bot fsid
+
+Decode a fingerprint id (FSID) back into a fingerprint
+
+### Synopsis
+
+Decode the FSID a challenged browser reports and rebuild the fingerprint object
+from it. Only what the FSID encodes can be recovered: bitmasks, screen geometry,
+cpu/memory, language and timezone. The remaining signals were folded into
+truncated hashes and are shown as such.
+
+Use '-' to read the FSID from stdin. With -o json the reconstructed fingerprint
+is printed instead of the summary.
+
+```
+cscli bot fsid <fsid> [flags]
+```
+
+### Examples
+
+```
+cscli bot fsid FS1_000000000000000000000_00000h1f2e_1920x1080c8m8b1000h-3ab1_f...
+cscli bot fsid -o json FS1_...
+echo 'FS1_...' | cscli bot fsid -
+```
+
+### Options
+
+```
+  -h, --help   help for fsid
+```
+
+### Options inherited from parent commands
+
+```
+      --color string    Output color: yes, no, auto (default "auto")
+  -c, --config string   path to crowdsec config file (default "/etc/crowdsec/config.yaml")
+      --debug           Set logging to debug
+      --error           Set logging to error
+      --info            Set logging to info
+  -o, --output string   Output format: human, json, raw
+      --trace           Set logging to trace
+      --warning         Set logging to warning
+```
+
+### SEE ALSO
+
+* [cscli bot](/cscli/cscli_bot.md)	 - Inspect bot detection: known-bot matching and fingerprint ids
+