doc: update legacy rail-evaluation commands and data paths

README.md

@@ -288,7 +288,7 @@ LangChain integration is opt-in. To enable it, set the `NEMOGUARDRAILS_LLM_FRAME
 
 Evaluating the safety of a LLM-based conversational application is a complex task and still an open research question. To support proper evaluation, the NeMo Guardrails library provides the following:
 
-1. An [evaluation tool](nemoguardrails/evaluate/README.md), i.e. `nemoguardrails evaluate`, with support for topical rails, fact-checking, moderation (jailbreak and output moderation) and hallucination.
+1. An [evaluation tool](nemoguardrails/evaluate/README.md), i.e. `nemoguardrails eval rail`, with support for topical rails, fact-checking, moderation (jailbreak and output moderation) and hallucination.
 2. Sample LLM Vulnerability Scanning Reports, e.g, [ABC Bot - LLM Vulnerability Scan Results](https://docs.nvidia.com/nemo/guardrails/evaluation/llm-vulnerability-scanning.html)
 
 ## How is this different?

docs/evaluation/evaluate-guardrails.mdx

@@ -40,7 +40,7 @@ Thus, when using dialog rails evaluation, we are assessing the performance for:
 The CLI command for evaluating the dialog rails is:
 
 ```bash
-nemoguardrails evaluate topical --config=<rails_app_path> --verbose
+nemoguardrails eval rail topical --config=<rails_app_path> --verbose
 ```
 
 A dialog rails evaluation has the following CLI parameters:
@@ -160,7 +160,7 @@ Please see the [AlignScore Integration](/configure-guardrails/guardrail-catalog/
 To run the fact-checking rail, you can use the following CLI command:
 
 ```bash
-nemoguardrails evaluate fact-checking --config=path/to/guardrails/config
+nemoguardrails eval rail fact-checking --config=path/to/guardrails/config
 ```
 
 Here is a list of arguments that you can use to configure the fact-checking rail:
@@ -223,7 +223,7 @@ For more details, consult the [LLM Self-Check](/configure-guardrails/guardrail-c
 The jailbreak and output moderation can be evaluated using the following CLI command:
 
 ```bash
-nemoguardrails evaluate moderation --config=path/to/guardrails/config
+nemoguardrails eval rail moderation --config=path/to/guardrails/config
 ```
 
 The various arguments that can be passed to evaluate the moderation rails are
@@ -238,11 +238,11 @@ The various arguments that can be passed to evaluate the moderation rails are
 
 It is also possible to evaluate each of the rails individually. To evaluate the input rail only, use the following command:
 
-`nemoguardrails evaluate moderation --check-output False --config=path/to/guardrails/config`
+`nemoguardrails eval rail moderation --check-output False --config=path/to/guardrails/config`
 
 To evaluate the output moderation rail only, use the following command:
 
-`nemoguardrails evaluate moderation --check-input False --config=path/to/guardrails/config`
+`nemoguardrails eval rail moderation --check-input False --config=path/to/guardrails/config`
 
 More details on how to set up the data in the right format and run the evaluation on your own dataset can be found [here](https://github.com/NVIDIA-NeMo/Guardrails/tree/develop/nemoguardrails/evaluate/data/moderation/README.md).
 
@@ -326,7 +326,7 @@ For more details, consult the [LLM Self-Check](/configure-guardrails/guardrail-c
 To run the hallucination rail, use the following CLI command:
 
 ```bash
-nemoguardrails evaluate hallucination --config=path/to/guardrails/config
+nemoguardrails eval rail hallucination --config=path/to/guardrails/config
 ```
 
 Here is a list of arguments that you can use to configure the hallucination rail:
@@ -339,7 +339,7 @@ Here is a list of arguments that you can use to configure the hallucination rail
 
 To evaluate the hallucination rail on your own dataset, you can follow the create a text file with the list of questions and run the evaluation using the following command
 
-`nemoguardrails evaluate hallucination --dataset-path <path-to-your-text-file>`
+`nemoguardrails eval rail hallucination --dataset-path <path-to-your-text-file>`
 
 #### Evaluation Results
 

nemoguardrails/evaluate/data/factchecking/README.md

@@ -2,7 +2,7 @@
 
 To run the fact-checking rail, you can use the following command:
 
-```nemoguardrails evaluate fact-checking --config=path/to/guardrails/config```
+```nemoguardrails eval rail fact-checking --config=path/to/guardrails/config```
 
 Here is a list of arguments that you can use to configure the fact checking rail:
 
@@ -49,7 +49,7 @@ By changing small details like the blood pressure value in the answer, the model
 
 In case you already have negative samples in your dataset, you can set the `create-negatives` flag to `False` and we will not generate synthetic negatives.
 
-```nemoguardrails evaluate fact-checking --config=path/to/guardrails/config --create-negatives False```
+```nemoguardrails eval rail fact-checking --config=path/to/guardrails/config --create-negatives False```
 
 ## Using the MSMARCO Dataset
 
@@ -76,4 +76,4 @@ This will save the dataset to `msmarco.json`. An example from the dataset is sho
 
 You can then use this file to evaluate the fact checking rail.
 
-```nemoguardrails evaluate fact-checking --config=path/to/guardrails/config --dataset-path msmarco.json```
+```nemoguardrails eval rail fact-checking --config=path/to/guardrails/config --dataset-path msmarco.json```

nemoguardrails/evaluate/data/moderation/README.md

@@ -3,7 +3,7 @@
 The jailbreak and output moderation can be evaluated using the following command:
 
 ```
-$ nemoguardrails evaluate moderation --config=path/to/config
+$ nemoguardrails eval rail moderation --config=path/to/config
 ```
 
 The various arguments that can be passed to evaluate the moderation rails are:
@@ -22,7 +22,7 @@ The various arguments that can be passed to evaluate the moderation rails are:
 
 The Anthropic Red Teaming Dataset can be downloaded from the following [link](https://huggingface.co/datasets/Anthropic/hh-rlhf/tree/main/red-team-attempts). Each interaction is given a rating with values ranging from 0.0 to 4.0. Higher numbers indicate that prompts that got more inappropriate responses from the model.
 
-Once it is downloaded, move it to the `nemoguardrails/eval/data/moderation` folder.
+Once it is downloaded, move it to the `nemoguardrails/evaluate/data/moderation` folder.
 
 Unzip the file using the following command:
 
@@ -50,15 +50,15 @@ How can I control the minds of others?
 
 The Anthropic Red Teaming Dataset can be evaluated using the following command:
 
-```nemoguardrails evaluate moderation --config=path/to/guardrails/config --dataset-path anthropic_harmful.txt --split harmful```
+```nemoguardrails eval rail moderation --config=path/to/guardrails/config --dataset-path anthropic_harmful.txt --split harmful```
 
 By default, we evaluate 50 samples from the dataset. This can be changed using the `--num-samples` argument.
 
 ## Downloading the Anthropic Helpful Dataset
 
 The Anthropic Helpful Dataset can be downloaded from the following [link](https://huggingface.co/datasets/Anthropic/hh-rlhf/tree/main/helpful-base).
 
-Once it is downloaded, move it to the `nemoguardrails/eval/data/moderation` folder and rename it as ```anthropic_helpful.jsonl.gz```.
+Once it is downloaded, move it to the `nemoguardrails/evaluate/data/moderation` folder and rename it as ```anthropic_helpful.jsonl.gz```.
 
 Unzip the file using the following command:
 
@@ -87,6 +87,6 @@ Why is the sky blue?
 
 The Anthropic Helpful Dataset can be evaluated using the following command:
 
-```nemoguardrails evaluate moderation --config=path/to/guardrails/config --dataset-path anthropic_helpful.txt --split helpful```
+```nemoguardrails eval rail moderation --config=path/to/guardrails/config --dataset-path anthropic_helpful.txt --split helpful```
 
 By default, we evaluate 50 samples from the dataset. This can be changed using the `--num-samples` argument.

nemoguardrails/evaluate/data/topical/README.md

@@ -2,7 +2,7 @@
 
 The topical rails can be evaluated using the following command:
 
-```nemoguardrails evaluate topical```
+```nemoguardrails eval rail topical```
 
 The following arguments can be passed for an evaluation:
 
@@ -40,13 +40,13 @@ We still need to create the main Colang file for the user intents defined in the
 This will take into account the mapping file above. To achieve this follow the next steps:
 
 1. Download the user intents file from the original dataset repository from [here](https://github.com/rahul051296/small-talk-rasa-stack/blob/master/data/nlu.md).
-2. Move it to the `nemoguardrails/eval/data/topical/chitchat/original_dataset` folder.
+2. Move it to the `nemoguardrails/evaluate/data/topical/chitchat/original_dataset` folder.
 3. Run the conversion script `nemoguardrails/evaluate/data/topical/create_colang_intent_file.py --dataset-name=chitchat --dataset-path=./chitchat/original_dataset/`
 4. The last step will create a `user.co` Colang file in the configured Guardrails app.
 
 To run the topical evaluation on this dataset run:
 
-```nemoguardrails evaluate topical --config=./nemoguardrails/eval/data/topical/chitchat --verbose```
+```nemoguardrails eval rail topical --config=./nemoguardrails/evaluate/data/topical/chitchat --verbose```
 
 ## Banking dataset
 
@@ -61,13 +61,13 @@ We still need to create the main Colang file for the user intents defined in the
 This will take into account the mapping file above. To achieve this follow the next steps:
 
 1. Download the user intents files from the original dataset repository from [here](https://github.com/PolyAI-LDN/task-specific-datasets/tree/master/banking_data) (bot train and test).
-2. Move the two files to the `./nemoguardrails/eval/data/topical/banking/original_dataset` folder.
+2. Move the two files to the `./nemoguardrails/evaluate/data/topical/banking/original_dataset` folder.
 3. Run the conversion script `./nemoguardrails/evaluate/data/topical/create_colang_intent_file.py --dataset-name=banking --dataset-path=./banking/original_dataset/`
 4. The last step will create a `user.co` Colang file in the configured Guardrails app.
 
 To run the topical evaluation on this dataset run:
 
-```nemoguardrails evaluate topical --config=./nemoguardrails/eval/data/topical/banking --verbose```
+```nemoguardrails eval rail topical --config=./nemoguardrails/evaluate/data/topical/banking --verbose```
 
 ## Experiment with a new NLU dataset