Rule Test JSON 5-1 update

.github/workflows/python-app.yml

@@ -65,6 +65,8 @@ jobs:
     runs-on: ubuntu-latest
     needs: detect-changes
     if: needs.detect-changes.outputs.ashrae9012019 == 'true'
+    env:
+      RCT_DISABLE_CACHE: "1"
     steps:
       - name: Checkout code
         uses: actions/checkout@v3
@@ -89,6 +91,8 @@ jobs:
     runs-on: ubuntu-latest
     needs: detect-changes
     if: needs.detect-changes.outputs.ashrae9012022 == 'true'
+    env:
+      RCT_DISABLE_CACHE: "1"
     steps:
       - name: Checkout code
         uses: actions/checkout@v3

AGENTS.md

@@ -0,0 +1,88 @@
+# AGENTS.md
+
+# Contribution Guide for AI Agents
+
+This document describes the expected workflow for contributing to this repository. Follow it for any change that may land in a pull request.
+
+## Repository Layout (rct229)
+The core package lives in `rct229/` with this structure:
+- `rct229/rule_engine` — RCT evaluation engine.
+- `rct229/report_engine` — reporting engine.
+- `rct229/rulesets` — ruleset plugins/play rulesets (currently supports ASHRAE 90.1 2019 and 2022).
+- `rct229/reports` — ruleset evaluation reports (currently supports ASHRAE 90.1 2019 and 2022).
+- `rct229/ruletest_engine` — rule test engine.
+- `rct229/schema` — ruleset evaluation schema (RES).
+- `rct229/utils` — helper functions.
+
+Related assets live under `docs/` (RDS), `examples/`, and `test/`.
+
+## General Guidance
+- The package targets Python 3.10 and uses Poetry for dependency management.
+- The default install includes the ASHRAE 90.1-2019 ruleset. The 2022 ruleset is optional.
+- Follow the rule definition and rule test workflows described in `README.md`.
+- Keep updates consistent with the Rule Definition Strategy (RDS) documents under `docs/`.
+
+## Building and Testing
+Install dependencies with Poetry:
+- Default ruleset (2019): `poetry install`
+- Add 2022 ruleset: `poetry install --extras ashrae9012022`
+- All available rulesets: `poetry install --all-extras`
+
+Run tests:
+- Unit tests: `poetry run pytest -v`
+- Coverage: `poetry run pytest --cov`
+- Rule tests (2019): `poetry run rct229 test -rs ashrae9012019`
+- Rule tests (2022): `poetry run rct229 test -rs ashrae9012022`
+
+CLI smoke test example:
+- `poetry run rct229 evaluate -rs ashrae9012019 -f examples/chicago_demo/baseline_model.json -f examples/chicago_demo/proposed_model.json -f examples/chicago_demo/user_model.json -r ASHRAE9012019_DETAIL`
+
+## CI/CD Pipeline (GitHub Actions)
+Workflow: `.github/workflows/python-app.yml` (runs on pull requests).
+- **Black formatting check** using `psf/black@stable` (version 22.6.0).
+- **Unit tests** on Ubuntu with Python 3.10 (`poetry run pytest -v`).
+- **Rule tests** run conditionally based on changed files:
+  - 2019 tests run when files change under `rct229/rulesets/ashrae9012019/**` or `rct229/ruletest_engine/ruletest_jsons/ashrae9012019/**`.
+  - 2022 tests run when files change under `rct229/rulesets/ashrae9012022/**` or `rct229/ruletest_engine/ruletest_jsons/ashrae9012022/**`.
+- Rule tests run with `RCT_DISABLE_CACHE=1` in CI.
+
+## Commit Messages and Pull Requests
+- **Branch/PR naming convention**: `CODE/INITIALS/DESCRIPTIVE_NAME`
+  - `RCT` for high-level repo changes (README, schema, etc.)
+  - `RDS` for RDS docs
+  - `RS` for ruleset code
+  - `RT` for rule test engine
+- Use concise, descriptive commit messages in the imperative mood.
+- Before committing, run:
+  1. `poetry run isort .`
+  2. `poetry run black .`
+  3. `poetry run pytest --cov`
+  4. `poetry run rct229 test` (add `-rs ashrae9012019` or `-rs ashrae9012022` as appropriate)
+
+## Review Checklist
+- **Formatting**: `isort` and `black` applied.
+- **Tests**: `pytest` passes; relevant rule tests executed for any ruleset changes.
+- **Ruleset changes**: updates to `rules_dict`, RDS docs, and rule tests are consistent.
+- **Schema changes**: RES updates align with schema files under `rct229/schema`.
+- **Docs/examples**: README and any referenced docs/examples remain accurate.
+- **Optional dependencies**: changes to 2022 ruleset use Poetry extras as needed.
+
+## Learning Mode
+When users explicitly expressed that they are currently onboarding or learning this repository, the agent shall follow the additional instructions in the learning mode.
+
+### BEFORE AGENT WRITING CODE
+- Explain what you're about to do and why
+- Break it down into steps the user can follow
+- Wait for the user's OK before proceeding
+
+### AFTER WRITING CODE
+- Explain what each part does
+- Ask the user **3 questions** to verify their understanding
+- If the user answer wrong, explain again until the user get it
+- **Do NOT let the user commit** until the user pass your questions
+
+### GENERAL RULES FOR LEARNING MODE
+- **Never** generate code the user can't explain
+- If the user asks for something complex, **suggest simpler alternatives**
+- Treat every session as a **teaching opportunity**
+- Be direct, **Tell the user when they are doing something wrong**

docs/ashrae_90p1_2019/section12/Rule12-1.md

@@ -29,7 +29,7 @@ match_data_element
     - Append the miscellaneous equipment info for reporting: `unexpected_misc_equipment_power.append({"id": misc_equipment_b.id, "baseline_power": misc_equipment_power_b, "proposed_power": misc_equipment_power_p})`
 **Rule Assertion:**  
   - Case 1: If all equipment power was modeled identically: PASS `if len(unexpected_misc_equipment_power) == 0 and len(reduced_misc_equipment_power) == 0: PASS`
-  - Case 2: Else if any equipment power is reduced in the proposed and the compliance path is not "CODE_COMPLIANT" (includes when compliance path is not specified): UNDETERMINED: `elif len(unexpected_misc_equipment_power) == 0 and compliance_path != "CODE_COMPLIANT": UNDETERMINED and raise_message="The proposed building miscellaneous equipment load is less than the baseline, which is only permitted when the model is being used to quantify performance that exceeds the requirements of Standard 90.1."`
+  - Case 2: Else if any equipment power is reduced in the proposed, and there is no unexpected equipment power, and the compliance path is not "CODE_COMPLIANT" (includes when compliance path is not specified): UNDETERMINED: `elif len(unexpected_misc_equipment_power) == 0 and len(reduced_misc_equipment_power) > 0 and compliance_path != "CODE_COMPLIANT": UNDETERMINED and raise_message="The proposed building miscellaneous equipment load is less than the baseline, which is only permitted when the model is being used to quantify performance that exceeds the requirements of Standard 90.1."`
   - Case 3: Else, proposed equipment power is greater than the baseline, or the compliance path is code-compliant and proposed equipment power is less than the baseline: `else: FAIL`
 
 **Notes/Questions:**

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "ruleset-checking-tool"
-version = "0.4.2"
+version = "0.5.0"
 description = "PNNL ruleset checking tool"
 authors = ["Weili Xu <weili.xu@pnnl.gov>", "Charlie Holly <charlie.holly@pnnl.gov>", "Juan Gonzalez <juan.gonzalez@pnnl.gov>", "Yun Joon Jung <yunjoon.jung@pnnl.gov>", "Jiarong Xie <jiarong.xie@pnnl.gov>", "Jackson Jarboe <jackson@karpmanconsulting.net>", "Karen Walkerman <kwalkerman@gard.com>"]
 license = "MIT"

rct229/reports/ashrae9012019/ashrae901_2019_detail_report.py

@@ -75,7 +75,10 @@ def output_node(output_result, output_eval_list, output_outcome_dict):
                     output_outcome_dict[RCTOutcomeLabel.UNDETERMINED] += 1
                 if outcome_label == RCTOutcomeLabel.NOT_APPLICABLE:
                     output_outcome_dict[RCTOutcomeLabel.NOT_APPLICABLE] += 1
-                evaluation_outcome["data_group_id"] = output_result["id"]
+                if "data_group_id" in output_result:
+                    evaluation_outcome["data_group_id"] = output_result["data_group_id"]
+                else:
+                    evaluation_outcome["data_group_id"] = output_result["id"]
                 evaluation_outcome["outcome"] = outcome_label
                 evaluation_outcome["messages"] = (
                     output_result["message"]

rct229/reports/ashrae9012022/__init__.py

@@ -0,0 +1,13 @@
+import importlib
+
+__all__ = ["ashrae901_2022_summary_report", "ashrae901_2022_detail_report"]
+
+
+def __getattr__(name):
+    if name in __all__:
+        return importlib.import_module("." + name, __name__)
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+def __dir__():
+    return sorted(__all__)

rct229/reports/ashrae9012022/ashrae901_2022_detail_report.py

@@ -0,0 +1,102 @@
+import json
+from copy import deepcopy
+
+from rct229.report_engine.rct_report import RCTReport
+from rct229.reports.utils import calc_vals_converter
+from rct229.rule_engine.rct_outcome_label import RCTOutcomeLabel
+
+
+class ASHRAE9012022DetailReport(RCTReport):
+    def __init__(self):
+        super(ASHRAE9012022DetailReport, self).__init__()
+        self.title = "ASHRAE STD 229P RULESET CHECKING TOOL"
+        self.purpose = "Project Testing Report"
+        self.ruleset = "ASHRAE 90.1-2022 Performance Rating Method (Appendix G)"
+        self.ruleset_report_file = f"{self.__class__.__name__}.json"
+
+    def initialize_ruleset_report(self, rule_outcome=None):
+        report_json = {}
+        report_json["title"] = self.title
+        report_json["purpose"] = self.purpose
+        report_json["tool_name"] = self.tool
+        report_json["tool_version"] = self.version
+        report_json["ruleset"] = self.ruleset
+        report_json["date_run"] = self.date_run
+        report_json["schema_version"] = self.schema_version
+        report_json["rpd_files"] = rule_outcome.get("rpd_files")
+        report_json["rules"] = []
+        return report_json
+
+    def generate_rule_report(self, rule_outcome, outcome_dict):
+        # set up the rule metadata
+        rule_id = rule_outcome["id"]
+        rule_report = {}
+        rule_report["rule_id"] = rule_id
+        rule_report["description"] = rule_outcome["description"]
+        rule_report["evaluation_type"] = (
+            "FULL" if rule_outcome["primary_rule"] else "APPLICABILITY"
+        )
+        rule_report["standard_section"] = rule_outcome["standard_section"]
+        rule_report["data_group_names"] = []
+        rule_report_list = []
+        self._rule_outcome_helper(rule_outcome, rule_report_list, outcome_dict)
+        rule_report["evaluations"] = rule_report_list
+        return rule_report
+
+    def add_rule_to_ruleset_report(self, ruleset_report, rule_report, rule_outcome):
+        # rule_report["rule_evaluation_outcome"] = rule_outcome
+        ruleset_report["rules"].append(deepcopy(rule_report))
+
+    def save_ruleset_report(self, ruleset_report, report_dir):
+        with open(report_dir, "w") as output_report:
+            output_report.write(json.dumps(ruleset_report, indent=4))
+
+    def _rule_outcome_helper(self, results, eval_list, outcome_dict):
+        def output_node(output_result, output_eval_list, output_outcome_dict):
+            # in this case, it is the rule checking component
+            evaluation_outcome = {}
+            if any(
+                [
+                    key.startswith("INVALID_") and key.endswith("_CONTEXT")
+                    for key in output_result
+                ]
+            ):
+                evaluation_outcome["invalid_msg"] = "".join(
+                    [output_result[key] for key in output_result]
+                )
+                output_outcome_dict[RCTOutcomeLabel.UNDETERMINED] += 1
+            else:
+                outcome_label = output_result["result"]
+                if outcome_label == RCTOutcomeLabel.PASS:
+                    output_outcome_dict[RCTOutcomeLabel.PASS] += 1
+                if outcome_label == RCTOutcomeLabel.FAILED:
+                    output_outcome_dict[RCTOutcomeLabel.FAILED] += 1
+                if outcome_label == RCTOutcomeLabel.UNDETERMINED:
+                    output_outcome_dict[RCTOutcomeLabel.UNDETERMINED] += 1
+                if outcome_label == RCTOutcomeLabel.NOT_APPLICABLE:
+                    output_outcome_dict[RCTOutcomeLabel.NOT_APPLICABLE] += 1
+                if "data_group_id" in output_result:
+                    evaluation_outcome["data_group_id"] = output_result["data_group_id"]
+                else:
+                    evaluation_outcome["data_group_id"] = output_result["id"]
+                evaluation_outcome["outcome"] = outcome_label
+                evaluation_outcome["messages"] = (
+                    output_result["message"]
+                    if output_result.get("message") is not None
+                    else ""
+                )
+                evaluation_outcome["calculated_values"] = (
+                    calc_vals_converter(output_result["calc_vals"])
+                    if output_result.get("calc_vals") is not None
+                    else ""
+                )
+            output_eval_list.append(evaluation_outcome)
+
+        outcome = results["result"]
+        if type(outcome) is str:
+            output_node(results, eval_list, outcome_dict)
+        elif type(outcome) is dict:
+            output_node(outcome, eval_list, outcome_dict)
+        else:
+            for result in outcome:
+                self._rule_outcome_helper(result, eval_list, outcome_dict)

rct229/reports/ashrae9012022/ashrae901_2022_software_test_report.py

@@ -0,0 +1,75 @@
+import json
+from copy import deepcopy
+
+from rct229 import __version__ as version
+from rct229.report_engine.rct_report import RCTReport
+from rct229.reports.utils import test_evaluation_converter
+
+
+class ASHRAE9012022SoftwareTestReport(RCTReport):
+    def __init__(self):
+        super(ASHRAE9012022SoftwareTestReport, self).__init__()
+        self.title = "ASHRAE STD 229P RULESET CHECKING TOOL"
+        self.purpose = "RCT Ruleset Software Testing Report"
+        self.ruleset = "ASHRAE 90.1-2022 Performance Rating Method (Appendix G)"
+        self.ruleset_report_file = f"{self.__class__.__name__}.json"
+
+    def initialize_ruleset_report(self, rule_outcome=None):
+        report_json = {}
+        report_json["title"] = self.title
+        report_json["purpose"] = self.purpose
+        report_json["tool_name"] = self.tool
+        report_json["tool_version"] = self.version
+        report_json["ruleset"] = self.ruleset
+        report_json["date_run"] = self.date_run
+        report_json["schema_version"] = self.schema_version
+        report_json["rule_tests"] = []
+
+        return report_json
+
+    def generate_rule_report(self, rule_test_dict, outcome_dict):
+        # Sum up the outcomes for every rule unit test evaluation in this rule test. The outcome of this is used by
+        # the calculate_rule_outcome function to determine the overall actual_rule_unit_test_evaluation_outcome
+        test_evaluations = []
+        for test_evaluation in rule_test_dict["rule_unit_test_evaluation"]:
+            result = test_evaluation["result"]
+            outcome_dict[result] += 1
+            test_evaluations.append(test_evaluation_converter(test_evaluation))
+        rule_test_dict["rule_unit_test_evaluation"] = test_evaluations
+        return rule_test_dict
+
+    def add_rule_to_ruleset_report(self, ruleset_report, rule_test_dict, rule_outcome):
+        # Record outcome of aggregated pass/fails as determined by calculate_rule_outcome
+        rule_test_dict["actual_rule_unit_test_evaluation_outcome"] = rule_outcome
+
+        # Pull out the expected outcome to compare it with the actual outcome
+        expected_outcome = rule_test_dict["expected_rule_unit_test_evaluation_outcome"]
+
+        # Record agreement between expected and actual outcome
+        rule_test_dict["rule_unit_test_outcome_agreement"] = (
+            expected_outcome == rule_outcome
+        )
+
+        # Reorder keys to match intended order
+        key_order = [
+            "rule_id",
+            "test_id",
+            "test_description",
+            "ruleset_section",
+            "ruleset_section_title",
+            "evaluation_type",
+            "rule_unit_test_outcome_agreement",
+            "expected_rule_unit_test_evaluation_outcome",
+            "actual_rule_unit_test_evaluation_outcome",
+            "rule_unit_test_evaluation",
+        ]
+
+        # Create new dictionary with proper order of keys
+        ordered_rule_test_dict = {key: rule_test_dict[key] for key in key_order}
+
+        # Append ruletest to ruletest report
+        ruleset_report["rule_tests"].append(deepcopy(ordered_rule_test_dict))
+
+    def save_ruleset_report(self, ruleset_report, report_dir):
+        with open(report_dir, "w") as output_report:
+            output_report.write(json.dumps(ruleset_report, indent=4))