Add: New azure-data-collection-rules skill for Azure Monitor DCR authoring (reopens #2575)

evals/azure-data-collection-rules/README.md

@@ -0,0 +1,41 @@
+# DCR Authoring Vally Eval Suite
+
+Integration tests for the `azure-data-collection-rules` skill using [Vally](https://aka.ms/vally).
+
+## Quick Start
+
+```bash
+cd tests
+npm run test:vally -- --skill azure-data-collection-rules
+```
+
+## Test Coverage
+
+| Name | Area | What it tests |
+|------|------|---------------|
+| `trigger-syslog-filter` | routing | Skill invoked for syslog DCR request |
+| `trigger-direct-ingestion` | routing | Skill invoked for direct ingestion request |
+| `trigger-windows-events` | routing | Skill invoked for Windows events DCR |
+| `trigger-multi-stage` | routing | Skill invoked for multi-stage transforms |
+| `trigger-negative-deploy` | routing | Skill NOT invoked for app deployment |
+| `trigger-negative-general` | routing | Skill NOT invoked for unrelated questions |
+| `output-dcr-json-structure` | correctness | Output contains streamDeclarations, dataFlows, destinations |
+| `output-kql-transform` | correctness | Output includes transformKql with proper KQL |
+| `output-direct-ingestion-kind` | correctness | Output has kind: Direct, Custom- stream prefix |
+
+## Structure
+
+```
+evals/
+├── eval.yaml           # Eval spec (stimuli + graders)
+├── README.md           # This file
+└── fixtures/
+    └── sample-logs.json  # Sample data for context
+```
+
+## Notes
+
+- Routing tests use `earlyTerminate` on skill-call to keep runs fast (~30s each)
+- Correctness tests allow up to 30 tool calls before terminating
+- Negative tests verify the skill is NOT loaded for unrelated prompts
+- No Azure subscription required (tests validate agent output, not real deployments)

evals/azure-data-collection-rules/eval.yaml

@@ -0,0 +1,153 @@
+# Vally eval config for azure-data-collection-rules skill
+# Tests skill routing (positive + negative) and output correctness
+
+name: azure-data-collection-rules-eval
+description: |
+  Integration evaluation for azure-data-collection-rules skill.
+  Tests skill routing for DCR authoring prompts and validates
+  output correctness (JSON structure, KQL transforms, Direct kind).
+
+tags:
+  type: integration
+  skill: azure-data-collection-rules
+
+environment:
+  skills:
+    - ../../plugin/skills/azure-data-collection-rules
+
+stimuli:
+  # ─── Routing Tests (skill invocation) ─────────────────────────────────────────
+  - name: "trigger-syslog-filter"
+    prompt: "Create a DCR to collect syslog and filter out info-level messages"
+    tags:
+      type: integration
+      tier: smoke
+      cost: low
+      area: routing
+      earlyTerminate: '[{"type":"skill-call","skill":"azure-data-collection-rules"}]'
+
+  - name: "trigger-direct-ingestion"
+    prompt: "Create a direct ingestion DCR for my custom application logs with a KQL transform"
+    tags:
+      type: integration
+      tier: smoke
+      cost: low
+      area: routing
+      earlyTerminate: '[{"type":"skill-call","skill":"azure-data-collection-rules"}]'
+
+  - name: "trigger-windows-events"
+    prompt: "Author a data collection rule that collects Windows Security events and drops EventID 4688"
+    tags:
+      type: integration
+      tier: smoke
+      cost: low
+      area: routing
+      earlyTerminate: '[{"type":"skill-call","skill":"azure-data-collection-rules"}]'
+
+  - name: "trigger-multi-stage"
+    prompt: "Help me create a multi-stage transformation DCR with processors for parsing JSON logs"
+    tags:
+      type: integration
+      tier: smoke
+      cost: low
+      area: routing
+      earlyTerminate: '[{"type":"skill-call","skill":"azure-data-collection-rules"}]'
+
+  - name: "trigger-negative-deploy"
+    prompt: "Deploy my Node.js web app to Azure App Service"
+    tags:
+      type: integration
+      tier: smoke
+      cost: low
+      area: routing
+      earlyTerminate: '[{"type":"tool-call-count","count":10}]'
+    graders:
+      - type: regex
+        config:
+          pattern: "(?!.*azure-data-collection-rules)"
+          target: skills_loaded
+
+  - name: "trigger-negative-general"
+    prompt: "What is the weather in Seattle today?"
+    tags:
+      type: integration
+      tier: smoke
+      cost: low
+      area: routing
+      earlyTerminate: '[{"type":"tool-call-count","count":5}]'
+    graders:
+      - type: regex
+        config:
+          pattern: "(?!.*azure-data-collection-rules)"
+          target: skills_loaded
+
+  # ─── Integration Tests (output correctness) ───────────────────────────────────
+  - name: "output-dcr-json-structure"
+    prompt: "Create a DCR that collects custom JSON logs from an application. The logs have fields: Time (datetime), Computer (string), Level (string), Message (string). Send them to a custom table called AppLogs_CL in my Log Analytics workspace."
+    tags:
+      type: integration
+      tier: regression
+      cost: medium
+      area: correctness
+      earlyTerminate: '[{"type":"tool-call-count","count":30}]'
+    graders:
+      - type: regex
+        config:
+          pattern: "streamDeclarations"
+          target: output
+      - type: regex
+        config:
+          pattern: "dataFlows"
+          target: output
+      - type: regex
+        config:
+          pattern: "destinations"
+          target: output
+      - type: regex
+        config:
+          pattern: "logAnalytics"
+          target: output
+
+  - name: "output-kql-transform"
+    prompt: "Author a DCR for direct ingestion that accepts logs with fields Timestamp, Host, App, Severity, Msg and transforms them with KQL to rename Timestamp to TimeGenerated and filter out Severity == 'Debug'"
+    tags:
+      type: integration
+      tier: regression
+      cost: medium
+      area: correctness
+      earlyTerminate: '[{"type":"tool-call-count","count":30}]'
+    graders:
+      - type: regex
+        config:
+          pattern: "transformKql"
+          target: output
+      - type: regex
+        config:
+          pattern: "TimeGenerated"
+          target: output
+      - type: regex
+        config:
+          pattern: "where.*Severity|where.*Debug"
+          target: output
+
+  - name: "output-direct-ingestion-kind"
+    prompt: "Create a direct ingestion DCR. My app sends JSON with fields RequestId, Duration, StatusCode, Endpoint. Route to CustomApiLogs_CL table."
+    tags:
+      type: integration
+      tier: regression
+      cost: medium
+      area: correctness
+      earlyTerminate: '[{"type":"tool-call-count","count":30}]'
+    graders:
+      - type: regex
+        config:
+          pattern: '"kind"\\s*:\\s*"Direct"'
+          target: output
+      - type: regex
+        config:
+          pattern: "Custom-"
+          target: output
+      - type: regex
+        config:
+          pattern: "CustomApiLogs_CL|Custom.*ApiLogs"
+          target: output

evals/azure-data-collection-rules/fixtures/sample-logs.json

@@ -0,0 +1,29 @@
+[
+  {
+    "Time": "2026-06-25T10:00:00Z",
+    "Computer": "web-server-01",
+    "Application": "OrderService",
+    "Level": "Error",
+    "Message": "Connection timeout to database after 30s",
+    "RequestId": "req-abc-123",
+    "DurationMs": 30012.5
+  },
+  {
+    "Time": "2026-06-25T10:00:05Z",
+    "Computer": "web-server-02",
+    "Application": "OrderService",
+    "Level": "Warning",
+    "Message": "Retry attempt 2 for payment processing",
+    "RequestId": "req-def-456",
+    "DurationMs": 5023.1
+  },
+  {
+    "Time": "2026-06-25T10:00:10Z",
+    "Computer": "web-server-01",
+    "Application": "AuthService",
+    "Level": "Info",
+    "Message": "User login successful",
+    "RequestId": "req-ghi-789",
+    "DurationMs": 142.3
+  }
+]

plugin/skills/azure-data-collection-rules/README.md

@@ -0,0 +1,66 @@
+# Azure Data Collection Rules Skill
+
+A Copilot agent skill for authoring, validating, and deploying Azure Monitor Data Collection Rules (DCRs).
+
+## Prerequisites
+
+- **VS Code** with GitHub Copilot Chat extension
+- **Azure CLI** (`az`) or **Azure PowerShell** (`Az.Accounts` module), authenticated
+- **Permissions**: Contributor or Monitoring Contributor role on the target resource group
+- For multi-stage transformations: API version `2025-05-11` support (enabled during preview)
+
+## Usage
+
+The skill triggers automatically when you ask about DCR authoring, data collection rules, KQL transforms, custom tables, or multi-stage transformations.
+
+### Example prompts
+- "Create a DCR that collects syslog auth events and filters by severity"
+- "Add a JSON parsing transform to extract user and action fields from custom text logs"
+- "Help me design a multi-stage DCR with client-side aggregation and ingestion-time KQL"
+- "Create a custom Log Analytics table for my aggregated performance data"
+- "Create a direct ingestion DCR so my app can send custom logs via the Log Ingestion API"
+- "Generate a script to send JSON data to Azure Monitor"
+
+## Contents
+
+```
+azure-data-collection-rules/
+├── SKILL.md                    # Entry point, quick reference, procedure overview
+├── version.json                # Version metadata for CI
+├── README.md                   # This file
+├── references/
+│   ├── procedure.md                        # Full step-by-step authoring workflow
+│   ├── dcr-kinds.md                        # Kind selection guide
+│   ├── dcr-schema.md                       # DCR JSON structure and REST API
+│   ├── stream-declarations.md              # Custom stream schema reference
+│   ├── processors-headers.md               # Header processor types and output columns
+│   ├── processors-operations.md            # Filter, map, parse, aggregate, enrich operations
+│   ├── processor-heuristics-filters.md     # Native filter intent mapping
+│   ├── processor-heuristics-transforms.md  # Transform intent mapping
+│   ├── processor-heuristics-staging.md     # Stage placement and cost optimization
+│   ├── destination-routing.md              # Stream-to-table routing rules
+│   ├── supported-tables.json               # Standard tables accepting custom streams
+│   ├── supported-tables.md                 # Supported tables documentation
+│   ├── kql-transforms.md                   # KQL patterns for ingestion-time transforms
+│   ├── la-tables.md                        # Custom table creation and management
+│   ├── direct-ingestion.md                 # Log Ingestion API (direct DCRs, auth)
+│   ├── decision-guide.md                   # Scenario-to-approach routing table
+│   └── limits.md                           # DCR structure limits, column constraints
+├── scripts/
+│   ├── get-dcr.ps1 / get-dcr.sh            # Retrieve existing DCR
+│   ├── put-dcr.ps1 / put-dcr.sh            # Create/update DCR
+│   ├── validate-dcr.ps1 / validate-dcr.sh  # Validate DCR JSON before deployment
+│   ├── get-table-schema.ps1 / .sh          # Get LA table columns
+│   ├── create-custom-table.ps1 / .sh       # Create/update custom table
+│   └── send-logs.ps1 / send-logs.sh        # Send data via Log Ingestion API
+└── examples/
+    ├── syslog-filter-drop.json             # Syslog client-side filter + column drop
+    ├── custom-json-log.json                # JSON log parsing with ingestion KQL
+    ├── perf-counter-aggregation.json       # Performance counter aggregation
+    ├── windows-events-split.json           # Windows events split to multiple tables
+    └── direct-ingestion-custom-table.json  # Direct ingestion via Log Ingestion API
+```
+
+## No External Dependencies
+
+This skill is fully self-contained. All reference material, scripts, and examples are included in the skill folder.

plugin/skills/azure-data-collection-rules/SKILL.md

@@ -0,0 +1,88 @@
+---
+name: azure-data-collection-rules
+description: "Author, edit, validate, and deploy Azure Monitor Data Collection Rules (DCRs), Log Analytics workspace tables, and KQL ingestion-time transformations. Covers single-stage and multi-stage transformation DCRs, client-side and ingestion-side processors, stream declarations, and custom table creation. Also covers direct ingestion DCRs for the Log Ingestion API. WHEN: create DCR, edit DCR, data collection rule, DCR JSON, add transformation, KQL transform, custom table, stream declaration, multi-stage transformation, processor, client-side transform, ingestion-time transform, parse JSON logs, filter syslog, aggregate events, custom log table, DCR schema, DCR authoring, rename columns, drop columns, CEF parsing, XML parsing, data collection, Log Ingestion API, direct ingestion, send custom logs, custom log ingestion, logs ingestion endpoint, DCR endpoint."
+argument-hint: "Describe the data source type, desired transformations, and destination table"
+license: MIT
+metadata:
+  author: Microsoft
+  version: "0.0.0-placeholder"
+---
+
+# Azure Data Collection Rules Skill
+
+Author, validate, and deploy Azure Monitor Data Collection Rules with single-stage, multi-stage, or direct ingestion configurations.
+
+## Quick Reference
+
+| Property | Value |
+|----------|-------|
+| Skill name | `azure-data-collection-rules` |
+| Scope | DCR authoring, validation, deployment, and Log Ingestion API |
+| API version | `2025-05-11` (multi-stage), `2023-03-11` (single-stage) |
+| Supported kinds | `Direct`, `Linux`, `Windows`, `WorkspaceTransforms` |
+| Scripts | PowerShell (`.ps1`) and Bash (`.sh`) |
+
+## When to Use This Skill
+
+- User wants to **create a new DCR** (agent-based or direct ingestion)
+- User wants to **edit an existing DCR** (add transforms, change routing, modify filters)
+- User asks about **KQL ingestion-time transformations** (filter, parse, project, extend)
+- User wants to **create a custom Log Analytics table** for ingestion
+- User asks about **stream declarations**, **destination routing**, or **dataFlow configuration**
+- User wants to **send custom logs** via Log Ingestion API
+- User asks about **multi-stage transformations** (client-side processors + ingestion-side KQL)
+- User wants to **validate a DCR** before deployment
+- User asks about **DCR limits**, column constraints, or structure constraints
+- User mentions **CEF parsing**, **XML parsing**, **JSON extraction**, or **syslog filtering** in DCR context
+
+## MCP Tools
+
+| Tool | Purpose |
+|------|---------|
+| `mcp_azure_mcp_ser_monitor` | Query Azure Monitor resources, DCRs, and diagnostics |
+| `mcp_azure_mcp_ser_subscription_list` | List available subscriptions |
+| `mcp_azure_mcp_ser_group_list` | List resource groups in subscription |
+| `mcp_azure_mcp_ser_group_resource_list` | List resources in a resource group |
+
+## Procedure
+
+Follow the [full procedure](./references/procedure.md):
+
+1. **Gather requirements** — ingestion method, data source, intent, destination, split/copy needs
+2. **Determine DCR kind** — per [DCR kinds guide](./references/dcr-kinds.md)
+3. **Design transformation pipeline** — native filters, processors, or `transformKql`
+4. **Author the DCR** — per [DCR schema](./references/dcr-schema.md) and [routing rules](./references/destination-routing.md)
+5. **Validate** — run [validate-dcr.ps1](./scripts/validate-dcr.ps1) or [validate-dcr.sh](./scripts/validate-dcr.sh)
+6. **Deploy** — run [put-dcr.ps1](./scripts/put-dcr.ps1) or [put-dcr.sh](./scripts/put-dcr.sh), prepare tables via [create-custom-table.ps1](./scripts/create-custom-table.ps1)
+7. **Verify** — query destination table, check `_LogOperation` for errors
+
+## Error Handling
+
+| Error | Cause | Remediation |
+|-------|-------|-------------|
+| `StreamDeclarationMissing` | Custom stream not declared in `streamDeclarations` | Add stream declaration for direct ingestion / logFiles DCRs |
+| `InvalidOutputStreamName` | `outputStream` doesn't match `Custom-*_CL` or `Microsoft-*` | Fix the outputStream to use correct prefix and table name |
+| `TransformKqlSyntaxError` | KQL parse error in `transformKql` | Check KQL syntax; ensure `source` is first; validate operator support |
+| `DestinationNotFound` | DataFlow references undefined destination name | Ensure destination `name` in `destinations` matches the reference |
+| `TableNotFound` | Custom table doesn't exist in workspace | Create the table first via `create-custom-table.ps1` or CLI |
+| `SchemaColumnMismatch` | Transform output doesn't match destination table schema | Ensure final `project` includes all required columns with correct types |
+| `TransformTooLong` | `transformKql` exceeds 15,360 characters | Simplify KQL or split into multiple dataFlows |
+
+## References
+
+- [Procedure](./references/procedure.md) — full step-by-step workflow
+- [DCR kinds](./references/dcr-kinds.md) — kind selection, data source types, transformation sections
+- [DCR schema](./references/dcr-schema.md) — top-level structure, column constraints, dataFlows, transformations, REST API
+- [Stream declarations](./references/stream-declarations.md) — custom stream schemas (Direct + logFiles only)
+- [Processors: headers](./references/processors-headers.md) — header processor types, stage availability, output columns
+- [Processors: operations](./references/processors-operations.md) — filter, map, parse, aggregate, enrich, KQL syntax
+- [Processor heuristics: filters](./references/processor-heuristics-filters.md) — native filter check, filtering intent map
+- [Processor heuristics: transforms](./references/processor-heuristics-transforms.md) — parsing, schema, aggregation, enrichment, routing intent maps
+- [Processor heuristics: staging](./references/processor-heuristics-staging.md) — stage placement, cost optimization, multi-processor chains
+- [Destination routing](./references/destination-routing.md) — stream-to-table mapping
+- [Supported tables](./references/supported-tables.md) — standard tables accepting custom streams
+- [KQL transforms](./references/kql-transforms.md) — common KQL patterns
+- [LA tables](./references/la-tables.md) — table creation, plans
+- [Direct ingestion](./references/direct-ingestion.md) — Log Ingestion API DCRs
+- [Decision guide](./references/decision-guide.md) — scenario routing table
+- [Limits](./references/limits.md) — DCR structure limits, column constraints, API quotas