fix: make Azure billing usage and forecast windows configurable

[stefans-elastic]

Proposed commit message

fix: make Azure billing usage and forecast windows configurable

The azure/billing metricset always queried a hardcoded window: exactly the
previous full UTC day for usage, and reference-2d..+30d for forecast. Azure
Cost Management data can change for up to 72 hours after a billing period
closes, but the metricset never re-queried a date it had already collected,
so late cost corrections were never ingested.

Add two duration config options to the shared azure.Config:

• billing_usage_lookback (default 24h) how far back to (re)query usage
• billing_forecast_window (default 720h) length of the forecast period

usageIntervalFrom and forecastIntervalFrom now take the window as a
parameter; defaults reproduce the previous behaviour exactly, so existing
deployments are unaffected. Operators can set billing_usage_lookback: 72h
to re-ingest Azure's late corrections.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

How to test this PR locally

Ran the azure billing metricset against a live Azure subscription with
billing_usage_lookback: 72h, publishing to a local Elasticsearch, and confirmed
that usage data is now collected for the whole lookback window instead of just the
previous day.

The metricset logs show the widened usage window (start moved back to -72h, end
unchanged at the previous full day):

"message":"Getting usage details for scope","billing.usage.start_time":"2026-07-19T00:00:00.000Z","billing.usage.end_time":"2026-07-21T23:59:59.000Z"

Verified in Elasticsearch that usage documents span three full days. The
azure.billing.usage_start filter isolates usage docs (excludes the forward-looking
forecast events), and a daily date_histogram on azure.billing.usage_date returns
one bucket per collected day:

Request / Response

Request:

POST metricbeat-*/_search
{
  "size": 0,
  "query": {
    "bool": {
      "filter": [
        { "term":  { "tags": "azure-billing-e2e" } },
        { "exists":{ "field": "azure.billing.usage_start" } }
      ]
    }
  },
  "aggs": {
    "days": {
      "date_histogram": {
        "field": "azure.billing.usage_date",
        "calendar_interval": "day",
        "format": "yyyy-MM-dd"
      }
    }
  }
}

Response:

{
  "took": 8,
  "timed_out": false,
  "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 },
  "hits": { "total": { "value": 839, "relation": "eq" }, "max_score": null, "hits": [] },
  "aggregations": {
    "days": {
      "buckets": [
        { "key_as_string": "2026-07-19", "key": 1784419200000, "doc_count": 277 },
        { "key_as_string": "2026-07-20", "key": 1784505600000, "doc_count": 289 },
        { "key_as_string": "2026-07-21", "key": 1784592000000, "doc_count": 273 }
      ]
    }
  }
}

Three usage_date buckets (2026-07-19, 2026-07-20, 2026-07-21) confirm the
72h window re-queried three full days. With the default 24h lookback the same
query returns a single bucket (2026-07-21), matching the previous behaviour.

A reviewer can reproduce this with a minimal config:

metricbeat.modules:
- module: azure
  metricsets: [billing]
  enabled: true
  period: 24h
  client_id: '${AZURE_CLIENT_ID}'
  client_secret: '${AZURE_CLIENT_SECRET}'
  tenant_id: '${AZURE_TENANT_ID}'
  subscription_id: '${AZURE_SUBSCRIPTION_ID}'
  billing_usage_lookback: 72h

Related issues

Use cases

Screenshots

Logs

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @stefans-elastic? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[github-actions]

🔍 Preview links for changed docs

• docs/reference/metricbeat/metricbeat-metricset-azure-billing.md
• docs/reference/metricbeat/metricbeat-module-azure.md
• x-pack/metricbeat/module/azure/billing/_meta/docs.md

[github-actions]

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!

---

The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

[stefans-elastic]

/test

[jakubgalecki0]

Function func (client *Client) GetMetrics sets following filter:

       filter := fmt.Sprintf(
		"properties/usageStart eq '%s' and properties/usageEnd eq '%s'",
		timeOpts.usageStart.Format(time.RFC3339Nano),
		timeOpts.usageEnd.Format(time.RFC3339Nano),
	)

	result, err := client.BillingService.GetUsageDetails(
		scope,
		"properties/meterDetails",
		filter,
		armconsumption.MetrictypeActualCostMetricType,
		timeOpts.usageStart.Format("2006-01-02"), // startDate
		timeOpts.usageEnd.Format("2006-01-02"),   // endDate
	)

Will this change work with that or should we use ge and le? We are passing start and end separately but it's worth checking out.

[stefans-elastic]

@jakubgalecki0 Verified this and changed the usage details $filter to use a range: properties/usageStart ge ... and properties/usageEnd le .... I kept startDate/endDate in the request for the modern account path and added TestClientUsesRangeFilterForUsageDetails to assert the generated filter plus request dates for a multi-day lookback. Pushed in 119be81.

[vishaangelova]

In which version will these config options become available?

If this will not be backported:

Suggested change

	`billing_usage_lookback`
	`billing_usage_lookback` {applies_to}`stack: ga 9.6+`

If it will be backported, then show the earliest version in which it will become available, e.g:

Suggested change

	`billing_usage_lookback`
	`billing_usage_lookback` {applies_to}`stack: ga 9.3+`

[vishaangelova]

Same here… If this will not be backported:

Suggested change

	`billing_forecast_window`
	`billing_forecast_window` {applies_to}`stack: ga 9.6+`

If it will be backported, then show the earliest version in which it will become available, e.g:

Suggested change

	`billing_forecast_window`
	`billing_forecast_window` {applies_to}`stack: ga 9.3+`