Agent / integration: Datadog Agent 7.80.2, lustre check 1.5.0 (latest; checked CHANGELOG — no related fix)
Environment: Amazon FSx for Lustre (PERSISTENT_2) clients on Ubuntu 22.04 — lustre-client-utils 2.15.6-1fsx28 (lctl 2.15.6), SageMaker HyperPod fleet
What happens
With node_type: client, only lustre.device.health and lustre.net.* (lnetctl-based) are emitted. All stats-based metrics (llite / mdc / osc stats, curated params, enable_extra_params) are silently missing.
The check invokes lctl get_param -ny <param>, but -y (YAML output) was only added in Lustre 2.16. The lctl shipped with the current FSx for Lustre client (2.15.x) rejects it:
$ lctl get_param --help
usage: get_param [-F|n|-N|-R] <param_path1 param_path2 ...> # no -y
Agent debug log — every stats param fails with rc=4, logged at DEBUG only, check status stays [OK]:
(check.py:258) Running command: ['sudo', '/usr/sbin/lctl', 'get_param', '-ny', 'llite.<fs>-ffff....kbytesfree']
(check.py:263) Command [...] exited with returncode 4. Captured stderr: get_param: invalid option -- 'y'
Why it matters
FSx for Lustre is presumably the largest managed-Lustre install base, and its newest client packages are 2.15-based. For all these users the integration's client-side metrics (and most of the Lustre - Overview dashboard) are empty, with no visible error — we only found this by running datadog-agent check lustre -l debug during an incident investigation.
Suggested fix
- Detect
-y support once at init (or on first rc=4) and fall back to lctl get_param -n with plain-text parsing, or
- At minimum, surface stats-collection failures as a check WARNING / service check instead of DEBUG, so the data gap is visible.
Repro: any FSx for Lustre client with lustre-client-utils 2.15.x → enable lustre check with node_type: client → datadog-agent check lustre -l debug.
Agent / integration: Datadog Agent 7.80.2, lustre check 1.5.0 (latest; checked CHANGELOG — no related fix)
Environment: Amazon FSx for Lustre (PERSISTENT_2) clients on Ubuntu 22.04 —
lustre-client-utils 2.15.6-1fsx28(lctl 2.15.6), SageMaker HyperPod fleetWhat happens
With
node_type: client, onlylustre.device.healthandlustre.net.*(lnetctl-based) are emitted. All stats-based metrics (llite / mdc / osc stats, curated params,enable_extra_params) are silently missing.The check invokes
lctl get_param -ny <param>, but-y(YAML output) was only added in Lustre 2.16. The lctl shipped with the current FSx for Lustre client (2.15.x) rejects it:Agent debug log — every stats param fails with rc=4, logged at DEBUG only, check status stays [OK]:
Why it matters
FSx for Lustre is presumably the largest managed-Lustre install base, and its newest client packages are 2.15-based. For all these users the integration's client-side metrics (and most of the Lustre - Overview dashboard) are empty, with no visible error — we only found this by running
datadog-agent check lustre -l debugduring an incident investigation.Suggested fix
-ysupport once at init (or on first rc=4) and fall back tolctl get_param -nwith plain-text parsing, orRepro: any FSx for Lustre client with lustre-client-utils 2.15.x → enable lustre check with
node_type: client→datadog-agent check lustre -l debug.