feat(flux): support omit_in_job attribute on submit args to drop args invalid inside a job

[jasonb5]

Summary

• Adds an omit_in_job boolean attribute to <arg> and <argument> submit args in config_batch.xsd; args marked omit_in_job="true" are dropped when submission happens from inside an already-running batch job.
• Adds EnvBatch.is_in_batch_job(), detecting in-job context via a small static scheduler→env-var mapping (FLUX_JOB_ID, SLURM_JOB_ID, PBS_JOBID, LSB_JOBID, COBALT_JOBID). Only the current batch system's env var is considered.
• Motivation: on Tuolumne, flux gives each job an ephemeral nested instance with no partitions, so a stale -p <partition> fails on in-job resubmit. The mechanism is generic/attribute-driven so other args can opt in without code changes.

Testing

• New pytest unit tests in CIME/tests/test_unit_xml_env_batch.py covering in-job, not-in-job, other-scheduler env var present (ignored), and unknown batch system cases; full file passes (29 tests).

Follow-up

• E3SM model config: add omit_in_job="true" to -p for Tuolumne's flux config_batch.xml.
• End-to-end validation on Tuolumne with RESUBMIT>=1.

Closes #5018

[jasonb5]

Needs testing on actual machine still.

[Copilot]

Pull request overview

This PR adds an omit_in_job attribute for batch submit arguments so args that are only valid when submitting outside an existing batch job can be automatically dropped when submissions occur from inside a running job (e.g., Flux nested instances where stale partition args like -p <partition> fail). It extends EnvBatch with an in-job detection helper based on scheduler-specific environment variables and adds unit tests for the new behavior.

Changes:

• Add omit_in_job (xs:boolean) to <arg> and <argument> submit args in config_batch.xsd.
• Add EnvBatch.is_in_batch_job() and apply omit_in_job filtering while resolving submit args.
• Add pytest unit tests covering omit/not-omit behavior based on in-job environment context.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
CIME/XML/env_batch.py	Adds in-job detection and omits submit args marked omit_in_job when running inside a batch job.
CIME/tests/test_unit_xml_env_batch.py	Adds unit tests for omit_in_job behavior and unknown scheduler handling.
CIME/data/config/xml_schemas/config_batch.xsd	Extends schema to allow omit_in_job attribute on submit argument elements.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.