Skip to content

feat(skills): add model verification cards - #3575

Draft
yaoyu-33 wants to merge 2 commits into
mainfrom
yaoyu-33/feat/model-verification-cards
Draft

feat(skills): add model verification cards#3575
yaoyu-33 wants to merge 2 commits into
mainfrom
yaoyu-33/feat/model-verification-cards

Conversation

@yaoyu-33

Copy link
Copy Markdown
Contributor

What does this PR do ?

Add an agent-readable model verification card workflow, validator, and the first AutoModel card for Qwen3-30B-A3B.

Changelog

  • Add the nemo-automodel-model-verification-card skill, schema reference, and validator.
  • Add a Qwen3-30B-A3B card beside its current benchmark examples using only verified and not_verified status buckets.
  • Record the completed 16x H100 RedPajama2 pretrain and checkpoint-resume evidence, including loss, step time, TFLOPS/GPU, and tokens/s/GPU.
  • Add the reproducible RP2 recipe with the Megatron Bridge convergence contract: GBS 1024, MBS 1, 100 steps, 40-step warmup, and cosine LR decay from 3e-4 to 3e-5.
  • Keep pretrain_performance separate and not verified; record the functional TFLOPS observation and canonical benchmark recipe for the future tuned run.
  • Define vllm_checkpoint_compatibility as the downstream checkpoint gate and define 128K CoderForge SFT under sft_long_context.
  • Exclude *_verification_card.yaml files from recipe-only lint, CI discovery, documentation coverage, and model-release scanning, with regression tests.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?

Verification

  • uv run pytest -q tests/unit_tests/skills/test_model_verification_card.py tests/unit_tests/config/test_example_yaml_linter.py tests/unit_tests/ci_tests/test_generate_ci_tests.py tests/unit_tests/_transformers/test_recipe_doc_coverage.py tests/unit_tests/tools/test_sync_model_coverage_tables.py — 51 passed, 1 skipped
  • Card validator passed for the Qwen3-30B-A3B card.
  • Skill structure validation passed.
  • Example YAML linter passed for all 508 YAML files.
  • ruff format --check . and ruff check . passed.

Additional Information

The verified evidence is bounded functional training and resume evidence. The functional AutoModel observation is 135.4928 model TFLOPS/GPU versus the referenced Megatron Bridge observation of 199.120 model TFLOPS/GPU (-31.9542%); it is intentionally not presented as a tuned performance result.

Signed-off-by: yaoyu-33 <yaoyu.094@gmail.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 18, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

github-actions Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

@akoumpa

akoumpa commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

/ok to test 62de5c4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants