Skip to content

[DevOps]: Add guidelines for specific material to ingest from performance archive directory #259

Description

@tomvothecoder

Task Details

Based on our 07/14/26 SimBoard meeting, we've decided to ingest this material from the archive directory:

  1. All material >= 2025-01
  2. All v3 material, which dates back to 2024.
  • We can refer to this link for the specific case names to ingest on NERSC under the "Simulation" column.

Steps to Complete

  1. Add default minimum for ARCHIVE_YEAR_START to "2025-01"
  2. Add a one-off script that ingests specifically v3 material on NERSC, using the case names under https://docs.e3sm.org/e3sm_data_docs/_build/html/v3/CoupledSystem/simulation_data/simulation_table.html.
    • It should provide a summary of matching case names to case directories, what is missing, complete executions, rejected incomplete executions, rejected invalid executions. There should be a dry-run mode too.
  3. Update documentation
    • Update NERSC Spin runbook with ARCHIVE_YEAR_START for the archive ingestor cron job
    • Update documentation to the specific material being ingested.

Additional Notes

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: devopsDevOps task (e.g., DOE site ingestion, CI/CD, Docker)

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions