Pyro SAI: Tasks 1 & 2 pipeline (FWI P95 extremes, VPD)

#	notebook	what it does
0	00_Data_Discovery.ipynb	lists the Hub buckets, finds daily files, writes pyrosai_catalog.json
1	Task1_FWI_P95_Extremes.ipynb	daily FWI, P95 extremes, seasonality, significance, AR6 stats, driver attribution
2	Task2_VPD_Evaporative_Demand.ipynb	VPD, P95-VPD days, T/RH decomposition, risk-offset zones
–	Regrid_E3SM_daily.ipynb	(E3SM only) ne30pg2 to 1° regridding

pyrosai.py holds all shared code (data-quality safeguards included).

Your settings: local_config.py

Copy local_config_example.py to local_config.py and edit it (dry run vs full run, WORK_ROOT, models...). Task 1/2 apply it after their CONFIG cell, so updating the notebooks never resets your settings. Commit it.

Running and sharing results

nohup bash run_headless.sh Task1_FWI_P95_Extremes Task2_VPD_Evaporative_Demand > runs.log 2>&1 &
tail -3 runs/*/progress.txt            # live progress (one line per FWI member, metrics per model)
git add runs/ local_config.py && git commit -m "run outputs" && git push

Each run writes runs/<date>_<notebook>/ with summary.txt (all outputs and errors), log.txt, the executed notebook, figures and regional CSVs. Large outputs (*.zarr, *.nc, work folders) are git-ignored.

Hand-added catalog entries

pyrosai_catalog_extra.json is merged into the catalog by load_catalog (it never overwrites existing entries). It currently adds UKESM daily tasmax: the G6-1.5K-SAI and SSP2-4.5 r2 tas files hold daily max, min and mean temperature, picked by their cell_methods.

Tests

bash tests/run_tests.sh runs the whole chain on a synthetic archive (no cloud access needed).