Skip to content

ROMAN-4156: Add MSOS L4 detection efficiency catalog schema#838

Open
alexstuk wants to merge 6 commits into
spacetelescope:mainfrom
alexstuk:ROMAN-4156-detection-efficiency-catalog
Open

ROMAN-4156: Add MSOS L4 detection efficiency catalog schema#838
alexstuk wants to merge 6 commits into
spacetelescope:mainfrom
alexstuk:ROMAN-4156-detection-efficiency-catalog

Conversation

@alexstuk

Copy link
Copy Markdown

Resolves N/A (external ticket: ROMAN-4156)

Closes # N/A

This PR adds a new MSOS Level 4 Detection Efficiency catalog schema:

  • latest/SSC/MSOS/wfi_microlensing_detection_efficiency_catalog_level_4.yaml

To wire this schema into RAD packaging/validation, this PR also:

  • adds versioned schema symlink:
    • src/rad/resources/schemas/SSC/MSOS/wfi_microlensing_detection_efficiency_catalog_level_4-1.0.0.yaml
  • updates pyproject.toml:
    • adds the new schema path to asdf_schema_xfail_tests

Tasks

  • Update or add relevant rad tests.
  • Update relevant docstrings and / or docs/ page.
  • Does this PR change any schema files?
  • Schema changes were discussed at RAD Review Board meeting.
  • Does this PR change any API used downstream? (No; please add label no-changelog-entry-needed.)
  • Write news fragment(s) in changes/.

I do not have maintainer permissions in spacetelescope/rad; maintainers may need to apply labels and run org-side workflow steps.

@alexstuk alexstuk requested review from a team and WilliamJamieson as code owners March 17, 2026 22:41
@PaulHuwe PaulHuwe requested a review from ejoliet March 22, 2026 20:53
@codecov

codecov Bot commented Mar 22, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.03%. Comparing base (5e16410) to head (6aafb09).
⚠️ Report is 14 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #838      +/-   ##
==========================================
- Coverage   86.56%   86.03%   -0.54%     
==========================================
  Files          15       15              
  Lines        1154     1131      -23     
==========================================
- Hits          999      973      -26     
- Misses        155      158       +3     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@PaulHuwe

PaulHuwe commented Apr 6, 2026

Copy link
Copy Markdown
Collaborator

Is there an update to this ticket?

@ejoliet

ejoliet commented Apr 8, 2026

Copy link
Copy Markdown
Collaborator

SSC is under AT period - Jen will get back to this asap.

@PaulHuwe

Copy link
Copy Markdown
Collaborator

Is this intended to be in the intermediate release? If so, we need this done and fully approved by COB Thursday.

@cjarnold

Copy link
Copy Markdown
Collaborator

Is this intended to be in the intermediate release? If so, we need this done and fully approved by COB Thursday.

detection efficiency is a commissioning product, so no we are not trying to get this into the intermediate release.

alexstuk added 3 commits June 9, 2026 16:12
Add Detection Efficiency-specific RAD metadata for the Level 4 MSOS
Detection Efficiency Catalog.

The original PR only wired the product to the shared MSOS metadata blocks.
This update adds a dedicated MSOS Detection Efficiency keyword block with
archive-queryable fields for:

- inject_medium
- tp_count
- fp_count
- tn_count
- fn_count

The TP/FP/TN/FN values are represented as file-level row counts, matching
the expected final product behavior where BinaryClass remains a row-level
catalog column.

The Detection Efficiency catalog schema now references the shared MSOS
basic/program/settings metadata plus the new DE-specific keyword block.
The spatial region block was removed because the DE catalog does not have
a clear file-level sky footprint requirement, and similar MSOS L4 catalog
products do not all require s_region.
@alexstuk

alexstuk commented Jun 9, 2026

Copy link
Copy Markdown
Author

Updated to include DE-specific metadata for inject_medium and file-level TP/FP/TN/FN row counts, based on science clarification.

@PaulHuwe PaulHuwe requested review from cjarnold and jbrookens June 10, 2026 16:07
@jbrookens

Copy link
Copy Markdown
Collaborator

@kdupriestsci @bhayden53

RAD Archive Diff Report

Generated: 2026-06-11 19:07:46Z

Summary

  • Roman db fields to add: 5
  • Roman db fields to modify: 0
  • Roman db fields to remove: 0
  • Rita db fields to add: 0
  • Rita db fields to modify: 0
  • Rita db fields to remove: 0
  • Destinations with datatype changes: 0

Roman — db fields to add

|fn_count|SSC|WFIMSOSCatalog|fn_count|0||SSC.fn_count|
|fp_count|SSC|WFIMSOSCatalog|fp_count|0||SSC.fp_count|
|inject_medium|SSC|WFIMSOSCatalog|inject_medium|1||SSC.inject_medium|
|tn_count|SSC|WFIMSOSCatalog|tn_count|0||SSC.tn_count|
|tp_count|SSC|WFIMSOSCatalog|tp_count|0||SSC.tp_count|

Roman — db fields to modify

Roman — db fields to remove

Rita — db fields to add

Rita — db fields to modify

Rita — db fields to remove

Datatype changes by destination

@WilliamJamieson WilliamJamieson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a change log fragment, see

<details><summary>News fragment change types:</summary>
- `changes/<PR#>.feature.rst`: new feature
- `changes/<PR#>.bugfix.rst`: fixes an issue
- `changes/<PR#>.doc.rst`: documentation change
- `changes/<PR#>.removal.rst`: deprecation or removal of public API
- `changes/<PR#>.misc.rst`: infrastructure or miscellaneous change

@alexstuk

Copy link
Copy Markdown
Author

Thanks for the review. I added the changelog fragment for the detection efficiency schema addition.

@alexstuk alexstuk requested a review from WilliamJamieson June 23, 2026 19:35

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slight difference in naming of this file vs what we called it in the interface
wfi_microlensing_detection_efficiency_catalog_level_4 (this file)
wfi_microlensing_detection_efficiency_product_level_4 (interface)

Nothing will break, so not a huge deal. If "catalog" is truly better than "product" we can migrate the interface naming in the future, but not high priority.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants