Skip to content

Add Auto header detection to Populate_Metadata.py - #232

Open
muhanadz wants to merge 3 commits into
ome:developfrom
muhanadz:auto_header
Open

Add Auto header detection to Populate_Metadata.py#232
muhanadz wants to merge 3 commits into
ome:developfrom
muhanadz:auto_header

Conversation

@muhanadz

@muhanadz muhanadz commented Jul 17, 2026

Copy link
Copy Markdown
Member

I noticed that much of the work ome/omero-metadata#67 to create an auto header detecting method and allow NaNs was not propagated to the Populate_Metadata.py script.

This PR proposes adding this functionality such that the header type is automatically detected and "allow_nan" is an option.

To test, use a working CSV without header type defined such as:

image,dataset,ROI_Area,Channel_Index,Channel_Name
3659,1451,0.0469,1,DAPI
3686,1451,0.142,2,GFP-AuroraB
3746,1451,0.093,3,TRITC-edit

Without PR, all the columns should be StringColumn. With PR:

"ImageColumn",
"DatasetColumn",
"DoubleColumn",
"LongColumn",
"StringColumn",
"StringColumn"

if a #header row is defined in the CSV, it should always take precedence over the auto-detection.

@muhanadz
muhanadz marked this pull request as draft July 17, 2026 11:02
@muhanadz
muhanadz marked this pull request as ready for review July 17, 2026 20:31

@sbesson sbesson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern with the proposed implementation is the brittle nature of using nested imports to try and bisect the version of the omero-metadata package.

With the modern Python requirements (Python 3.10+) of the OMERO environment, we should be able to use standard libraries such as importlib.metadata for introspecting the presence and the version of the omero-metadata package and packaging for checking the minimal version

>>> from importlib.metadata import version
>>> from packaging.version import Version
>>> version('omero_metadata')
'0.13.0'
>>> Version(version('omero_metadata')) >= Version('0.11')
True
>>> Version(version('omero_metadata')) >= Version('0.14')
False
>>> 

A second question is: what should happen if the minimal requirements are not met. This PR maintains the current statu quo of supporting multiple paths (no omero-metadata, omero-metadata<0.11 and omero-metadata>=0.11).
An alternative thought especially in the context of the upcoming OMERO.server 5.7.0 milestone would be to put a hard requirement of this script on omero-metadata 0.11+ being installed server-side and make sure this is documented appropriately

try:
# Hopefully this will import
# https://github.com/ome/omero-metadata/blob/v0.3.1/src/populate_metadata.py
# Automatic header detection requires omero-metadata >= 0.11.0.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the modern Python requirements (Python 3.10+), I wonder if we want to start using importlib.metadata for introspecting the package version and packaging for checking a minimal version

>>> from importlib.metadata import version
>>> from packaging.version import Version
>>> version('omero_metadata')
'0.13.0'
>>> Version(version('omero_metadata')) >= Version('0.11')
True
>>> Version(version('omero_metadata')) >= Version('0.14')
False
>>> 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants