[DISCUSSION] Representing time

[prockenschaub]

A core decision we will need to take is how we want to represent time. Generally, there have been two approaches in the literature :

1. Represent absolut time as a timestamp (e.g., MIMIC, HiRID, OMOP, MEDS, CLIF)
2. Represent time as a number of time units relative to an origin (e.g., eICU, AUMCdb¹, SICdb, ricu)

There are arguments for both and as we can see above, existing datasets, packages, and standards are split between them.

Proposal

I propose to go with absolute time instead of relative time. The main argument is that this aligns with how OMOP and MEDS — the from my point of view most important standards — are handling things, and I think we should do our best to stay as close to them as possible.

It also saves us from choosing a specific reference point (most commonly start of ICU admission) or even multiple reference points for the same patient (e.g., in the case of multiple ICU admissions of the same patient). The latter can lead to weird behaviour. For example, labs in ricu are assigned to the closest ICU stay, which can be a problem when trying to embed the entire previous patient history at the start of the second stay.

Additional considerations

No matter which time representation we choose, we will need to convert some datasets that natively use the representation we didn't choose. A second decision we also need to make is when we are doing the conversion. Options include

1. when importing the dataset (i.e., when we are converting it into parquet or whatever fast format we will be using)
2. when loading concepts from the imported dataset

ricu has opted for the second option. I am not sure I agree with that choice and I think there is an argument to be made for already standardising time representations when we are importing a dataset.

There is also a good chance that users, at some point of their pipeline, want to switch back to a relative representation of time. A common example is to bin data into hours since admission in order to use a conventional machine learning model. This functionality will have to be provided at some point of the pipeline, which may be seen as an argument against absolute timestamps.

Footnotes

1. At least in the legacy version <= 1.0.2. ↩

[aegis301]

I strongly vote for using absolute timestamps for everything we're doing. As @prockenschaub already mentioned we are following well established standards and working with relative timepoints can have weird effects IMO. Also I share the concern on arbitrarly setting a specific reference point in a patient's journey. This seems short sighted, especially when data becomes more complex in the future.

Converting data from datasets that opted for other design decisions should be farely simple by setting a fixed reference date. I don't have a strong opinion on the timepoint of conversion. Standardising it once on import should save computational time later and avoid repeatedly converting timestamps. On the other hand leaving the original data untouched and only convert it on load-time also can be a reasonable design decision. Again, no strong opinion here.

To the third point: I understand the concern on the possibility of converting back to relative time points later down the line. However I think this will be only in very specific scenarios and will be fairly simple as a computation of differences between time points doesn't require that much resources and should only happen once for a specific experiment. In my opinion, this should not deter us from making the better design choice.