Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions modules/nf-core/custom/orfnormalise/meta.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,11 @@ description: |
`orf_type_native` carries the caller's own ORF-type label verbatim, so every
harmonisation decision below stays auditable without re-running the caller.
ORF-type tokens are matched exactly (casefolded) against each caller's closed
vocabulary. Tokens matching no entry cause the process to fail; successful
outputs report `unmapped_orf_type=0` on the `# parser_columns:` provenance line.
vocabulary. Where a caller qualifies a positional label after a colon, as
Ribo-TISH does with `Novel:CDSFrameOverlap`, only the part before the colon is
matched and the qualifier survives in `orf_type_native`. Tokens whose location
matches no entry cause the process to fail; successful outputs report
`unmapped_orf_type=0` on the `# parser_columns:` provenance line.

`orf_class` is purely positional: it records where the ORF sits relative to
the annotated CDS and never encodes its length. Select small ORFs with the
Expand Down
19 changes: 15 additions & 4 deletions modules/nf-core/custom/orfnormalise/templates/orfnormalise.py
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,9 @@ def write_outputs(bed_path, tsv_path, bed_lines, tsv_rows, parser_columns, unmap
# Every caller's ORF-type vocabulary is a closed enum, so tokens are matched
# exactly (casefolded) rather than by substring. Substring matching mis-fired
# on the overlap forms, because "uorf" is a substring of "overlap_uorf".
#
# Keys hold the positional part only. Ribo-TISH qualifies it after a colon
# (`Novel:CDSFrameOverlap`); classify() splits there, so no key may contain one.
# ----------------------------------------------------------------------------

CLASS_TOKENS = {
Expand Down Expand Up @@ -498,13 +501,21 @@ def write_outputs(bed_path, tsv_path, bed_lines, tsv_rows, parser_columns, unmap
def classify(caller, orf_type):
"""Map a caller's native ORF-type token to the harmonised class.

Returns (orf_class, matched); `matched` is False when a non-empty token
matched no entry, so unmapped labels can be counted rather than silently
absorbed into `other`.
Callers may qualify a positional label rather than replace it: Ribo-TISH
emits `Novel:CDSFrameOverlap`, `3'UTR:CDSFrameOverlap` and similar. Only the
part before the first colon describes where the ORF sits, and `orf_class`
is positional, so the qualifier is dropped here and preserved verbatim in
`orf_type_native`. No CLASS_TOKENS key contains a colon, so this cannot
shorten a token that was meant to match whole.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we reduce the AI historical narrative in the comments? I get this problem a lot, but comments should be stand-alone and represent the current state, not reference a previous state you're fixing.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trimmed in 45d5122 — comment-only across the three files, and no executable change: orfnormalise.py's token stream is identical apart from the docstring literal itself.

  • classify() docstring: the paragraph describing what the previous code did is gone. It now states only that the part before the first colon is matched and that the full token survives in orf_type_native.
  • CLASS_TOKENS header: the Ribo-TISH narrative is replaced by the single constraint a future editor of that table actually needs — No key may contain a colon: classify() matches only the part before one. That invariant is load-bearing (a colon-bearing key would be silently truncated by the split), so I kept it, but moved it onto the table it constrains rather than leaving it as a justification in the docstring.
  • meta.yml and the new test assertion: same treatment.

One thing I left alone: there is a pre-existing instance of exactly this pattern a few lines above, from #12498Substring matching mis-fired on the overlap forms, because "uorf" is a substring of "overlap_uorf". Happy to fold that in here if you'd like it gone now; I left it out so this PR stays scoped to comments it introduced itself.


Returns (orf_class, matched); `matched` is False when a non-empty token's
location matched no entry, so unmapped labels can be counted rather than
silently absorbed into `other`.
"""
if not orf_type:
return "other", True
cls = CLASS_TOKENS[caller].get(orf_type.strip().casefold())
location = orf_type.split(":", 1)[0]
cls = CLASS_TOKENS[caller].get(location.strip().casefold())
if cls is None:
return "other", False
return cls, True
Expand Down
47 changes: 47 additions & 0 deletions modules/nf-core/custom/orfnormalise/tests/main.nf.test
Original file line number Diff line number Diff line change
Expand Up @@ -431,4 +431,51 @@ nextflow_process {
}
}

test("ribotish composite ORF types map on the location, keeping the qualifier native") {

when {
process {
"""
input[0] = channel
.of(
'Tid\tGid\tGenomePos\tTisType\tAALen\tFisherPvalue',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a real input we could use as a proper fixture in test-datasets? The existing one (genomics/homo_sapiens/riboseq_expression/orf_predictions/sample1.ribotish.pred.txt) obviously isn't representative enough, so we should consider adding another one rather than embedding dummy data here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes — real output with these tokens exists, so this needn't stay synthetic. The composite labels are what made a pipeline-level snapshot-recording run fail on genuine chr20 Ribo-TISH output, with this distribution:

Novel:CDSFrameOverlap     29
Novel:Known                4
3'UTR:CDSFrameOverlap      1
Internal:CDSFrameOverlap   1

That is also why the module tests missed the gap — the existing fixture only contains bare tokens, exactly as you say.

Two choices I would rather you made before I open the test-datasets PR, since both affect other modules:

  1. A second file, or make the existing one representative? My instinct is to add a new file (sample1.ribotish.composite.pred.txt or similar) rather than modify sample1.ribotish.pred.txt in place. Editing a shared fixture is what briefly broke custom/orfcollapse when Refresh the custom/orfcollapse fixture for the positional orf_class vocabulary test-datasets#2193 merged — there is no version gate, so every consumer picks the change up immediately and only notices when something next touches it. But if you would rather the existing fixture simply became representative, that is fine too and I will sequence the module PRs behind it.
  2. Region. The existing sample1.* family is chr20, and real composite tokens are plentiful there, so the family can stay on one region unless you want otherwise.

Once you have picked, the order is test-datasets first, then this PR swaps the inline rows for the fixture. Happy to do both.

@pinin4fjords pinin4fjords Aug 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try not to overwrite files in test-datasets, they should really be write-once (though it's patchily followed). Add a new one.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added as a new file in nf-core/test-datasets#2208, generated by running the pipeline on its own chr20 data in extended-ORF mode. Worth flagging: real output has seven colon-qualified forms, not the four we knew about -- 5'UTR:Known, 5'UTR:CDSFrameOverlap and Truncated:Known also occur, and all seven already resolve through the existing table. I will point the test at the fixture and drop the inline rows once that merges.

't1\tg1\tchr1:1000-1100:+\tNovel:CDSFrameOverlap\t30\t0.001',
"t1\tg1\tchr1:1100-1200:+\t3'UTR:CDSFrameOverlap\t30\t0.001",
't1\tg1\tchr1:1200-1300:+\tInternal:CDSFrameOverlap\t30\t0.001',
't1\tg1\tchr1:1300-1400:+\tNovel:Known\t30\t0.001'
)
.collectFile(name: 'composite.ribotish.pred.txt', newLine: true, sort: false)
.map { orfs -> [[id: 'sample1'], orfs, 'ribotish'] }
input[1] = channel
.of('chr1\ttest\texon\t1001\t2000\t.\t+\t.\tgene_id "g1"; transcript_id "t1";')
.collectFile(name: 'composite.gtf', newLine: true)
.map { gtf -> [[id: 'reference'], gtf] }
"""
}
}

then {
def lines = path(process.out.tsv[0][1]).text.readLines()
def header = lines.find { it.startsWith('orf_id') }.split('\t') as List
def data = lines.findAll { !it.startsWith('#') && !it.startsWith('orf_id') }
def i_class = header.indexOf('orf_class')
def i_native = header.indexOf('orf_type_native')
def classes = data.collect { it.split('\t')[i_class] } as Set
def natives = data.collect { it.split('\t')[i_native] }
def qualified = natives.findAll { it.contains(':') }
def provenance = lines.find { it.startsWith('# parser_columns:') }

assertAll(
// The colon-qualified tokens no longer abort the process: only the part
// before the colon is positional, so it alone selects the class.
{ assert process.success },
{ assert classes == ['novel_u', 'dORF', 'intORF'] as Set },
// The qualifier is dropped from orf_class but not lost.
{ assert qualified.size() == natives.size() },
{ assert natives.contains('Novel:CDSFrameOverlap') },
{ assert provenance.contains('unmapped_orf_type=0') }
)
}
}

}