Make orf_class purely positional and add uoORF, doORF and intORF

CHANGELOG.md

@@ -30,6 +30,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - [#166](https://github.com/nf-core/riboseq/issues/166) - Add per-ORF in-frame P-site quantification, emitting an ORF x sample count matrix ([@pinin4fjords](https://github.com/pinin4fjords))
 - [#168](https://github.com/nf-core/riboseq/issues/168) - Add ORF-level differential translation analysis (anota2seq / deltaTE / DOTSeq) on top of the gene-level DTE ([@pinin4fjords](https://github.com/pinin4fjords))
 - [#146](https://github.com/nf-core/riboseq/issues/146), [#149](https://github.com/nf-core/riboseq/issues/149) - Add per-sample UMI handling through an optional `with_umi` samplesheet column ([@pinin4fjords](https://github.com/pinin4fjords))
+- [#226](https://github.com/nf-core/riboseq/pull/226) - Add `--smorf_max_aa` (default 100) controlling the catalogue's `is_smorf` flag and which ORFs are eligible for the peptide-level collapse. It never affects `orf_class` ([@FelixKrueger](https://github.com/FelixKrueger))
+- [#226](https://github.com/nf-core/riboseq/pull/226) - Add `uoORF`, `doORF` and `intORF` to the `orf_class` vocabulary, recovering the CDS-overlap and internal distinctions that RiboCode, ribotricer, Ribo-TISH and PRICE already report ([@FelixKrueger](https://github.com/FelixKrueger))
+- [#226](https://github.com/nf-core/riboseq/pull/226) - Add `is_smorf` and `orf_type_native` columns to the normalised and catalogue tables. `orf_type_native` carries each caller's own ORF-type label, so every harmonisation decision is auditable without re-running callers ([@FelixKrueger](https://github.com/FelixKrueger))
 
 ### `Fixed`
 
@@ -81,6 +84,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - [#197](https://github.com/nf-core/riboseq/issues/197) - Honour `--fastp_merge` and `--save_merged_fastq`, which were overridden by hardcoded `false` values in the preprocessing subworkflow call ([@pinin4fjords](https://github.com/pinin4fjords))
 - [#197](https://github.com/nf-core/riboseq/issues/197) - Make `--pseudo_aligner` select the tool used by `--te_quantification_method pseudo`, which always ran Salmon regardless of the setting. kallisto is now a working alternative, with `--kallisto_index`, `--kallisto_quant_fraglen`, `--kallisto_quant_fraglen_sd` and `--extra_kallisto_quant_args`. TE pseudo-alignment outputs move to `quantification/<pseudo_aligner>_te_pseudo`, unchanged for the default `salmon` ([@pinin4fjords](https://github.com/pinin4fjords))
 - [#205](https://github.com/nf-core/riboseq/pull/205) - Template update for nf-core/tools v4.0.3 ([@nf-core-bot](https://github.com/nf-core-bot), [@pinin4fjords](https://github.com/pinin4fjords))
+- [#226](https://github.com/nf-core/riboseq/pull/226) - **Breaking:** `orf_class` is now purely positional and no longer encodes ORF length. The `smORF` value is gone from every catalogue output; a short ORF keeps its positional class (`uORF`, `dORF`, `canonical_cds`, …) and is flagged by the new `is_smorf` column instead. Migrate a filter on `orf_class == "smORF"` to `is_smorf == "1"` (equivalently `0 < aa_length <= 100`, the default `--smorf_max_aa`). The peptide-collapse scope is unchanged at the default threshold ([@FelixKrueger](https://github.com/FelixKrueger))
+- [#226](https://github.com/nf-core/riboseq/pull/226) - **Breaking:** `orf_class` values change for ORFs the callers place relative to the CDS. RiboCode's and ribotricer's `Overlap_uORF`/`Overlap_dORF` were being folded into `uORF`/`dORF` by substring matching and now map to `uoORF`/`doORF`; RiboCode's and Ribo-TISH's `internal` and PRICE's `iORF` now map to `intORF` rather than `other`; PRICE's `uoORF` maps to `uoORF` rather than `uORF` and its `orphan` to `novel_u` rather than `other`. ribotricer's `internal` stays `other` because it is that tool's fall-through rather than a frame-tested call. To keep a pre-existing filter's meaning, read the old `uORF` as the new `uORF` ∪ `uoORF`, the old `dORF` as `dORF` ∪ `doORF`, and note that the old `other` no longer holds the internal ORFs. Anything consuming `orf_class` — including the ORF-level DOTSeq analysis, which matches `uORF`/`dORF` literally — sees the new vocabulary ([@FelixKrueger](https://github.com/FelixKrueger))
+- [#226](https://github.com/nf-core/riboseq/pull/226) - Catalogue row counts change in both directions. Down: ORFs whose callers disagreed on class now merge instead of producing one row per caller, and an ORF two callers size either side of the small-ORF threshold now merges. Up: a short truncated CDS variant is no longer folded into its transcript's full-length CDS, and short transcript-anchored ORFs are keyed on their exact span, so two callers whose bounds differ by a few nucleotides now yield one row each rather than one shared row. That last effect lowers `called_by_*` and `n_samples` for those ORFs, so a stricter `--orf_min_callers` drops more of them ([@FelixKrueger](https://github.com/FelixKrueger))
+- [#226](https://github.com/nf-core/riboseq/pull/226) - Cross-caller clustering now measures reciprocal overlap on summed exon-block intersection rather than on the outer genomic span, which for a spliced ORF is mostly intron, and uses complete linkage so a chain of partial overlaps cannot fold distinct ORFs into one row ([@FelixKrueger](https://github.com/FelixKrueger))
 
 ### `Parameters`
 
@@ -133,6 +140,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 |                           | `--extra_orf_deltate_args`               |
 |                           | `--extra_orf_anota2seq_run_args`         |
 |                           | `--extra_dotseq_args`                    |
+|                           | `--smorf_max_aa`                         |
 
 ### `Dependencies`
 

conf/modules.config

@@ -1098,6 +1098,7 @@ process {
     }
 
     withName: '.*:ORFTABLE_FASTA_GTF_BUILDORFCATALOGUE:CUSTOM_ORFNORMALISE' {
+        ext.args   = { "--smorf-max-aa ${params.smorf_max_aa}" }
         publishDir = [
             path: { "${params.outdir}/orf_predictions/catalogue/normalised" },
             mode: params.publish_dir_mode,
@@ -1126,7 +1127,9 @@ process {
         publishDir = [ enabled: false ]
     }
     withName: '.*:ORFTABLE_FASTA_GTF_BUILDORFCATALOGUE:CUSTOM_ORFCOLLAPSE' {
-        ext.args   = { "--min-callers ${params.orf_min_callers} --min-samples ${params.orf_min_samples}" }
+        // smorf_max_aa must match CUSTOM_ORFNORMALISE's, or the published
+        // is_smorf flag and the collapse scope describe different ORF sets.
+        ext.args   = { "--min-callers ${params.orf_min_callers} --min-samples ${params.orf_min_samples} --smorf-max-aa ${params.smorf_max_aa}" }
         publishDir = [
             [
                 path: { "${params.outdir}/orf_predictions/catalogue" },

docs/output.md

@@ -490,10 +490,10 @@ Produced only when `--run_price true` is set. PRICE is invoked once across the r
 
 - `orf_predictions/catalogue/normalised/`
   - `*.normalised.bed12`: per-sample, per-caller BED12 of called ORFs in genomic coordinates (multi-exon blocks for spliced ORFs).
-  - `*.normalised.tsv`: matching sidecar with caller, sample id, ORF class (`canonical_cds`, `uORF`, `dORF`, `novel_u`, `smORF`, `other`), amino-acid length and the caller's score.
+  - `*.normalised.tsv`: matching sidecar with caller, sample id, ORF class (`canonical_cds`, `uORF`, `uoORF`, `dORF`, `doORF`, `intORF`, `novel_u`, `other`), amino-acid length, the caller's score, the caller's own label in `orf_type_native`, and the `is_smorf` length flag.
 - `orf_predictions/catalogue/`
   - `*.catalogue.bed12`: merged, deduplicated ORF catalogue (BED12, stable `orf_NNNNNNNN` ids in column 4).
-  - `*.catalogue.tsv`: per-ORF table with `called_by_<caller>` binary columns for every caller in the runtime-enabled set and `score_<caller>` columns for Ribo-TISH / RiboCode / Rp-Bp / PRICE (Ribotricer scores are excluded). Includes `orf_class`, `aa_length`, host `gene_id` and `transcript_id`.
+  - `*.catalogue.tsv`: per-ORF table with `called_by_<caller>` binary columns for every caller in the runtime-enabled set and `score_<caller>` columns for Ribo-TISH / RiboCode / Rp-Bp / PRICE (Ribotricer scores are excluded). Includes `orf_class`, `aa_length`, `is_smorf`, host `gene_id` and `transcript_id`, plus `orf_type_native` listing every caller-native label in the cluster.
   - `*.orf_to_gene.tsv`: ORF-to-gene mapping. An ORF that maps to multiple host transcripts/genes gets multiple rows here; downstream gene-level aggregation collapses by `gene_id`.
   - `*.catalogue.aa.fasta`: amino-acid FASTA of every catalogue ORF, keyed by `orf_NNNNNNNN`. Produced by `bedtools getfasta` on the merged BED12 followed by `seqkit translate`.
   - `*.catalogue.mqc.tsv`: per-class ORF counts surfaced as a MultiQC custom-content table.
@@ -502,13 +502,16 @@ Produced only when `--run_price true` is set. PRICE is invoked once across the r
 
 </details>
 
-Produced only when `--extended_orf_analysis true` is set and at least one ORF caller is enabled. The catalogue runs once per pipeline invocation (cohort-level, not per sample) and merges per-sample per-caller calls with a class-aware strategy:
+Produced only when `--extended_orf_analysis true` is set and at least one ORF caller is enabled. The catalogue runs once per pipeline invocation (cohort-level, not per sample) and merges per-sample per-caller calls by clustering strategy. `orf_class` selects the strategy but is never part of a grouping key, because callers disagree on class for the same ORF and keying on it would emit one row per disagreeing caller:
 
-- **Annotated multi-exon CDS** are collapsed by `transcript_id` so that different intron chains aren't accidentally merged into a single entry.
-- **Single-exon novel intergenic ORFs** (class code `u`) are clustered by 80% reciprocal overlap on the outer genomic span.
-- **smORFs** (amino-acid length ≤ 100) are clustered by 80% reciprocal overlap and then peptide-level deduplicated with MMseqs2 (`--min-seq-id 0.9 -c 0.8`) on the `*.catalogue.aa.fasta` (from `bedtools getfasta` + `seqkit translate`), folding each multi-member cluster to one representative (GENCODE convention, Mudge et al. 2022). `--skip_orf_collapse` publishes the coordinate-merged catalogue without this step.
+- **Annotated CDS** are grouped by `transcript_id` and then clustered by 80% reciprocal overlap within the transcript, so different intron chains aren't merged into a single entry while a short truncated variant still stays a separate row from the full-length CDS.
+- **Transcript-anchored ORFs** (`uORF`, `uoORF`, `dORF`, `doORF`, `intORF`, `other`) are collapsed by transcript, strand and outer span.
+- **Novel intergenic ORFs** are clustered by 80% reciprocal overlap on the outer genomic span.
+- **Small ORFs** (`is_smorf`, i.e. `aa_length` ≤ `--smorf_max_aa`, default 100) are additionally peptide-level deduplicated with MMseqs2 (`--min-seq-id 0.9 -c 0.8`) on the `*.catalogue.aa.fasta` (from `bedtools getfasta` + `seqkit translate`), folding each multi-member cluster to one representative (GENCODE convention, Mudge et al. 2022). `--skip_orf_collapse` publishes the coordinate-merged catalogue without this step.
 - **Cross-caller consensus** is recorded by setting `called_by_<caller> = 1` for every caller that contributed a member of the cluster, plus a per-caller score column.
 
+`orf_class` is purely positional and never encodes length; use `is_smorf` to select small ORFs. Not every caller can report every class — Ribo-TISH's `5'UTR` covers both `uORF` and `uoORF`, and PRICE's vocabulary has no `doORF` — so a class missing for one caller is a tool limitation rather than evidence about the ORF. `orf_type_native` preserves each caller's own label so these decisions stay auditable.
+
 Host `gene_id` and `transcript_id` are resolved, and transcript-coordinate calls lifted to the genome, against the union of the full multi-isoform annotation (`--gtf`) and the filtered novel transcripts. The callers do not all receive the same annotation (PRICE takes the full multi-isoform reference, the genome-BAM callers the canonical backbone), so only that union covers every transcript an ORF can be reported on. It is also the reference the ORF-level DTE RNA denominator is quantified against, so catalogue gene ids join directly against that matrix.
 
 ### Per-ORF P-site quantification

docs/usage.md

@@ -672,18 +672,21 @@ The default `--extended_orf_analysis false` keeps the pre-#165 behaviour unchang
 
 ### Cross-sample ORF catalogue
 
-When `--extended_orf_analysis true` is set and at least one ORF caller is enabled, the pipeline produces a cohort-level ORF catalogue under `<outdir>/orf_predictions/catalogue/`. The catalogue normalises each per-sample, per-caller output into a unified BED12 (genomic blocks, multi-exon-aware), then merges across samples and callers with a class-aware strategy:
+When `--extended_orf_analysis true` is set and at least one ORF caller is enabled, the pipeline produces a cohort-level ORF catalogue under `<outdir>/orf_predictions/catalogue/`. The catalogue normalises each per-sample, per-caller output into a unified BED12 (genomic blocks, multi-exon-aware), then merges across samples and callers by clustering strategy. `orf_class` selects the strategy but never enters a grouping key, so callers that disagree on an ORF's class still merge into one row:
 
-- annotated multi-exon CDS are collapsed by `transcript_id` to preserve intron-chain identity;
-- single-exon novel intergenic ORFs are clustered by 80% reciprocal overlap on the outer genomic span;
-- smORFs (≤ 100 aa) are clustered by 80% reciprocal overlap, then peptide-level deduplicated: the catalogue amino-acid FASTA is clustered with MMseqs2 (`--min-seq-id 0.9 -c 0.8`) and each multi-member smORF cluster is folded to one representative, following the GENCODE Ribo-seq ORF catalogue convention (Mudge et al. 2022). Pass `--skip_orf_collapse` to publish the coordinate-merged catalogue without this sequence-level collapse;
+- annotated CDS are grouped by `transcript_id` to preserve intron-chain identity, then clustered by 80% reciprocal overlap within the transcript so a short truncated variant stays distinct from the full-length CDS;
+- transcript-anchored ORFs (`uORF`, `uoORF`, `dORF`, `doORF`, `intORF`, `other`) are collapsed by transcript, strand and outer span;
+- novel intergenic ORFs are clustered by 80% reciprocal overlap on the outer genomic span;
+- small ORFs (`is_smorf`, i.e. ≤ `--smorf_max_aa` aa, default 100) are additionally peptide-level deduplicated: the catalogue amino-acid FASTA is clustered with MMseqs2 (`--min-seq-id 0.9 -c 0.8`) and each multi-member cluster is folded to one representative, following the GENCODE Ribo-seq ORF catalogue convention (Mudge et al. 2022). Pass `--skip_orf_collapse` to publish the coordinate-merged catalogue without this sequence-level collapse;
 - cross-caller consensus is recorded in `called_by_<caller>` binary columns plus `score_<caller>` columns for Ribo-TISH / RiboCode / Rp-Bp / PRICE (Ribotricer scores are excluded from rank aggregation).
 
+`orf_class` records only where an ORF sits relative to the annotated CDS, never how long it is; `is_smorf` carries the length flag. Raise `--smorf_max_aa` (e.g. `150`) to widen the microprotein window — this flags more ORFs as small and brings more of them into the peptide collapse, but leaves every `orf_class` value untouched. Not every caller can report every class: Ribo-TISH's `5'UTR` covers both `uORF` and `uoORF`, PRICE has no `doORF`, and ribotricer's `internal` is a fall-through rather than a frame-tested call so it stays `other`. `orf_type_native` preserves each caller's own label for auditing.
+
 Host gene and transcript ids are resolved against the union of the full multi-isoform annotation (`--gtf`) and the filtered novel transcripts. The callers do not all receive the same annotation (PRICE takes the full multi-isoform reference, the genome-BAM callers the canonical backbone), so only that union covers every transcript an ORF can be reported on; an ORF called on an isoform absent from the annotation falls back to whatever gene label its caller emitted, which for PRICE is every gene its genomic span overlaps, concatenated. With a novel-transcript source the union is also the reference the ORF-level DTE RNA denominator is quantified against, so catalogue gene ids share a namespace with that matrix. Without one the denominator is the primary Salmon matrix, keyed on `--gtf` alone: a `--canonical_gtf` whose gene ids are absent from `--gtf` puts ORFs called against it outside that namespace, and the join drops them (the count is reported on `DTE_COUNTS_PREP`'s stderr).
 
 The catalogue runs once per pipeline invocation (cohort-level, not per sample) and gates on `--extended_orf_analysis true` plus a non-empty enabled-caller set. The default-off path keeps the pre-#167 behaviour unchanged.
 
-Alongside the full catalogue, a consensus view is published under `<outdir>/orf_predictions/catalogue/consensus/` containing only ORFs supported by at least `--orf_min_callers` distinct callers and recurring in at least `--orf_min_samples` samples (both default 1, so the consensus view equals the full catalogue out of the box). Raise either threshold (e.g. `--orf_min_callers 2`) for a higher-confidence catalogue that tames downstream ORF-level multiple testing; the full unfiltered catalogue is always published regardless. The threshold is applied after the smORF collapse, so the consensus is the high-confidence subset of the de-redundified catalogue and a micropeptide folded across several loci is judged on its combined cross-caller / cross-sample evidence (when `--skip_orf_collapse` is set it comes from the merged catalogue instead).
+Alongside the full catalogue, a consensus view is published under `<outdir>/orf_predictions/catalogue/consensus/` containing only ORFs supported by at least `--orf_min_callers` distinct callers and recurring in at least `--orf_min_samples` samples (both default 1, so the consensus view equals the full catalogue out of the box). Raise either threshold (e.g. `--orf_min_callers 2`) for a higher-confidence catalogue that tames downstream ORF-level multiple testing; the full unfiltered catalogue is always published regardless. The threshold is applied after the small-ORF collapse, so the consensus is the high-confidence subset of the de-redundified catalogue and a micropeptide folded across several loci is judged on its combined cross-caller / cross-sample evidence (when `--skip_orf_collapse` is set it comes from the merged catalogue instead).
 
 See [ORF catalogue (cross-sample)](output.md#orf-catalogue-cross-sample) in the output docs for the full list of published files.
 

modules.json

@@ -67,17 +67,17 @@
                     },
                     "custom/orfcollapse": {
                         "branch": "master",
-                        "git_sha": "76e959312e636388810442416972071734c4144a",
+                        "git_sha": "be91fb1578e47bf1a8acc301bf7d755cb7313fc2",
                         "installed_by": ["orftable_fasta_gtf_buildorfcatalogue"]
                     },
                     "custom/orfmerge": {
                         "branch": "master",
-                        "git_sha": "76e959312e636388810442416972071734c4144a",
+                        "git_sha": "be91fb1578e47bf1a8acc301bf7d755cb7313fc2",
                         "installed_by": ["orftable_fasta_gtf_buildorfcatalogue"]
                     },
                     "custom/orfnormalise": {
                         "branch": "master",
-                        "git_sha": "c13f18dbfc4910d9287d9d3fff7b83124e108907",
+                        "git_sha": "be91fb1578e47bf1a8acc301bf7d755cb7313fc2",
                         "installed_by": ["orftable_fasta_gtf_buildorfcatalogue"]
                     },
                     "custom/tx2gene": {
@@ -499,7 +499,7 @@
                     },
                     "orftable_fasta_gtf_buildorfcatalogue": {
                         "branch": "master",
-                        "git_sha": "23ca2b32dacbfc86c23af4ea6413ce96a022ea85",
+                        "git_sha": "be91fb1578e47bf1a8acc301bf7d755cb7313fc2",
                         "installed_by": ["subworkflows"]
                     },
                     "quant_tximport_summarizedexperiment": {