Release 3.10.0 - #2257
Open
FriederikeHanssen wants to merge 28 commits into
Open
Conversation
<!-- # nf-core/sarek pull request Many thanks for contributing to nf-core/sarek! Please fill in the appropriate checklist below (delete whatever is not relevant). These are the most common things requested on pull requests (PRs). Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release. Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md) --> ## PR checklist - [ ] This comment contains a description of changes (with reason). - [ ] If you've fixed a bug or added code that should be tested, add tests! - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md) - [ ] If necessary, also make a PR on the nf-core/sarek _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository. - [ ] Make sure your code lints (`nf-core pipelines lint`). - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`). - [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`). - [x] Usage Documentation in `docs/usage.md` is updated. - [ ] Output Documentation in `docs/output.md` is updated. - [x] `CHANGELOG.md` is updated. - [ ] `README.md` is updated (including new tool citations and authors/contributors).
Co-authored-by: Maxime U. Garcia <max.u.garcia@gmail.com>
…`--rm-dup all` → `exact`) (#2216) ## Description Closes #2215. When `--normalize_vcfs` is enabled, `VCFS_NORM` runs `bcftools norm --multiallelics -both --rm-dup all`. Because `-m -both` splits a multiallelic record into per-allele rows **at the same position**, the subsequent `--rm-dup all` (which de-duplicates **by position**) deletes all but the first row, **silently dropping a real ALT allele** of heterozygous two-alt (`1/2`) sites. This switches the argument to `--rm-dup exact`, which still removes truly identical duplicate records (the stated intent of the existing comment) but no longer collapses two distinct alleles that share a position. ## Evidence Test profile, `chr22:13575` (DeepVariant `G → C,T`, GT `1/2`): | `bcftools norm` args | records | site 13575 | |---|---|---| | `-m -both --rm-dup all` (before) | 17 | only `G→C` — `G→T` lost | | `-m -both --rm-dup exact` (after) | 18 | `G→C` and `G→T` | Verified end-to-end by re-running `-profile test,docker --tools deepvariant --normalize_vcfs --filter_vcfs` with the change: the normalized VCF keeps both alleles. ## PR checklist - [x] Minimal, focused change (one config argument + CHANGELOG) - [x] Branched off `dev`, targets `dev` - [x] `CHANGELOG.md` updated - [ ] CI green Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Maxime U Garcia <max.u.garcia@gmail.com>
…#2184) ## Summary `snpeff_cache`, `vep_cache` and `igenomes_base` are declared in the schema with `format: directory-path` and default to S3 buckets. nf-schema 2.5+ resolves these defaults against the storage backend and fails the launch if the path is unreachable - no credentials, no network, or an SSO/IAM session that can't see the bucket. Any user without read access to `s3://annotation-cache/` (e.g. on-prem clusters, restricted cloud accounts) currently can't launch the pipeline at all, even when they have no intention of running snpEff or VEP. ## Details The full debate of where this fix belongs is in [nextflow-io/nf-schema#204](nextflow-io/nf-schema#204). The plugin maintainer's settled position is that: 1. The validator should fail when a configured path is unreachable - that's the point. 2. The right fix for default-only S3 paths that nobody explicitly opted into is to: - drop `format: "directory-path"` so the existence probe never runs, and - add the param to `validation.defaultIgnoreParams` so nf-schema skips it entirely. That guidance has been applied in nf-core/rnaseq for `igenomes_base` (PRs [#1696](nf-core/rnaseq#1696) and [#1739](nf-core/rnaseq#1739)). This PR applies the same treatment to sarek's three S3-defaulted directory params. PR [#2083](#2083) already removed `exists: true` from `snpeff_cache`/`vep_cache`, but `format: directory-path` alone is enough to trigger the existence check, so launches still fail for users without access to the default bucket. Fixes #2079 ## Changes - `nextflow.config`: add `igenomes_base`, `snpeff_cache`, `vep_cache` to `validation.defaultIgnoreParams`. - `nextflow_schema.json`: drop `"format": "directory-path"` from the same three params. Derived/resolved paths are still validated individually downstream. ## Test plan - [ ] Launch from an environment without read access to `s3://annotation-cache/` - validation should now pass. - [ ] Launch with valid `--snpeff_cache`/`--vep_cache` overrides - annotation should still work. - [ ] Launch with `--tools snpeff,vep` and confirm download-cache flows still error appropriately if the path is bad at the point of use (not at schema validation). --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Maxime U Garcia <max.u.garcia@gmail.com> Co-authored-by: Friederike Hanssen <friederike.hanssen@seqera.io>
<!-- # nf-core/sarek pull request Many thanks for contributing to nf-core/sarek! Please fill in the appropriate checklist below (delete whatever is not relevant). These are the most common things requested on pull requests (PRs). Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release. Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md) --> ## PR checklist - [ ] This comment contains a description of changes (with reason). - [ ] If you've fixed a bug or added code that should be tested, add tests! - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md) - [ ] If necessary, also make a PR on the nf-core/sarek _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository. - [ ] Make sure your code lints (`nf-core pipelines lint`). - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`). - [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`). - [ ] Usage Documentation in `docs/usage.md` is updated. - [ ] Output Documentation in `docs/output.md` is updated. - [ ] `CHANGELOG.md` is updated. - [ ] `README.md` is updated (including new tool citations and authors/contributors). --------- Co-authored-by: Maxime U Garcia <max.u.garcia@gmail.com> Co-authored-by: Friederike Hanssen <friederike.hanssen@seqera.io> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
## Description
Part of preparing nf-core/sarek for the Nextflow **26.04/26.10**
strict-syntax parser. The uppercase `Channel.` factory is deprecated in
favour of the lowercase `channel.` factory.
This PR replaces **all 438 uppercase `Channel.` occurrences across 42
local files** (`main.nf`, `workflows/`, `subworkflows/local/`,
`modules/local/`, incl. `.nf.test` files) with lowercase `channel.`.
- Pure mechanical casing change — verified that every added line is
identical to its removed counterpart after normalising `Channel.` →
`channel.` (no collateral edits).
- Channel semantics are unchanged; `channel.empty/of/value/fromPath`
behave identically to their uppercase forms.
- Vendored `modules/nf-core/` and `subworkflows/nf-core/` are **not**
touched (they come via `nf-core ... update`).
### Follow-ups (not in this PR, to keep it single-purpose)
Other strict-syntax items surfaced during the audit, tracked separately:
- `channel.from` → `channel.of` at
`subworkflows/local/prepare_genome/main.nf` (`from` is itself
deprecated; casing was applied here but the operator still needs
changing).
- Implicit closure params (`{ it.foo }` → `{ v -> v.foo }`) — ~29
occurrences.
- Remove deprecated `nextflow.enable.configProcessNamesValidation` flag
in `nextflow.config`.
- `workflow.onComplete`/`onError` in the utils subworkflow — will come
via nf-core template sync.
## PR checklist
- [x] This comment contains a description of changes (with reason).
- [x] `CHANGELOG.md` is updated.
- [ ] Make sure your code lints (`nf-core pipelines lint`) — pre-commit
(prettier) passes locally; full lint/nf-test left to CI.
- [ ] Ensure the test suite passes — deferred to CI (mechanical change,
semantically equivalent).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Maxime U Garcia <max.u.garcia@gmail.com>
…#2233) ## Description Replace the last remaining `channel.from` call with `channel.of` in `subworkflows/local/prepare_genome/main.nf`. `channel.from` is deprecated in favour of `channel.of`/`channel.fromList` and prints a runtime deprecation warning. It is not flagged by `nextflow lint` (only the capital-`Channel` namespace is), and it does not break on 25.10 or 26.x — this is deprecation cleanup, not a compatibility fix: [https://docs.seqera.io/nextflow/tutorials/static-types#avoid-deprecated-patterns](https://docs.seqera.io/nextflow/tutorials/static-types#avoid-deprecated-patterns) ## Changes - `subworkflows/local/prepare_genome/main.nf` line 132: `channel.from(...)` → `channel.of(...)` - `CHANGELOG.md`: added entry under `### Changed` ## Semantics The argument is a single file object (`file(bbsplit_fasta_list_in, checkIfExists: true)` returns a single Path), so `channel.of` is a direct drop-in with identical behaviour. The downstream `.splitCsv()` chain is unaffected.
…2234) ## Description Replaces the deprecated `.set {}` channel terminator with plain leading assignment in four local subworkflows. This pattern still runs in Nextflow 26.x but is flagged by the static-types type checker, so this is part of the granular strict-syntax cleanup series. Pure mechanical, semantics-preserving change (`.set { name }` → `name = <chain>`): - `subworkflows/local/prepare_snpsift_databases/main.nf` — `ch_branched` - `subworkflows/local/fastq_preprocess_gatk/main.nf` — `reads_grouping_key` - `subworkflows/local/fastq_preprocess_parabricks/main.nf` — `reads_grouping_key` - `subworkflows/local/prepare_genome/main.nf` — `ch_bbsplit_fasta_list` ## PR checklist - [x] Targets the `dev` branch - [x] `CHANGELOG.md` updated (Developer section) - [ ] No functional/output changes — no tests or docs affected 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…mline CNVKIT cleanups) (#2235) ## What changed & why Nextflow **strict-syntax / 26.x readiness** for sarek's own code, plus two small functional improvements that surfaced along the way. `nextflow lint` is now clean for all local `.nf` and `conf/` files (only vendored `modules/nf-core/` warnings remain — those clear via module updates). ### 🟢 Mechanical — no behaviour change (safe to skim) - **Closures now have explicit, named parameters** across `subworkflows/local/`, `workflows/`, `main.nf` — replacing implicit/generic `it` (e.g. `.branch { meta, vcf -> … }`; single-channel maps named after the source channel, `dbsnp.map { dbsnp_ -> … }`). *Why: strict syntax deprecates implicit `it`.* - **`publishDir` `saveAs` closures** in `conf/modules/*.config` use an explicit `{ filename -> … }`. *Why: same.* - **Unused closure params prefixed `_`; 5 genuinely-unused `take:` inputs removed** (`bam`, `dbsnp_vqsr` ×2, `intervals_bed_gz_tbi`, `bcftools_columns`) with their call args. *Why: dead inputs / lint.* ### 🟡 Functional — worth a look - **germline CNVKit now consumes the shared BAM instead of CRAM.** *Why: `CNVKIT_BATCH` internally re-converts CRAM→BAM (slow), duplicating the pipeline's `CRAM_TO_BAM`; germline now reuses that single conversion — one conversion instead of two on paired runs.* Analysis output content is unchanged. - **`CRAM_TO_BAM` sets `ext.prefix` to the input basename** so the converted BAM (and therefore the CNVKit output filenames) mirror the source CRAM name. *Why: without it the BAM switch above would have renamed cnvkit outputs to `meta.id`, dropping the `…recalibrated…` provenance from the filenames. This keeps the existing naming.* - **Manta now emits complete candidate VCF + tbi pairs** (`candidate_small_indels_vcf(_tbi)`, `candidate_sv_vcf(_tbi)`) with `variantcaller:'manta'` on all outputs, and names SV outputs by type (`diploid_sv_vcf`, `tumor_sv_vcf`). *Why: candidates were previously assigned but never emitted (some missing their tbi / variantcaller tag).* Strelka's annotation input is unchanged. ### 🔵 Snapshots - Updated cnvkit snapshots (`variant_calling_cnvkit`, `variant_calling_all`): cnvkit output filenames now follow the source-CRAM basename, and `samtools` is dropped from `CNVKIT_BATCH` versions (both consequences of the BAM switch). **Filename renames only — file md5 content is unchanged.** `save_output_as_bam` is unaffected (its cnvkit input basename was already the source name). **No new parameters. No output-path changes; cnvkit output filenames are preserved via the `ext.prefix` above.** Pairs with #2232 (the `channel` factory migration) as the closure-`it` half of the strict-syntax work. ## PR checklist - [x] Description of changes (with reason) above. - [x] Code lints (`nextflow lint` — 0 errors, 0 local warnings). - [x] `CHANGELOG.md` updated. - [ ] Test suite: cnvkit snapshots updated for the BAM switch + `ext.prefix` naming; CI re-running to confirm green. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…2237) <!-- # nf-core/sarek pull request Many thanks for contributing to nf-core/sarek! Please fill in the appropriate checklist below (delete whatever is not relevant). These are the most common things requested on pull requests (PRs). Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release. Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md) --> ## PR checklist Closes #2236 - [ ] This comment contains a description of changes (with reason). - [ ] If you've fixed a bug or added code that should be tested, add tests! - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md) - [ ] If necessary, also make a PR on the nf-core/sarek _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository. - [ ] Make sure your code lints (`nf-core pipelines lint`). - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`). - [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`). - [ ] Usage Documentation in `docs/usage.md` is updated. - [ ] Output Documentation in `docs/output.md` is updated. - [ ] `CHANGELOG.md` is updated. - [ ] `README.md` is updated (including new tool citations and authors/contributors).
…2238) Migrates all `gatk4` and `gatk4spark` modules to the `versions` topic channel (updated to nf-core/modules latest via nf-core/tools 4.0.2). ### Changes - Updates 24 modules (21 `gatk4` + 3 `gatk4spark`) to their topic-channel versions. - Removes the corresponding `.mix(<module>.out.versions)` wiring from subworkflows — versions now flow through `channel.topic("versions")` (already consumed in `workflows/sarek.nf`). - Fixes the `MUTECT2` / `MUTECT2_PAIRED` call sites for the new input signature: `gzi` added to the fai tuple, new `alleles` / `alleles_tbi` inputs passed as `[]` (unused in sarek). - `gatk4spark` tool bump 4.6.1.0 → 4.6.2.0 (rides along with the module update). ### Notes - Test snapshots still need regenerating in CI. - First of a stacked series migrating the remaining modules to the versions topic channel; subsequent PRs are based on this branch. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ic channel (#2239) Migrates alignment, UMI and utility modules to the `versions` topic channel. **Stacked on #2238** (base: `topic/gatk`). ### Changes - Updates `bwa`, `bwamem2`, `dragmap`, `fgbio`, `fastp`, `cat`, `gawk`, `gunzip`, `untar`, `unzip`, `tabix`, `spring` to their topic-channel versions. - Removes the corresponding `.out.versions` wiring from subworkflows. - Fixes the `FASTP` call site for the new input signature (adapter fasta folded into the reads tuple). - Tool bumps ride along (see changelog) — notably **fastp 0.24.0 → 1.1.0** (major) and **fgbio 2.4.0 → 3.1.2**. Test snapshots need regenerating in CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nnel (#2240) Migrates variant-calling / VCF modules to the `versions` topic channel. **Stacked on #2239** (base: `topic/alignment-utils`). ### Changes - Updates `freebayes`, `strelka`, `manta`, `tiddit`, `lofreq`, `svdb`, `vcflib`, `vcftools` to their topic-channel versions. - Removes the corresponding `.out.versions` wiring from subworkflows (empty `versions` channels left in place for now — a follow-up PR will strip now-unused init/emit pipeline-wide). - Tool bumps: svdb 2.8.2 → 2.8.4, tiddit 3.6.1 → 3.9.5, vcftools 0.1.16 → 0.1.17. Test snapshots need regenerating in CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…2241) Migrates the nf-core `samtools/*` modules to the `versions` topic channel. **Stacked on #2240** (base: `topic/variant-calling`). ### Changes - Updates `samtools/{bam2fq,collatefastq,convert,faidx,index,merge,mpileup,stats,view}` to their topic-channel versions. - Removes the corresponding `.out.versions` wiring. - Reworks call sites for the new input signatures: `fasta`+`fai` combined into one `[meta, fasta, fai]` tuple (faidx/convert/merge/view), `index_files`/`gzi` added to merge inputs, `view` given fai + qname-tuple + bed-tuple. - Tool bumps: samtools 1.21 → 1.24, htslib 1.21 → 1.24 (for the samtools modules). Test snapshots need regenerating in CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#2242) Migrates QC / coverage modules to the `versions` topic channel. **Stacked on #2241** (base: `topic/samtools`). ### Changes - Updates `fastqc`, `ngscheckmate/ncm`, `goleft/indexcov`, `msisensor2/msi`, `msisensorpro/{msisomatic,scan}`, `ascat` to their topic-channel versions (fastqc + ngscheckmate were already migrated on dev). - Removes the corresponding `.out.versions` wiring. Test snapshots need regenerating in CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2243) Migrates `cnvkit/*` modules to the `versions` topic channel and updates tabix. Base: `dev`. ### Changes - Updates `cnvkit/{antitarget,batch,call,export,genemetrics,reference}` to their topic-channel versions. - Removes the corresponding `.out.versions` wiring. - Rewires `cnvkit/batch` for its new input signature: `fasta`+`fai` combined into one tuple; tumor/normal now carry index slots (`[]`, unused by the script — it indexes internally). - Tool bump: cnvkit 0.9.11 → 0.9.12. - Removes dead `versions` channel plumbing left behind by modules that had already moved to the topic channel via unrelated upstream syncs (`sentieon/*`, `bcftools/sort`, `bcftools/isec`, `bcftools/concat`, `ensemblvep/vep`, `snpsift/annmem`, `consensus_from_sites`), across `bam_joint_calling_germline_sentieon`, `bam_variant_calling_sentieon_dnascope`/`_haplotyper`, `bam_variant_calling_somatic_tnscope`/`_tumor_only_tnscope`, `vcf_consensus`, `vcf_annotate_all`, and `bam_variant_calling_tumor_only_all`. - Replaces the upstream-deprecated `tabix/tabix` and `tabix/bgziptabix` modules (both now `assert false` on any future module update) with `htslib/bgziptabix` across all 15 real call sites plus the vendored `vcf_annotate_snpeff` subworkflow. Test snapshots need regenerating in CI. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migrates the local modules to the `versions` topic channel. **Stacked on #2243** (base: `topic/cnvkit`). ### Changes - Converts `add_info_to_vcf`, `create_intervals_bed` and `samtools/reindex_bam` from `versions.yml` to the versions topic channel, matching the `consensus_from_sites` pattern. - Removes the corresponding `.out.versions` wiring. No tool version changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This PR adds the Parabricks Haplotypecaller as an option for variant calling. ## PR checklist - [x] This comment contains a description of changes (with reason). - [x] If you've fixed a bug or added code that should be tested, add tests! - [x] Make sure your code lints (`nf-core pipelines lint`). - [x] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`). - [x] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`). - [x] Usage Documentation in `docs/usage.md` is updated. - [x] Output Documentation in `docs/output.md` is updated. - [x] `CHANGELOG.md` is updated. - [x] `README.md` is updated (including new tool citations and authors/contributors). --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Maxime U Garcia <max.u.garcia@gmail.com> Co-authored-by: nf-core-bot <core@nf-co.re> Co-authored-by: Friederike Hanssen <friederike.hanssen@seqera.io>
Updates all 16 modules flagged by `nf-core pipelines lint` for having a newer version: `bbmap/bbsplit`, `gatk4/applybqsr`, `parabricks/fq2bam`, `rbt/vcfsplit`, `sentieon/*` (9 modules), `snpsift/annmem`+`annmemcreate`, `yte`. Mostly mechanical container/registry bumps. Two real tool-version bumps: `parabricks/fq2bam` (4.6.0-1 → 4.7.1-1), `snpsift` (5.4.0a → 5.4.0c). `gatk4/applybqsr` had a breaking signature change upstream (fasta/fai/dict merged into one tuple, `ext.suffix` replaced by an explicit `output_suffix` arg) — fixed in `subworkflows/local/bam_applybqsr/main.nf`. **CI follow-ups:** - 3 snapshots need regenerating: `aligner-parabricks` (GPU runner), `annotation_snpsift`, `annotation_merge` - Watch `tests/save_output_as_bam.nf.test` — exercises the `gatk4/applybqsr` fix; only verified locally via `nextflow run -preview` (no Docker available here) - `parabricks/fq2bam` module lint fails on the `nvcr.io` container prefix — pre-existing nf-core-tools limitation, unrelated to this change ## PR checklist - [x] This comment contains a description of changes (with reason). - [ ] If you've fixed a bug or added code that should be tested, add tests! - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/sarek/tree/master/docs/CONTRIBUTING.md) - [ ] If necessary, also make a PR on the nf-core/sarek _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository. - [x] Make sure your code lints (`nf-core pipelines lint`). - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`) — not runnable locally (Docker unavailable); verify on CI. - [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`). - [ ] Usage Documentation in `docs/usage.md` is updated. - [ ] Output Documentation in `docs/output.md` is updated. - [x] `CHANGELOG.md` is updated. - [ ] `README.md` is updated (including new tool citations and authors/contributors). Generated with Claude Code
## Description Resolves the template sync up to nf-core/tools v4.0.3 (squashed into a single commit for a clean diff). ### Key resolutions: - Kept sarek's custom `main.nf`, `nextflow_schema.json`, `tests/default.nf.test`, and `.nftignore` - Accepted updated CI workflows, `.pre-commit-config.yaml`, `nf-test.config`, fastqc/multiqc module updates (with conda-lock files), and new `docs/CONTRIBUTING.md` location - Fixed `UTILS_NFCORE_PIPELINE` call in `utils_nfcore_sarek_pipeline/main.nf` — removed extra `null` argument that no longer matches updated subworkflow signature ### Template changes included: - Nextflow minimum bumped to 25.10.4 - `prek` replaces `pre-commit` - Container configs auto-generated (conda-lock files) - `CONTRIBUTING.md` moved from `.github/` to `docs/` - Updated FastQC and MultiQC modules - New `pr-comment.yml` workflow replaces `linting_comment.yml` --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
## Description Template update for nf-core/tools v4.1.0. Closes #2253. Work on top of the sync: - `conf/modules/*.config`: collapsed list-form `publishDir` into single maps routing through `saveAs`. Required because 4.1.0 lints via `nextflow config -o json`, which fails on a closure nested inside a collection. Published layouts unchanged. - `utils_nfschema_plugin`: renamed `parametersSchema:` → `parameters_schema:` to match the recorded git_sha (fixes `check_local_copy`). No runtime effect — sarek passes `null`. - `tests/nextflow.config`, `tests/lib/UTILS.groovy`, `AGENTS.md`: small missed/stale fixes. ## Checklist - [x] This comment contains a description of changes (with reason) - [ ] If you've fixed a bug or added code that should be tested, add tests! - [x] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/sarek/blob/master/docs/CONTRIBUTING.md) - [ ] If necessary, also make a PR on the nf-core/sarek _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository. - [x] Make sure your code lints (`nf-core pipelines lint`). - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`). - [ ] Usage Documentation in `docs/usage.md` is updated. - [ ] Output Documentation in `docs/output.md` is updated. - [x] `CHANGELOG.md` is updated. - [ ] `README.md` is updated (including new tool citations and authors/contributors). Generated with Claude Code --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
maxulysse
approved these changes
Aug 4, 2026
|
|
Cloud test test_full_germline_ncbench_agilent launched on Seqera Platform |
|
Cloud test test_full_aws launched on Seqera Platform |
|
Cloud test test_full_germline_aws launched on Seqera Platform |
pinin4fjords
reviewed
Aug 4, 2026
| // (LoginException: invalid null input: name), because the container's own /etc/passwd | ||
| // has no entry for the host UID that docker.runOptions maps it to. Bind-mounting the | ||
| // host's /etc/passwd/group (which do have that entry) fixes the native lookup. | ||
| containerOptions = { workflow.containerEngine in ['singularity', 'apptainer'] |
Member
There was a problem hiding this comment.
Don't quite follow this- but does it belong in the module somehow?
Member
There was a problem hiding this comment.
should there be a params.use_gatk_spark conditional here?
Contributor
Author
There was a problem hiding this comment.
yeah we probably should do that.
as for why: Pontus explained it here: https://nfcore.slack.com/archives/C043FMKUNLB/p1785588223796839?thread_ts=1785426088.069839&cid=C043FMKUNLB
Contributor
Author
There was a problem hiding this comment.
pinin4fjords
requested changes
Aug 4, 2026
pinin4fjords
left a comment
pinin4fjords
left a comment
Member
There was a problem hiding this comment.
AI-assisted review. Requesting changes for two silent-output paths: LoFTEE annotations are missing under conda despite a successful VEP process, and Parabricks joint germline runs complete without producing a VCF. I have also left two low-severity changelog corrections.
erikrikarddaniel
left a comment
erikrikarddaniel
left a comment
Member
There was a problem hiding this comment.
Looks great. I have a few comments in the code, discretionary.
Since it's so large and I don't know Sarek well, I let Claude loose, and this is what he found:
Worth a close look
1. --normalize_vcfs bug fix (#2216) — verified the fix itself is correct: bcftools norm --rm-dup all → --rm-dup exact.
--rm-dup all was removing any duplicate-position record, --rm-dup exact only removes true exact duplicates — the old flag was silently dropping a real distinct ALT allele at 1/2 multiallelic sites.
However, I don't see a new test scenario for this specific fix: subworkflows/local/post_variantcalling/tests/main.nf.test only changed via mechanical Channel. → channel. renames (plus dropping workflow.out.versions from snapshot assertions, part of the topics migration) — no added test case.
Same story for tests/postprocess_normalization.nf.test and tests/postprocess_concatenation_normalization.nf.test: only their .snap files changed, not the test definitions.
So coverage for this exact scenario depends entirely on whether the existing shared fixture (genomics/sarscov2/illumina/vcf/test.vcf.gz) happens to already contain a real 1/2 multiallelic record — I can't confirm that without inspecting the fixture directly.
Could you confirm whether that's the case, or would it be worth adding a small dedicated regression test/fixture with a known 1/2 site so this doesn't silently regress again?
2. mosdepth silent-empty-coverage bug (#2241) — old container's htslib couldn't decode CRAM 3.1 from samtools 1.24, silently producing empty coverage files (no error, just missing MultiQC sections), "most visible in the parabricks path" per your own note.
Was this caught by a test that actually asserts non-empty coverage output, or only noticed by inspection? If the latter, same concern as above — a test could regress silently again the same way.
Two behavior changes I'd want explicit confirmation are intentional
3. CNVKit output naming changed for --step variant_calling (#2244 side effect) — when resuming from user-supplied CRAM/BAM, CNVKit output files are now named after the input filename instead of the sample ID (runs from FASTQ are unaffected).
This is a user-facing filename change for a subset of run modes — anyone with downstream scripts/globs keyed to the old sample-based naming breaks silently.
Confirming this is an accepted/intentional side effect, and — is it worth a callout in docs/usage.md in addition to the CHANGELOG, since it's the kind of thing someone would only discover after their pipeline output changed shape?
4. GATK4_APPLYBQSR ext.suffix override silently stops working (#2252) — anyone with a custom modules.config setting ext.suffix on GATK4_APPLYBQSR now has that override silently ignored (output format is controlled by params.save_output_as_bam instead); GATK4SPARK_APPLYBQSR is unaffected.
A silently-ignored user config is worse than an error.
Is there any way to have nf-core pipelines lint or a runtime check warn if ext.suffix is set on that specific process now that it's a no-op?
❌ nf-test failed with latest Nextflow versionNote Tests with Nextflow's latest version failed but it will not cause a CI workflow failure.
See the full run for details. |
Co-authored-by: Jonathan Manning <pininforthefjords@gmail.com> Co-authored-by: Daniel Lundin <erik.rikard.daniel@gmail.com> Co-authored-by: Friederike Hanssen <friederike.hanssen@seqera.io>
## Description RunsOn self-hosted runners are available again, so this reverts the `ubuntu-latest` stopgap introduced in 861f27b (squashed into #2248). Restores across `nf-test.yml`, `nf-test-gpu.yml` and `nf-test-sentieon.yml`: - `4cpu-linux-x64` for the three `*-changes` shard-calculation jobs - `4cpu-linux-x64` + `volume=80gb` for the `nf-test` and `nf-test-sentieon` matrix jobs - `2cpu-linux-x64` for `confirm-pass` - `g4dn.xlarge` / `ubuntu24-gpu-x64` for `nf-test-gpu` — the GPU matrix has had no real GPU since the stopgap Scope is limited to the `runs-on:` blocks. `nf-test-gpu.yml` and `nf-test-sentieon.yml` are now byte-identical to their pre-#2248 state; `nf-test.yml` keeps the template changes from that PR (checkout v7 pin, `tags`/`max_shards` key order, and the latest-everything PR-comment machinery). The GATK Spark `/etc/passwd` bind-mounts in `conf/modules/` are Spark fixes, not runner workarounds, and are left in place. ### CI coverage This PR's own run exercises all three restored runner classes end to end — the full sharded suite resolved (15 cpu shards, 10 sentieon shards, 2 gpu shards), so this is a real validation of the runners rather than a smoke test. ## PR checklist - [x] This comment contains a description of changes (with reason). - [x] Make sure your code lints (`nf-core pipelines lint`). - [ ] Usage Documentation in `docs/usage.md` is updated. - [ ] Output Documentation in `docs/output.md` is updated. - [ ] `CHANGELOG.md` is updated. - [ ] `README.md` is updated (including new tool citations and authors/contributors). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
pinin4fjords
approved these changes
Aug 7, 2026
pinin4fjords
left a comment
pinin4fjords
left a comment
Member
There was a problem hiding this comment.
Trust you to sort any final bits and appropriately disregard any invalid comments from me :-)
|
Cloud test test_full_germline_aws launched on Seqera Platform |
FriederikeHanssen
added a commit
that referenced
this pull request
Aug 7, 2026
Rewrites the merged entry, which joined the two fixes with a comma splice, and adds the missing '#' to the 2243 link text. Generated with Claude Code
Contributor
Author
given that the snapshots changed, I would say it catches the regression?
I am not sure I am following the reasoning here. The snapshots failed when I updated the tools separately because the encoding is different, so the tests caught it as intended. Once I updated both tools together, all expected output files were there again.
this is actually an effect of #2235. the strict syntax clean up surfaced a bug where the wrong (not analytically but efficiently) input channel was used. we are doing cram to bam conversion for specific tools ahead of time because they cannot handle cram input. Out of them CNVKit is the only module that has an additional samtools call to do this inside, which is how this hasn't surfaced until now. Root cause for the naming is that the CNVkit module is doing some unusual hardwiring. We got three options: fiddle with the module, make cnvkit make use of the same naming as all other tools, or update the naming of all other tools in sarek to adhere to sareks. For now I went with the easiest option, changing cnvkits output to be more canonical with the other tools. I can update the module to force keeping the naming, but I am happier if the naming is consistent.
I don't think so. We called it out in the changelog and I can call it out again in the release notes |
17 tasks
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com><!-- # nf-core/sarek pull request Many thanks for contributing to nf-core/sarek! Please fill in the appropriate checklist below (delete whatever is not relevant). These are the most common things requested on pull requests (PRs). Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release. Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md) --> ## PR checklist - [ ] This comment contains a description of changes (with reason). - [ ] If you've fixed a bug or added code that should be tested, add tests! - [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/sarek/tree/master/.github/CONTRIBUTING.md) - [ ] If necessary, also make a PR on the nf-core/sarek _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository. - [ ] Make sure your code lints (`nf-core pipelines lint`). - [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`). - [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`). - [ ] Usage Documentation in `docs/usage.md` is updated. - [ ] Output Documentation in `docs/output.md` is updated. - [ ] `CHANGELOG.md` is updated. - [ ] `README.md` is updated (including new tool citations and authors/contributors). --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
Author
|
Thank you both for taking the time to review 🙏 I have addressed all points I marked in the comments here: #2260 The remaining open ones are the container options, which is waiting for the corresponding modules PR to be blessed and merged |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Main changes:
The modules are completed migrated to using topics and a lot of strict syntax fixes (Channel -> channel, it replacements etc).
New items and bug fixes:
PR checklist
nf-core pipelines lint).nextflow run . -profile test,docker --outdir <OUTDIR>).nextflow run . -profile debug,test,docker --outdir <OUTDIR>).docs/usage.mdis updated.docs/output.mdis updated.CHANGELOG.mdis updated.README.mdis updated (including new tool citations and authors/contributors).