Skip to content

Add optional parallel genome alignment support - #86

Open
SergeWielhouwer wants to merge 3 commits into
StevenWingett:developmentfrom
SergeWielhouwer:copilot/parallelise-genome-alignment
Open

Add optional parallel genome alignment support #86
SergeWielhouwer wants to merge 3 commits into
StevenWingett:developmentfrom
SergeWielhouwer:copilot/parallelise-genome-alignment

Conversation

@SergeWielhouwer

Copy link
Copy Markdown

Hi Steven,

Thanks for your work on FastQ-Screen :).

I was looking into ways to speed up FastQ-Screen and implemented an optional parallelisation of the genome alignment step through GitHub Copilot (with Claude Opus) and manually reviewed and tested the proposed changes.

The main idea is to allow multiple reference genome alignments to run concurrently using Parallel::ForkManager, which can improve runtime performance on systems with sufficient CPU resources. I think this could be particularly beneficial for larger datasets (e.g. WGS) or when screening against multiple reference genomes.

The implementation is fully backwards compatible: the original sequential behaviour is preserved by default, and parallel execution is only enabled when using the new --parallel_genomes option (so users don't need to install ForkManager per se).

On a subsampled HG002 dataset, I observed approximately a 2.5–3× runtime improvement in standard alignment mode.

Please see some benchmark results below.

afbeelding Figure 1. FastQ-Screen runtime scaling for existing v0.16.0 implementation in standard alignment mode. The performance plateaus after >16 threads. afbeelding Figure 2. FastQ-Screen runtime scaling for v0.16.0-GS implementation in standard alignment mode. The optimal performance is observed around 32 threads + 4 parallel_genomes (t32_pg4) equalling 128 threads. A ~2.5–3× maximum increase is observed when comparing to the existing implementation, especially for the higher thread counts (>16).

I also evaluated the implementation in Bismark (Bowtie2) mode, where I observed speed-ups of approximately 3–5× under the tested conditions. Of course, the actual performance gain will depend on the used system, dataset sizes, and the number and sizes of the configured reference genomes.

The functional output (plots/tables) remained identical to the original implementation in both standard alignment mode and Bismark mode.

I would really appreciate your feedback on the implementation and whether you think this would be a useful addition to a future FastQ-Screen release.

Thanks!

Serge

Copilot AI and others added 3 commits May 6, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants