Add optional parallel genome alignment support - #86
Open
SergeWielhouwer wants to merge 3 commits into
Open
Conversation
Agent-Logs-Url: https://github.com/SergeWielhouwer/FastQ-Screen/sessions/363f6cc7-2c64-4c68-adaf-5512e9001628 Co-authored-by: SergeWielhouwer <87863674+SergeWielhouwer@users.noreply.github.com>
Agent-Logs-Url: https://github.com/SergeWielhouwer/FastQ-Screen/sessions/363f6cc7-2c64-4c68-adaf-5512e9001628 Co-authored-by: SergeWielhouwer <87863674+SergeWielhouwer@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi Steven,
Thanks for your work on FastQ-Screen :).
I was looking into ways to speed up FastQ-Screen and implemented an optional parallelisation of the genome alignment step through GitHub Copilot (with Claude Opus) and manually reviewed and tested the proposed changes.
The main idea is to allow multiple reference genome alignments to run concurrently using Parallel::ForkManager, which can improve runtime performance on systems with sufficient CPU resources. I think this could be particularly beneficial for larger datasets (e.g. WGS) or when screening against multiple reference genomes.
The implementation is fully backwards compatible: the original sequential behaviour is preserved by default, and parallel execution is only enabled when using the new --parallel_genomes option (so users don't need to install ForkManager per se).
On a subsampled HG002 dataset, I observed approximately a 2.5–3× runtime improvement in standard alignment mode.
Please see some benchmark results below.
I also evaluated the implementation in Bismark (Bowtie2) mode, where I observed speed-ups of approximately 3–5× under the tested conditions. Of course, the actual performance gain will depend on the used system, dataset sizes, and the number and sizes of the configured reference genomes.
The functional output (plots/tables) remained identical to the original implementation in both standard alignment mode and Bismark mode.
I would really appreciate your feedback on the implementation and whether you think this would be a useful addition to a future FastQ-Screen release.
Thanks!
Serge