wip: lexicmap and logan workflows w download by d-callan · Pull Request #1210 · galaxyproject/iwc

d-callan · 2026-04-17T14:13:03Z

wip as i try to figure how to make this happen well. mostly right now it represents what id like to be able to do for brc using kmindex/ logan as a guinea pig

FOR CONTRIBUTOR:

I have read the Adding workflows guidelines
License permits unrestricted use (educational + commercial)
Please also take note of the reviewer guidelines below to facilitate a smooth review process.

FOR REVIEWERS:

.dockstore.yml: file is present and aligned with creator metadata in workflow. ORCID identifiers are strongly encouraged in creator metadata. The .dockstore.yml file is required to run tests
Workflow is sufficiently generic to be used with lab data and does not hardcode sample names, reference data and can be run without reading an accompanying tutorial.
In workflow: annotation field contains short description of what the workflow does. Should start with This workflow does/runs/performs … xyz … to generate/analyze/etc …
In workflow: workflow inputs and outputs have human readable names (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless it is generally understood. Altering input or output labels requires adjusting these labels in the the workflow-tests.yml file as well
In workflow: name field should be human readable (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless generally understood
Workflow folder: prefer dash (-) over underscore (_), prefer all lowercase. Folder becomes repository in iwc-workflows organization and is included in TRS id
Readme explains what workflow does, what are valid inputs and what outputs users can expect. If a tutorial or other resources exist they can be linked. If a similar workflow exists in IWC readme should explain differences with existing workflow and when one might prefer one workflow over another
Changelog contains appropriate entries
Large files (> 100 KB) are uploaded to zenodo and location urls are used in test file

github-actions · 2026-04-17T14:25:58Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	1
Passed	0
Error	1
Failure	0
Skipped	0

Errored Tests

❌ kmindex-logan-sra-retrieval.ga_0

Execution Problem:

Failed to run workflow, invocation ended in [failed] state.

Workflow invocation details

Invocation Messages
- Invocation scheduling failed because step 8 requires a dataset collection created by step 6, but dataset collection entered a failed state.

Steps

Step 1: Index To Query:
- step_state: scheduled
Step 2: Query Sequence:
- step_state: scheduled
Step 3: Z-value (k-mer extension factor):
- step_state: scheduled
Step 4: Minimum shared k-mer proportion:
- step_state: scheduled
Step 5: How many accessions to keep:
- step_state: scheduled

Step 6: Unlabelled step (toolshed.g2.bx.psu.edu/repos/iuc/kmindex/kmindex_query/0.6.0+galaxy2):

step_state: scheduled

Jobs

Job 1:

Job state is error

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"2b4c20183a6911f1898a7ced8dcd6908"`
chromInfo	`"/tmp/tmp1fkqltoc/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
db_opts	`{"__current_case__": 1, "db_opts_selector": "db", "kmindex": "VIRALRNA_ROD"}`
dbkey	`"?"`
fast	`false`
format	`"json"`
threshold	`"0.5"`
verbose	`"info"`
zvalue	`"6"`

Step 7: (Boolean) Subset Accessions (toolshed.g2.bx.psu.edu/repos/iuc/map_param_value/map_param_value/0.2.0):

step_state: scheduled

Jobs

Job 1:

Job state is running

Command Line:

```
cd ../; python _evaluate_expression_.py
```

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"2b4c20183a6911f1898a7ced8dcd6908"`
chromInfo	`"/tmp/tmp1fkqltoc/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
input_param_type	`{"__current_case__": 1, "input_param": "5", "mappings": [{"__index__": 0, "from": "0", "to": "false"}], "type": "integer"}`
output_param_type	`"boolean"`
unmapped	`{"__current_case__": 0, "on_unmapped": "input"}`

Step 8: Parse SRA Accessions:
- step_state: new
Step 9: Unlabelled step:
- step_state: new
Step 10: Sort Accessions By Best Match:
- step_state: new
Step 11: Unlabelled step:
- step_state: new
Step 12: Subset Accessions:
- step_state: new
Step 13: Choose Subset or Full Set of Accessions:
- step_state: new
Step 14: Unlabelled step:
- step_state: new
- Subworkflow Steps

Other invocation details
- error_message
  - Failed to run workflow, invocation ended in [failed] state.
- history_id
  - 2c89423437022e58
- history_state
  - running
- invocation_id
  - 2c89423437022e58
- invocation_state
  - failed
- messages
  - [{'dependent_workflow_step_id': 5, 'hdca_id': '2c89423437022e58', 'reason': 'collection_failed', 'workflow_step_id': 7}]
- workflow_id
  - c4b5026932dea8b0

github-actions · 2026-04-18T12:21:37Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	2
Passed	0
Error	2
Failure	0
Skipped	0

Errored Tests

❌ kmindex-logan-sra-retrieval.ga_0

Execution Problem:

Failed to run workflow, invocation ended in [failed] state.

Workflow invocation details

Invocation Messages
- Invocation scheduling failed because step 8 requires a dataset collection created by step 6, but dataset collection entered a failed state.

Steps

Step 1: Index To Query:
- step_state: scheduled
Step 2: Query Sequence:
- step_state: scheduled
Step 3: Z-value (k-mer extension factor):
- step_state: scheduled
Step 4: Minimum shared k-mer proportion:
- step_state: scheduled
Step 5: How many accessions to keep:
- step_state: scheduled

Step 6: Unlabelled step (toolshed.g2.bx.psu.edu/repos/iuc/kmindex/kmindex_query/0.6.0+galaxy2):

step_state: scheduled

Jobs

Job 1:

Job state is error

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"f55726b63b2011f1898a7ced8d636b86"`
chromInfo	`"/tmp/tmpf04uph3m/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
db_opts	`{"__current_case__": 1, "db_opts_selector": "db", "kmindex": "VIRALRNA_ROD"}`
dbkey	`"?"`
fast	`false`
format	`"json"`
threshold	`"0.5"`
verbose	`"info"`
zvalue	`"6"`

Step 7: (Boolean) Subset Accessions (toolshed.g2.bx.psu.edu/repos/iuc/map_param_value/map_param_value/0.2.0):

step_state: scheduled

Jobs

Job 1:

Job state is running

Command Line:

```
cd ../; python _evaluate_expression_.py
```

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"f55726b63b2011f1898a7ced8d636b86"`
chromInfo	`"/tmp/tmpf04uph3m/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
input_param_type	`{"__current_case__": 1, "input_param": "5", "mappings": [{"__index__": 0, "from": "0", "to": "false"}], "type": "integer"}`
output_param_type	`"boolean"`
unmapped	`{"__current_case__": 0, "on_unmapped": "input"}`

Step 8: Parse SRA Accessions:
- step_state: new
Step 9: Unlabelled step:
- step_state: new
Step 10: Sort Accessions By Best Match:
- step_state: new
Step 11: Unlabelled step:
- step_state: new
Step 12: Subset Accessions:
- step_state: new
Step 13: Choose Subset or Full Set of Accessions:
- step_state: new
Step 14: Unlabelled step:
- step_state: new
- Subworkflow Steps

Other invocation details
- error_message
  - Failed to run workflow, invocation ended in [failed] state.
- history_id
  - 465437169756c487
- history_state
  - running
- invocation_id
  - 465437169756c487
- invocation_state
  - failed
- messages
  - [{'dependent_workflow_step_id': 5, 'hdca_id': '465437169756c487', 'reason': 'collection_failed', 'workflow_step_id': 7}]
- workflow_id
  - 18b8775abfc02bad

❌ lexicmap-sra-retrieval.ga_0
Execution Problem:
- Failed to run workflow, invocation ended in [failed] state.
Workflow invocation details
- Invocation Messages
  - Invocation scheduling failed because an unexpected failure occurred.
- Steps
  
  Step 1: Query Sequence:
  
  step_state: scheduled
  
  Step 2: LexicMap Index:
  
  step_state: scheduled
  
  Step 3: Minimum alignment length (bp):
  
  step_state: scheduled
  
  Step 4: Minimum percent identity (%):
  
  step_state: scheduled
  
  Step 5: Maximum E-value:
  
  step_state: scheduled
  
  Step 6: How many accessions to keep:
  
  step_state: scheduled
  
  Step 7: Minimum total query coverage per genome (%):
  
  step_state: new
- Other invocation details
  
  error_message
  
  Failed to run workflow, invocation ended in [failed] state.
  
  history_id
  
  2bf28cee78a42a42
  
  history_state
  
  ok
  
  invocation_id
  
  2bf28cee78a42a42
  
  invocation_state
  
  failed
  
  messages
  
  [{'details': None, 'reason': 'unexpected_failure', 'workflow_step_id': None}]
  
  workflow_id
  
  58aafee06b486990

d-callan · 2026-04-20T13:55:49Z

so this is conceptually complete now i think @mvdbeek in terms of what im hoping to do for brc. i still dont know what to do for tests though

mvdbeek · 2026-04-20T14:11:07Z

We can mark certain workflows as requiring an external galaxy instance, that's probably what we'll need to do here. That work still needs finishing, I'll see where we are with this.

github-actions · 2026-04-20T15:07:35Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	2
Passed	0
Error	2
Failure	0
Skipped	0

Errored Tests

❌ kmindex-logan-sra-retrieval.ga_0

Execution Problem:

Failed to run workflow, invocation ended in [failed] state.

Workflow invocation details

Invocation Messages
- Invocation scheduling failed because step 8 requires a dataset collection created by step 6, but dataset collection entered a failed state.

Steps

Step 1: Index To Query:
- step_state: scheduled
Step 2: Query Sequence:
- step_state: scheduled
Step 3: Z-value (k-mer extension factor):
- step_state: scheduled
Step 4: Minimum shared k-mer proportion:
- step_state: scheduled
Step 5: How many accessions to keep:
- step_state: scheduled

Step 6: Unlabelled step (toolshed.g2.bx.psu.edu/repos/iuc/kmindex/kmindex_query/0.6.0+galaxy2):

step_state: scheduled

Jobs

Job 1:

Job state is error

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"b6a145d83cc311f1898a7ced8d4c99b6"`
chromInfo	`"/tmp/tmpafnerey0/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
db_opts	`{"__current_case__": 1, "db_opts_selector": "db", "kmindex": "VIRALRNA_ROD"}`
dbkey	`"?"`
fast	`false`
format	`"json"`
threshold	`"0.5"`
verbose	`"info"`
zvalue	`"6"`

Step 7: (Boolean) Subset Accessions (toolshed.g2.bx.psu.edu/repos/iuc/map_param_value/map_param_value/0.2.0):

step_state: scheduled

Jobs

Job 1:

Job state is running

Command Line:

```
cd ../; python _evaluate_expression_.py
```

Traceback:

Job Parameters:

Job parameter	Parameter value
__input_ext	`"input"`
__workflow_invocation_uuid__	`"b6a145d83cc311f1898a7ced8d4c99b6"`
chromInfo	`"/tmp/tmpafnerey0/galaxy-dev/tool-data/shared/ucsc/chrom/?.len"`
dbkey	`"?"`
input_param_type	`{"__current_case__": 1, "input_param": "5", "mappings": [{"__index__": 0, "from": "0", "to": "false"}], "type": "integer"}`
output_param_type	`"boolean"`
unmapped	`{"__current_case__": 0, "on_unmapped": "input"}`

Step 8: Parse SRA Accessions:
- step_state: new
Step 9: Unlabelled step:
- step_state: new
Step 10: Sort Accessions By Best Match:
- step_state: new
Step 11: Unlabelled step:
- step_state: new
Step 12: Subset Accessions:
- step_state: new
Step 13: Choose Subset or Full Set of Accessions:
- step_state: new
Step 14: Unlabelled step:
- step_state: new
- Subworkflow Steps

Other invocation details
- error_message
  - Failed to run workflow, invocation ended in [failed] state.
- history_id
  - ea88001d610288e9
- history_state
  - running
- invocation_id
  - ea88001d610288e9
- invocation_state
  - failed
- messages
  - [{'dependent_workflow_step_id': 5, 'hdca_id': 'ea88001d610288e9', 'reason': 'collection_failed', 'workflow_step_id': 7}]
- workflow_id
  - be2119f89e3a54d6

❌ lexicmap-sra-retrieval.ga_0
Execution Problem:
- Failed to run workflow, invocation ended in [failed] state.
Workflow invocation details
- Invocation Messages
  - Invocation scheduling failed because an unexpected failure occurred.
- Steps
  
  Step 1: Query Sequence:
  
  step_state: scheduled
  
  Step 2: LexicMap Index:
  
  step_state: scheduled
  
  Step 3: Minimum alignment length (bp):
  
  step_state: scheduled
  
  Step 4: Minimum percent identity (%):
  
  step_state: scheduled
  
  Step 5: Maximum E-value:
  
  step_state: scheduled
  
  Step 6: Minimum total query coverage per genome (%):
  
  step_state: new
- Other invocation details
  
  error_message
  
  Failed to run workflow, invocation ended in [failed] state.
  
  history_id
  
  fc7a1cb47820416b
  
  history_state
  
  ok
  
  invocation_id
  
  fc7a1cb47820416b
  
  invocation_state
  
  failed
  
  messages
  
  [{'details': None, 'reason': 'unexpected_failure', 'workflow_step_id': None}]
  
  workflow_id
  
  108b7495c18dfa85

mvdbeek · 2026-07-05T07:31:45Z

If it helps we can test against usegalaxy.org now, see https://github.com/galaxyproject/iwc/tree/main/workflows#request-testing-against-an-external-galaxy-instance

…segalaxy.org

- kmindex: bump tool version to 0.6.1+galaxy3, update awk regex to match [ESD]RR accessions (SRR, ERR, DRR), lower threshold to 0.25 - lexicmap: add sort step, add null values for new optional coverage parameters - Replace file-based expected accession list comparisons with regex assertions (has_line_matching [ESD]RR[0-9]+) for index resilience - Add element_tests to collection outputs for planemo compatibility - Reduce accessions to keep from 5 to 1 to minimize test runtime - Remove expected-accession-list.txt files (no longer needed)

- Update .ga files with new boolean 'Download hits' input that controls whether SRA reads are downloaded or only the accession list is returned - Set Download hits: true in test YAMLs to exercise download path - Fix ORCID URI and add release field in new .ga files

feat: first attempt logan/ kmindex w download

93f997b

mvdbeek reviewed Apr 17, 2026

View reviewed changes

Comment thread workflows/data-fetching/kmindex-logan-sra-retrieval/kmindex-logan-sra-retrieval.ga Outdated

feat: add lexicmap search w download

2f30851

polish up lexicmap wf and test

7f069b3

d-callan added 3 commits July 29, 2026 14:14

Merge branch 'main' into lmls-download

8360783

Add .wt_instance for kmindex and lexicmap workflows to test against u…

50c34a9

…segalaxy.org

d-callan had a problem deploying to secret-env August 5, 2026 00:21 — with GitHub Actions Failure

Fix ORCID URI and add release field for lint compliance

29c6f20

d-callan had a problem deploying to secret-env August 5, 2026 02:28 — with GitHub Actions Failure

d-callan had a problem deploying to secret-env August 5, 2026 16:13 — with GitHub Actions Failure

d-callan marked this pull request as ready for review August 5, 2026 19:21

Navigation Menu

Uh oh!

wip: lexicmap and logan workflows w download - #1210

wip: lexicmap and logan workflows w download#1210
d-callan wants to merge 8 commits into
galaxyproject:mainfrom
d-callan:lmls-download

d-callan commented Apr 17, 2026

Uh oh!

Uh oh!

github-actions Bot commented Apr 17, 2026

Workflow invocation details

Uh oh!

github-actions Bot commented Apr 18, 2026

Workflow invocation details

Workflow invocation details

Uh oh!

d-callan commented Apr 20, 2026

Uh oh!

mvdbeek commented Apr 20, 2026

Uh oh!

github-actions Bot commented Apr 20, 2026

Workflow invocation details

Workflow invocation details

Uh oh!

mvdbeek commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

d-callan commented Apr 17, 2026

Uh oh!

Uh oh!

github-actions Bot commented Apr 17, 2026

Test Results (powered by Planemo)

Test Summary

Workflow invocation details

Uh oh!

github-actions Bot commented Apr 18, 2026

Test Results (powered by Planemo)

Test Summary

Workflow invocation details

Workflow invocation details

Uh oh!

d-callan commented Apr 20, 2026

Uh oh!

mvdbeek commented Apr 20, 2026

Uh oh!

github-actions Bot commented Apr 20, 2026

Test Results (powered by Planemo)

Test Summary

Workflow invocation details

Workflow invocation details

Uh oh!

mvdbeek commented Jul 5, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants