feat: Automate configuration YAML generation (Resolves #24) - #43
Open
dyrpsf wants to merge 1 commit into
Open
Conversation
Resolves draeger-lab#24. Introduces scripts/sync_configs.py to automatically parse hqtb_config_default.yaml and generate basic and advanced variations, stripping dev tags (@todo, @IdEA) and omitting advanced parameter blocks.
3 tasks
There was a problem hiding this comment.
Pull request overview
This PR implements Issue #24 by adding a developer script to generate/synchronize the derived HQTB YAML configs (advanced/basic) from the default config, reducing manual duplication and drift across config variants.
Changes:
- Added
scripts/sync_configs.pyto regeneratehqtb_advanced_config_expl.yamlandhqtb_basic_config_expl.yamlfromhqtb_config_default.yaml. - Updated the advanced/basic YAMLs to reflect regenerated output (including new/realigned sections like
template.namespace,validation,performance). - Attempted to make the “basic” config minimal by omitting “advanced” blocks.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| scripts/sync_configs.py | Adds generator that rewrites advanced/basic configs from the default source-of-truth. |
| src/specimen/data/config/hqtb_basic_config_expl.yaml | Regenerated “basic” config; currently omits sections the workflow expects. |
| src/specimen/data/config/hqtb_advanced_config_expl.yaml | Regenerated “advanced” config to align with the default config contents/structure. |
Comments suppressed due to low confidence (1)
src/specimen/data/config/hqtb_advanced_config_expl.yaml:109
- The comment references
isreaction_comlete(), which looks like a misspelling ofisreaction_complete(). Since this file is generated from the default config, fix the reference inhqtb_config_default.yaml(source of truth) and regenerate rather than editing only the generated output.
formula-check: 'existence' # When checking, if a metabolite can be added to the model
# also check the formula. For more information about
# available options, please refer to the docs of
# the function isreaction_comlete().
exclude-dna: True # Exclude reactions containing 'DNA' in their name
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+1
to
+2
| import os | ||
| import re |
Comment on lines
+13
to
+21
| # Blocks to entirely skip for the basic "quick-and-dirty" configuration | ||
| ADVANCED_BLOCKS = [ | ||
| "refinement_cleanup:", | ||
| "refinement_smoothing:", | ||
| "GeneGapFiller:", | ||
| "media_gap:", | ||
| "mcc:", | ||
| "egc:", | ||
| ] |
Comment on lines
117
to
+118
| ########## step 3.2: cleanup ########## | ||
| refinement_cleanup: | ||
|
|
||
| # gap-filling | ||
| # """"""""""" | ||
|
|
||
| # general options _________________________________________ | ||
| # parameters, that apply to all the gap filling algorithms | ||
| # current default means no gapfilling | ||
| formula-check: 'existence' # When checking, if a metabolite can be added to the model | ||
| # also check the formula. For more information about | ||
| # available options, please refer to the docs of | ||
| # the function isreaction_comlete(). | ||
|
|
||
| # enable algorithms ________________________________________ | ||
| GeneGapFiller: False # Activate gap filling via GFF | ||
| GeneGapFiller parameters: | ||
| fasta: USER # FASTA (containing CDS) for the GeneGapFiller. | ||
| # Depending on the case, might be the same as annotated genome. | ||
| gff: USER # Path to a gff file (does not have to be the RefSeq). | ||
| # Needs to be from the same genome the model was build on. | ||
| type: 'swissprot' # Type of database. Can either be 'swissprot' or 'user'. | ||
| dmnd-database: USER # Path to the SwissProt/User DIAMOND database file. | ||
| database-mapping: USER # Path to the SwissProt/User mapping file (against EC / BRENDA in case of SwissProt) | ||
| check-NCBI: False # Enable checking NCBI accession numbers for EC numbers - time costly. | ||
| sensitivity: 'more-sensitive' # Sensitivity option for the DIAMOND run. | ||
| coverage: 90.0 # Coverage (parameter for DIAMOND). | ||
| percentage identity: 90.0 # Percentage identity threshold value for accepting | ||
| # matches found by DIAMOND as homologous. | ||
| # via COBRApy - medium ........ | ||
| media_gap: null # path to a medium config file. Set it to null to skip this gapfiller | ||
| ########## step 3.3: annotation ########## |
Comment on lines
106
to
+109
| # also check the formula. For more information about | ||
| # available options, please refer to the docs of | ||
| # the function isreaction_comlete(). | ||
| exclude-dna: True # Exclude reactions containing 'DNA' in their name |
Comment on lines
101
to
+104
| sensitivity: more-sensitive # change the sensitivity of DIAMOND | ||
| coverage: 95.0 # Set the min. coverage | ||
| pid: 90.0 # set the minimal percentage identity value for valid hits | ||
| # params for adding entities during extension | ||
| pid: 90.0 # set the minimal percentage identify value for valid hits | ||
| # params for adding entities during |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR addresses Issue #24 to improve QoL for developers maintaining configuration files[cite: 8]. Instead of maintaining
basic,advanced, anddefaultYAMLs manually and risking inconsistencies, a Python script has been added to automate the generation process[cite: 8].Key Changes
scripts/sync_configs.py.# @IDEA,# @TODO,# @DEV)[cite: 7, 8].refinement_cleanup,GeneGapFiller,mcc) to create a minimal "quick-and-dirty" file[cite: 7, 8].Usage
Developers only need to edit
src/specimen/data/config/hqtb_config_default.yaml, then run:python scripts/sync_configs.pyfrom the root directory to safely overwrite and sync the advanced and basic configuration files.