Skip to content
Dfam-consortiumPublic

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

70 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

TE_Idx

Usage

  • --data-dir : (Optional, usually for testing) path to prepared data
  • --exp-dir : (Optional, usually for testing) path to source data files
  • --assembly : Name of assembly/assembly folder
  • command : see below

Commands

bgzf-filter

This function reads through a compressed BED file and produces another BED file based on the filter parameters. Used as a subroutine for ReadFamilyAssemblyAnnotations.

  • --data-type : Type of data to be searched (Assembly -> assembly_alignments, Benchmarks -> benchmark_alignments, Simple Repeats -> masks)
  • --fam : Family name, corresponds to compressed TSV file prefix. For Simple Repeats, use the sequence ID
  • --position : number corresponding to the search field (column), 1-indexed
  • --term : (Optional) Term to be searched for. If absent, all rows will be returned
  • --outfile : (Optional )Path to file to save filtered data. Should end in .bed.bgz, defaults to stdout
  • --web-fmt : (Optional) Flag to reformat the feild order to match Dfam.org download file format

build-idx

This function generates a <data type>_idx.dat file of the relevant data type for a given assembly. Used as a subroutine for Prepare-Assembly.

  • --data-type : Type of data to index (Assembly -> assembly_alignments, Benchmarks -> benchmark_alignments, Simple Repeats -> masks)

prep-beds

Splits the TSV files generated by buildFullRegion.py into BED files by sequence. Used as a subroutine for Prepare-Assembly.

  • --in-tsv : Path to input TSV file, should be in sequence ID order, or at least grouped by sequence ID.
  • --data-type : The type of data being indexed (Assembly -> assembly_alignments, Benchmarks -> benchmark_alignments, Simple Repeats -> masks)

prepare-assembly

This command checks for the presence of export files and prepared data files related to the given assembly. It will plan to prep each data type if exports exist and prepared data files do not. It will then print out the plan, create whatever folders it needs, and begin preparation. Preparation for TSV files is a two step process of prep-beds followed by build-idx for each type, while JSON data files are just copied into the appropriate folder.

idx-query

This command is used directly by the API and returns a JSON of indexed hits.

  • --data-type : The type of data being indexed (Assembly -> assembly_alignments, Benchmarks -> benchmark_alignments, Simple Repeats -> masks)
  • --chrom : Sequence ID
  • --start :Start position
  • --end : End position
  • --family : (Optional): Only return hits matching accession
  • --nrph : Only return NRPH hits

json-query

This command is used both as a subcommand for bgzf-filter for finding sequence and model lengths, and directly by the API to test if sequences are valid members of an assembly. If a key:target pair is provided, it will return the target value of key entity. If only key is supplied, it will return 1 if key is found, and -1 if not.

  • --data-type : The type of data being indexed (Sequences -> sequences, Model Lengths -> model_lengths)
  • --key : Key value to search by, such as a sequence ID or family accession
  • --target : (Optional) Target attribute to return, such as sequence length or family model length

read-family-assembly-annotations

This function is used directly by the API. It uses bgzf-filter to return Assembly Alignments of a specified family from the given assembly. The only option is to filter by NRPH or not.

  • --id : Family Accession
  • --nrph : (Optional) Only Return NRPH hits
  • --outfile : (Optional) Output file, used for testing

Export Sources

  • hg38-byacc-bench_region.tsv -> buildFullRegion.py
  • hg38-byacc-full_region.tsv -> buildFullRegion.py
  • hg38-mask.tsv -> importNewAssembly.py
  • hg38-model_lengths.json -> importNewAssembly.py
  • hg38-sequence.json -> importNewAssembly.py

BED Files

The Data Types stored in BED files are Assembly Annotations, Benchmark Annotations, and Simple Repeats.

Column Order For Assembly And Benchmark Annotations

Base Order Download Format Order
1. seq_id 1. sequence name
2. seq_start 2. model accession
3. seq_end 3. model name
4. family_accession 4. bit score
5. hit_bit_score 5. e-value
6. strand 6. hmm start
7. bias 7. hmm end
8. ali_start 8. hmm length
9. ali_end 9. strand
10. model_start 10. alignment start
11. model_end 11. alignment end
12. hit_evalue_score 12. envelope start
13. nrph_hit 13. envelope end
14. divergence 14. sequence length
15. *family_name
16. seq_len
17. *cigar
18. *caf

Mask File Column Order

  1. seq_acc
  2. seq_start
  3. seq_end
  4. repeat_str
  5. repeat_length

Testing

cargo test

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages