Overview

This library provides reference implementations for parsing and normalizing sequence identifiers in the Smitten format. The format allows for the encoding of DNA sequence ranges and strand orientation to existing sequence identifiers. These ranges may be recursively defined by adding further subranges. This format has been in-use in many sequence analysis tools/scripts developed by Arian Smit to process Transposable Element sequences. The format has evolved over the years and now encompases two legacy version (V0/V1), and the current supported identifier format (V2).

Reference implementations are provided in Rust, Perl and Python in the rust, perl and python subdirectories. All three are at version 1.2.0.

Usage

Each implementation offers a format-detecting parser for legacy (V0/V1) and V2 identifiers, a strict V2 parser, and a normalizer that collapses chained ranges into a single range:

Operation	Rust	Perl	Python
Detect format, convert	Identifier::from_unknown_format()	convertID( id => ... )	parseID()
Strict legacy parse	Identifier::from_v0(), from_v1()
Strict V2 parse	Identifier::from_v2()	parseID()	parseV2ID()
Normalize (V2 only)	Identifier::normalize()	normalizeID()	normalizeID()

Normalization accepts V2 identifiers only; legacy identifiers must be converted to V2 first. For example:

hg38:chr1:100-200_-:10-20_+   normalizes to   hg38:chr1:181-191_-

Coordinates are 1-based and fully closed, so position 10 of the reversed 100-200 span is 200-10+1 = 191 and position 20 is 200-20+1 = 181.

Strict V2 parsing only treats trailing :start-end_+ / :start-end_- tokens as ranges. Legacy range text is therefore part of the sequence identifier: scaffold_164_5 and chr1_100_200 are bare names with no ranges, and normalize to themselves. Format detection instead reads _164_5 as a V0 range and rejects it, because V0 ranges must be increasing. Use the strict parser when identifiers are known to be V2.

Note: Python does not yet provide a function that converts a legacy identifier to a V2 string; parseID() returns its components.

Format Specification

There are three versions of the Smitten format that are supported by this library. Each format is described below first with a formal specification in Augmented Backus-Naur Form (ABNF), and then with examples.

V0 Format

ID = sequence_identifier *[ “_” start_position “_” end_position [“_R”]]
special_chars = ":" / “/” / “?” / “#” / “[“ / “]” / “@” / “!” / “$” / “&” / “’” / “(“ / “)”
                “*” / “+” / “,” / “;” / “=“ / “~” / “|” / “^” / “”” / “>” / “<“ / “.” / “%”
                "-" / "_"
sequence_identifier = 1*(ALPHA / DIGIT / special_chars)
start_position = 1*(DIGIT) ; 1-based sequence position
end_position = 1*(DIGIT) ; 1-based sequence position, fully closed coordinates

Examples:                          Meaning:
Chr1                               seq_id="Chr1", whole chromosome
1_10_30                            seq_id="1", from 10-30, forward strand     
seq1_50_100                        seq_id="seq1", from 50-100, forward strand
seq1_1_10_30                       seq_id="seq1_1", from 10-30, forward strand
seq1:2_10_30                       seq_id="seq1:2", from 10-30, forward strand
seq1_1_100_10_30_R                 seq_id="seq1", from 10-30 reverse strand of 1-100 forward strand
chr1_11023_38232_R_100_200         seq_id="chr1", from 100-200 forward strand of 11023-38232 reverse strand   
seq1_exon2_100_200_R               seq_id="seq1_exon2", from 100-200 reverse strand

Pitfalls: This format relies on the suffix sequence always containing a meaningful set of range/orientation tokens separated by a non-reserved underscore ("") character. Identifiers ending in a "#_#" pattern are indistinguishable from a subrange specification.

V1 Format

ID = sequence_identifier *[ “:” (forward_orient-range / reverse_orient-range) ]
special_chars = ":" / “/” / “?” / “#” / “[“ / “]” / “@” / “!” / “$” / “&” / “’” / “(“ / “)”
                “*” / “+” / “,” / “;” / “=“ / “~” / “|” / “^” / “”” / “>” / “<“ / “.” / “%”
                "-" / "_"
reverse_orient_range = upper_bound “-” lower_bound
forward_orient_range = lower_bound “-” upper_bound
lower_bound = 1*(DIGIT) ; 1-based sequence position
upper_bound = 1*(DIGIT) ; 1-based sequence position, fully closed coordinates

Examples:                          Meaning:
Chr1                               seq_id="Chr1", whole chromosome
Seq1:10-30                         seq_id="Seq1", from 10-30, forward strand
Seq1:30-10                         seq_id="Seq1", from 10-30, reverse strand
Seq1:100-200:10-30                 seq_id="Seq1", from 10-30 forward strand of 100-200 forward strand
Seq1:100-200:30-10                 seq_id="Seq1", from 10-30 reverse strand of 100-200 forward strand

Pitfalls: The use of range ordering to denote orientation is problematic in a 1-based coordinate system. The strand of a single base position range is ambiguous.

V2 Format

ID = [assembly_identifier “:”] sequence_identifier *[“:” start_position “-” end_position “_” orient]
special_chars = “/” / “?” / “#” / “[“ / “]” / “@” / “!” / “$” / “&” / “’” / “(“ / “)”
                “*” / “+” / “,” / “;” / “=“ / “~” / “|” / “^” / “”” / “>” / “<“ / “.” / “%”
                "-" / "_"
orient = “+” / “-”
assembly_identifier = *(ALPHA / DIGIT / special_chars )
sequence_identifier = 1*(ALPHA / DIGIT / special_chars )
start_position = 1*(DIGIT) ; 1-based sequence position
end_position = 1*(DIGIT) ; 1-based sequence position, fully closed coordinates

Examples:                          Meaning:
Chr1                               seq_id="Chr1", whole chromosome
seq_1:30-40_+                      seq_id="seq_1", from 30-40, forward strand
Seq1;contig4:100-103_-             seq_id="Seq1;contig4", from 100-103, reverse strand
chr1:11023-38232_-:100-200_+       seq_id="chr1", from 100-200 forward strand of 11023-38232 reverse strand
hg38:chr1                          assembly_id="hg38", seq_id="chr1", whole chromosome

Changes

1.2.0

• Version numbers are now aligned across the GitHub release tags and all three implementations. The earlier tags v1.0.0-alpha, v1.0.1-alpha and 1.0.2 correspond to internal versions 0.1.0, 0.1.0 and 0.1.1.
• Stricter validation in all implementations: whitespace, empty assembly identifiers, V2 ranges without an orientation, and sub-ranges outside their parent range are rejected.
• Rust: from_v0() and from_v1() reject identifiers in other formats, and the detected version is stored on the returned Identifier.
• Fixed normalizeID orientation for forward-on-forward (+/+) sub-ranges in Perl and Rust, which were reported on the reverse strand.
• Python: added parseV2ID(); normalizeID() now accepts V2 identifiers only.
• Python: parseID() classifies each range independently, rejects mixed formats, and supports the :start-end_R variant.
• Rust: regular expressions are compiled once per process.

-Robert Hubley, 2022-2026