Repository-level data-table linting for data-manager / reference-data bundles

[jmchilton]

Implements the repository-level lint block proposed in galaxyproject/planemo#1672:
validates a data-manager / reference-data repository as a connected bundle
(data_manager_conf.xml + tool_data_table_conf.xml* + .loc/.loc.sample
fixtures + consumer from_data_table references), rather than only the individual
tool wrappers.

Adds eight linters, a standalone galaxy-tool-data-lint <repo> CLI, and wiring
into Planemo shed_lint. The linters run cleanly across the entire tools-iuc tree
(780 repositories) and surfaced real bugs, each fixed in its own PR. This
consolidates several long-standing lint requests: galaxyproject/planemo#96,
galaxyproject/planemo#531, galaxyproject/planemo#706, galaxyproject/planemo#869.

Architecture

Discovery and analysis are split from linting. build_repository_data_tables(repo_root, ...)
assembles a single RepositoryDataTables model — the realized bundle — from the
manager configs, table configs, loc assets, and expanded consumer tool sources; the
linters are then pure functions over that model and never touch the filesystem. This
keeps each check small and independently testable, and lets one traversal feed every
linter.

Key model facts the linters key on:

• Loc resolution is tri-state per referenced asset, not a boolean. found is what
  Galaxy's own loader resolves; is_sample marks a production loc satisfied only via
  the loader's .sample fallback; sample_backed recognizes the real Tool Shed layout
  (tool-data/<name>.loc.sample, materialized on install) that the loader's fallback
  misses. Only the combination distinguishes a genuinely missing fixture from a
  reference-data repo that is correctly shipped — without it every such repo would be
  falsely flagged.
• Assembly is defensive. A table declared with conflicting schemas would make the
  loader's merge raise, so assembly falls back to raw declarations and records the
  conflict as data rather than crashing; ConflictingTableSchema then reports it.
• Evidence is graded. Only literal, fully macro-expanded names are checked;
  unresolved dynamic/@TOKEN@ references are treated as "not checked," never as
  demonstrably missing (the Data manager update for metaphlan tools-iuc#5003 false-positive guard).

Each linter is dispatched through LintContext.lint(name, ...), so every check is
individually skippable by name the same way Planemo drives the tool linters.
Severity is deliberate: a broken producer/consumer contract provable from static
evidence is an error; a condition that a legitimately external supplier could
explain is a warning (see ConsumerTableDefined). The same model drives all three
entry points — lint_repository_data_tables (model in hand),
lint_repository_data_tables_bundle (caller did discovery), and
find_and_lint_repository_data_tables (one-call discover-and-lint, used by the CLI and
shed_lint).

Linters

Linter	Severity	Checks
LocRowShape	error	loc rows use the configured separator and can fill every declared column
MissingLocFixture	error	a configured table's referenced loc resolves to a file or a shipped sample
ManagerTableConfigured	error	every data-manager table has a matching tool_data_table_conf entry
OutputRefValid	error	each output_ref names a real output of the expanded manager wrapper
DuplicateColumnNames	error	no table declares the same column name twice
ConflictingTableSchema	error	a table is declared with one consistent schema across the repo
ConsumerTableDefined	warning	a consumed from_data_table table is defined locally or supplied externally
EmptyLocFile	warning	a .loc / .loc.sample that is empty carries a format comment

All eight pass across every tools-iuc repository that declares a data-table bundle;
the row, fixture, consumer, and empty-loc linters found real defects.

LocRowShape and MissingLocFixture — proven against real repositories

LocRowShape flags space- instead of TAB-delimited rows and truncated rows that
cannot fill the declared columns: galaxyproject/tools-iuc#8234 (checkm2 data manager),
galaxyproject/tools-iuc#8235 (dram), galaxyproject/tools-iuc#8237 (tools/checkm2),
galaxyproject/tools-iuc#8239 (coverm), galaxyproject/tools-iuc#8240 (vsnp_genbank).
MissingLocFixture flags a configured table whose loc resolves to nothing and no
shipped .sample backs: galaxyproject/tools-iuc#8236 (hisat2 index builder — a stray
bwa_mem_indexes table). Loc resolution honours the shed layout
(tool-data/<name>.loc.sample, materialized on install) so real reference-data
repositories are not falsely flagged. Addresses galaxyproject/planemo#96.

ManagerTableConfigured / OutputRefValid / DuplicateColumnNames / ConflictingTableSchema — structural

Guard rarer structural misconfigurations. All 285 bundle repositories in tools-iuc
pass them (verified by running the linters over the current tree); each is covered by
unit fixtures. ManagerTableConfigured addresses galaxyproject/planemo#706 and
galaxyproject/planemo#531; the remaining three are defensive checks with no live
tools-iuc incident, exercised only by fixtures.
Representative repositories they validate cleanly: data_manager_fetch_genome_dbkeys_all_fasta
(a multi-table producer of all_fasta + __dbkeys__), data_manager_bwa_mem2_index_builder,
and data_manager_star_index_builder (index builders whose output_ref columns and
configured tables all resolve).

ConsumerTableDefined — undefined table references

A warning, never an error: a consumed table may legitimately be supplied by Galaxy
core or another installed repository, so an undefined reference is advisory. The
common core tables (all_fasta, fasta_indexes, __dbkeys__) are treated as
externally supplied by default (DEFAULT_EXTERNAL_TABLE_NAMES); callers extend the
set via external_table_names. Across tools-iuc it flagged one genuine
inconsistency, fixed in galaxyproject/tools-iuc#8255: tools/artic consumed from_data_table="clair3_models"
but declared that table locally as model, shipped tool-data/models.loc.sample
while the config pointed at model.loc, and wired model to
test-data/clair3_models.loc, so its cached-model selector had no local data.

EmptyLocFile — undocumented empty fixtures (galaxyproject/planemo#869)

A warning: a .loc / .loc.sample that is empty and carries no # comment gives
no hint of its expected columns — the remedy is a one-line header, not data, so a
documented-but-dataless fixture passes. Unlike the configured-table linters this
reads a fresh repository-tree walk (model.loc_files) rather than the loader's
resolved loc_assets: an empty .loc.sample never resolves into an asset (the
loader's .sample fallback misses the shed tool-data/ layout), so a walk is the
only way to reach them. Across tools-iuc this flagged 32 header-less files
(14 .loc.sample, 18 test-data/*.loc) out of 660; the corpus cleanup landed in
galaxyproject/tools-iuc#8256, and the 384 comment-only files already passed.

Design decisions

• ConsumerTableDefined only walks from_data_table references in repositories
  that ship a bundle (a data-manager or tool-data-table config); pure-consumer tool
  repositories are deliberately left uninspected. A pure consumer's table is by
  definition supplied elsewhere, so flagging it as "not defined locally" would warn
  on nearly every tool repo in the shed — noise, not a defect — and loading every
  wrapper only earns its cost when there is a local bundle to validate against. This
  is a sensible boundary for the initial pass; it can be revisited once real usage
  shows whether cross-repo consumer validation (against a supplier index) is wanted.

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.

Unit coverage: test/unit/tool_util/data/test_repository_data_table_lint.py,
test_repository_data_tables.py, and test_data_lint_cli.py, driven by real fixture
repositories under test/unit/tool_util/data/repositories/. The linters were also run
over the full current tools-iuc tree (285 bundle repositories) with no crashes.

License

• I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

---

🤖 Generated with Claude Code