Skip to content

feat(data-collections): tag samples from a metadata sheet during import#840

Open
diegomayorga-dept wants to merge 1 commit into
developmentfrom
feat/egv-215-tags-from-spreedsheet
Open

feat(data-collections): tag samples from a metadata sheet during import#840
diegomayorga-dept wants to merge 1 commit into
developmentfrom
feat/egv-215-tags-from-spreedsheet

Conversation

@diegomayorga-dept

Copy link
Copy Markdown
Contributor

Title*

[EGV-215] Tag samples from a metadata sheet during import

Type of Change*

  • New feature
  • Bug fix
  • Documentation update
  • Refactoring
  • Hotfix
  • Security patch
  • UI/UX improvement

Description

Adds a new, optional "Tags" step to the sample-import wizard (EGImportDataWizard.vue), between "Group files" and "Review samples", so a lab tech can tag a batch of samples from their own metadata sheet at import time instead of tagging one-by-one afterward.

Flow: upload a sheet → map which column holds the sample name and which holds the tag(s) → rows are joined to the file-derived proposed samples by normalized name → the preview shows tags to be applied/created, per-tag sample counts, typo warnings, and two unmatched reports (sheet rows with no matching sample, samples with no sheet row) → on confirm, any missing tags are created (createTag) and the resolved TagIds are passed into the existing bulkCreateSamples request's per-sample TagIds field.

Notably, this required no new backend endpoint or routebulkCreateSamples already accepted and applied a per-sample TagIds array (laboratory-sample-service.ts), but nothing in the wizard populated it. This PR wires that existing path up from the UI.

New files:

  • shared-lib/src/app/utils/delimited-text.ts — small RFC-4180-style CSV/TSV parser (quoted fields, escaped quotes, CRLF/LF, trailing blank lines). No parsing library was in use anywhere in the repo, and the existing file.text() + split convention can't handle quoted commas, so this is new but intentionally minimal.
  • front-end/src/app/utils/levenshtein.ts — edit-distance helper for "possible typo" warnings.
  • front-end/src/app/utils/sheet-tag-matching.ts — the core pure logic: name-matching, tag-cell parsing/dedup, existing-vs-new tag resolution, rejecting workflow/batch/permanent-kind tags with a per-tag reason, and typo detection.

Modified:

  • EGImportDataWizard.vue — new step, file upload + column-mapping UI, preview panel, and wiring resolveSampleTagIds() into confirmImport(). Inserting the step renumbered the two steps after it (Review samples 3→4, Confirm 4→5); all step-index references (watch(step, ...), template v-else-ifs, footer nav buttons) were updated accordingly.
  • laboratory-data-tagging-service.ts — added a permanent-kind guard to applyTagsToSamples (add and remove loops). The service already rejected workflow and batch tags on samples; permanent was not blocked. Defense-in-depth: the front-end also excludes permanent tags from creation/application in the preview.
  • EGSamplesTab.vue, EGSampleTagSidebar.vue — refactored to use a new shared TAG_PRESET_COLORS/DEFAULT_TAG_COLOR constant (shared-lib/constants/data-collections.ts) instead of each keeping its own copy of the same hardcoded palette. This PR's new "create missing tag" code is a third consumer of that palette, so it was extracted per the project's DRY rule. No visible behavior change — same colors, same default.

Scope decisions (from ticket grooming — see design doc): samples only (files deferred), import-time only (a "reconcile already-imported samples" dialog is deferred to reuse the same matching util), create-then-apply for unmatched tags (no forced vocabulary), and no saved/named column-mapping presets in this PR (deferred as a fast-follow).

Testing*

Automated (all run and passing):

  • delimited-text.test.ts — 7/7 passing (quoted commas, escaped quotes, CRLF/LF, trailing blank lines, tab delimiter).
  • levenshtein.test.ts — 5/5 passing.
  • sheet-tag-matching.test.ts — 8/8 passing, including a constructed case with tags of every rejected kind (workflow/batch/permanent) mixed with an accepted tag on the same sample.
  • laboratory-data-tagging-service.test.ts — new permanent-guard test passing; full file re-run 39/39, no regressions.
  • These suites were run during implementation and independently re-run during code review (values above are from that independent run, not just the implementer's report).

Manual (performed against the local dev server + real AWS, in this test lab):

  • Happy path: imported a real paired-end sample, uploaded a sheet with a quoted multi-tag cell ("E. coli, Needs Review") and one row naming a non-existent sample, mapped columns, confirmed the preview classified both tags correctly and reported the extra row as unmatched, completed the import, and verified on the Samples tab that the sample was persisted with both tags.
  • Malformed sheet: an empty file and a header-only file both produced the expected "needs a header row and at least one data row" message without blocking the (optional) step.
  • Batch-tag rejection: a sheet cell naming an existing batch tag was correctly shown under "Skipped — Batch tags are not supported on samples", not applied or offered for creation.
  • Not tested live: workflow/permanent-kind tag rejection — this test lab had no pre-existing tag of either kind, so a sheet cell naming one would just be treated as a new tag rather than exercising the guard. That exact branch (including the mixed accepted/rejected case) is covered by the unit test above.
  • Full test log: superpowers/2026-07-10-bulk-tag-from-sheet-during-import-test-log.md (kept outside the repo per project convention).

Not done:

  • No Playwright E2E spec was added. EG's E2E suite runs against a deployed quality environment, not locally, so none was run as part of this change. A follow-up spec mirroring the manual happy-path steps is noted as a TODO in the test log — no existing E2E coverage of the import wizard was found to extend from.
  • Lint on the modified .vue files could not be run in the implementation environment (missing parserOptions.extraFileExtensions for .vue) — confirmed as a pre-existing gap reproducible on untouched .vue files, and the repo's actual lint script only targets .ts/.js anyway, so this is not a gap introduced by this PR.

Impact

  • Front-end only for the new "Tags" step and the palette refactor; shared-lib gains two small, dependency-free pure utilities; back-end gets a 2-line defensive guard, no schema or route change.
  • No new npm dependencies (the CSV parser is hand-written, per the ticket's explicit constraint against adding a library).
  • No new network calls beyond what already existed for tagging: on confirm, up to N sequential createTag calls (N = distinct new tag names in the sheet, not sample count), then the existing single bulkCreateSamples call — no new endpoint, no new chunking logic.
  • Behavior change: EGSamplesTab.vue and EGSampleTagSidebar.vue now import their preset colors from a shared constant instead of a local literal array — same values, no visible change.
  • No DynamoDB schema change, no new IaC/CDK resources.

Additional Information

This PR resulted from reconciling an initial ticket draft (fixed Identifier,TagNames CSV, match-existing-only, new backend endpoint) against actual grooming feedback, which asked for format-agnostic column mapping with tag creation and a preview — a materially different and smaller-backend design. The full design rationale, the corrected technical premises, the implementation plan, and the manual test log are kept outside the repo under superpowers/ per this project's CLAUDE.md convention (2026-07-10-bulk-tag-from-sheet-during-import-{design,plan,test-log}.md).

Known pre-existing issues surfaced during manual testing, out of scope here: the default file-grouping regex preset doesn't match filenames with a trailing underscore before the extension (worked around via the wizard's existing editable regex field); and the wizard's auto-generated batch name (upload-<date>) can collide with a batch tag already created earlier the same day, throwing a 409 from setBatchForSamples (per-sample tag application happens earlier in the same request, so this does not prevent tags from being applied).

Checklist*

  • (lint on the modified .vue files could not be verified in the implementation sandbox — pre-existing environment gap, not introduced here; the repo's own lint script doesn't target .vue files)
  • All tests pass successfully and new tests added as necessary.
  • (N/A — no user-facing docs affected; design/plan/test-log kept outside the repo per project convention, not repo documentation)
  • Code adheres to the coding and style guidelines of the project.
  • Code has been commented in particularly hard-to-understand areas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant