Audit date: 2026-09-28. This is a diagnostic review of the current LGU model and source files. It does not replace a prospective live-signer evaluation. No trained weights, labels, thresholds, or dataset files were changed by this audit.
The new indicator reads the state of RealtimeEngine; it does not alter the
visual class scores or acceptance rules. The offscreen GUI smoke check confirms
that an accepted label still reaches TranscriptAssembler and appears in both
windows after the output-settling timer. We cannot infer from that check alone
that a particular webcam attempt was accepted.
The FSL video loader trains on complete source
videos. It extracts every readable frame and resamples the whole recording
to (30, 162). This includes any neutral lead-in and ending in the video.
The live worker instead starts collecting
only while hand landmarks are detected, clears its 30-frame buffer whenever
hands disappear, and runs the classifier on a rolling window. It then requires
a score of at least 0.70, three matching observations (or four outside a
supported context branch), and a 0.7-second emission cooldown. Training and
live inference therefore present different temporal portions of a sign to
the same model. An indicator can truthfully show hand detection while no label
passes these checks.
python evaluation/replay_fsl_live.py compares the full-video and current
rolling-window paths on the first listed clip of each requested label. A
replay of four training-manifest clips, without webcam mirroring or queue
drops, produced:
| Intended label | Full-video top label (score) | Live-path accepted labels |
|---|---|---|
NEW |
NEW (0.753) |
None |
PWD ID |
PWD ID (0.999) |
PHOTOCOPY (0.883), then FS_X (0.760) |
WHERE |
WHERE (0.903) |
None |
FS_A |
FS_A (0.998) |
FS_A (0.994) |
These are diagnostic examples, not independent accuracy estimates. The
replay directly processes source frames; a real webcam may drop queued frames,
mirror the image, have different lighting, and show a different signer. It
nonetheless demonstrates a concrete path mismatch. Changing the indicator
text or the 1.2-second display delay would not repair missing model
acceptances. A quick fixed idle-frame padding was also tried on these four
clips; it helped some labels but reduced NEW confidence below 0.70, so it
is not a validated drop-in fix.
python evaluation/audit_fsl_sources.py recomputes folder and manifest counts
and hashes byte-identical sources. Selected classes currently show:
| Label | Files in class folder | Rows in original train.csv |
Rows in original test.csv |
Rows used by LGU model |
|---|---|---|---|---|
NEW |
20 | 6 | 1 | 6 |
PWD ID |
20 | 14 | 4 | 14 |
WHERE |
20 | 12 | 3 | 12 |
FS_A |
20 | 14 | 4 | 14 |
FS_V |
20 | 12 | 3 | 12 |
Folders include files not referenced by either CSV. Their label, sign quality,
and provenance must be reviewed before they can be added; files in the
original test.csv must not be silently moved into training. A folder count
is not a count of independent usable training examples.
The reduced LGU manifest has 929 rows. Content hashing found 15
byte-identical groups spanning 36 rows. Eight groups, spanning 21 rows,
cross the saved LGU train/validation/test split. Their labels are CHANGE,
LAST, OFFICE, PWD ID, and WHEN. The PWD ID folder has 20 files but
only 9 unique SHA-256 contents; among its 14 LGU rows, 7 unique contents
remain. An exact copy of a training video is present in the LGU test split.
This is source-content leakage despite distinct file paths.
The original train.csv and test.csv together have 2,953 rows, of which
2,951 paths exist. Ten exact-content groups cross those original manifests;
four duplicate groups even carry conflicting labels somewhere in the original
corpus. Therefore the original test.csv should also be content-audited
before use as an independent benchmark. The previously reported 92.86%
LGU held-out source-path score was computed by the recorded pipeline, but
the split is not content-disjoint. Its possible inflation has not been
quantified; do not present it as clean independent performance. None of these
checks establishes signer independence because verified signer IDs are absent.
Do not restart all recordings or force every video to five seconds. The
929 videos listed in the current LGU manifest have a median metadata duration
of 4.07 seconds; the 10th and 90th percentiles are 3.63 and 6.02
seconds. The first six listed NEW clips have median duration 5.70 seconds,
and the 14 PWD ID clips have median 7.56 seconds. A hard five-second cutoff
could truncate a full sign. The duration of a clip is not the duration of
useful sign motion: several clips contain a long neutral lead-in.
A two-second pause between separate recording attempts is reasonable as an operator cue, but it is not a model requirement. Do not automatically include two seconds of waiting at the start of every five-second training clip: that would devote 40% of its time to a neutral pose and leave too little room for a complete sign plus an ending. More importantly, the current live worker discards no-hand frames, so adding a fixed neutral lead-in to training cannot by itself make the live input match training.
Proposed pilot protocol for new source clips, subject to review with an FSL validator:
- Use the same camera orientation and landmark framing planned for live use. Keep both hands and upper body visible through the sign.
- Give the signer a cue before the clip. Record a brief neutral beginning, one complete sign at a natural pace, and a brief neutral ending. Save the entire motion without forcing a fixed total duration; trim excess waiting only after confirming the sign boundaries.
- Record genuinely distinct performances from multiple identified signers, with documented camera, lighting, and distance variations. Record a pseudonymous signer ID and source ID for every clip. Have an FSL-competent reviewer confirm the intended label, especially for fingerspelling and visually similar signs.
- Preserve the original clip and record any trim boundaries in metadata. Do not count copied files, augmented versions, or frames from one source as independent recordings.
- First pilot a few classes that show live misses or confusion (
NEW,WHERE,PWD ID,FS_V). Validate the capture-to-model pipeline on new signers before scaling collection to all 70 classes.
The next coordinated model iteration should use one documented temporal segmentation rule for both offline training and live inference. One option is to detect sign onset/end and resample that bounded segment; another is an explicit capture control per sign. The rule needs a validation set with negative/rest clips so it does not accept transition movements as words. After correcting duplicate and label conflicts, make group-disjoint splits by verified signer where feasible; at minimum, keep every identical source content in one split. Then retrain the FSL model, calibrate confidence and stability on validation data, and report untouched test and live-conversation results separately. Do not lower 0.70 or the streak count merely to make a pilot case pass; false acceptances must be measured too.
This recommendation follows the code's train/live mismatch and the audit, not a claim that one clip length guarantees accuracy. Published sign-language recognition work emphasizes signer variation and signer-independent testing (Artiaga et al., 2025; Papadimitriou et al., 2022). MediaPipe's video API processes timestamped frames rather than treating a whole clip as one image (Google Hand Landmarker API).
- Run from
C:\Users\CJ\ThesiswithSIGNBRIDGE_FSL_PROFILE=lguset in PowerShell and confirm the workspace reports Model loaded. Press Start Signing and wait for GO. - If ready remains while hands are in view, check the MediaPipe hand overlay, camera orientation, lighting, and which checkout is running.
- If detecting sign appears but no preparation notice appears, no label passed the visual acceptance checks. Log the attempted label, signer, lighting, and no-output result. The indicator is not a successful recognition event.
- If cooldown or the preparation notice appears but text never publishes, that points to a GUI/output problem; record the exact status and run the offscreen smoke check.
The comparison commands are read-only:
cd C:\Users\CJ\Thesis
python evaluation/audit_fsl_sources.py
python evaluation/replay_fsl_live.py NEW "PWD ID" WHERE FS_A
$env:SIGNBRIDGE_FSL_PROFILE = "lgu"
python tests/smoke_transaction_gui.pyThese commands do not improve the current weights. Changing collection, segmentation, splits, thresholds, or evaluation claims changes the thesis methodology and requires a new documented experiment.