fix: retain unpunctuated tail in sentence timestamps

[Arthur031221]

Summary

Users combining token timestamps with online punctuation can get an empty sentence_info for a nonempty transcript when the punctuation model leaves the last token unpunctuated. Both fallback timestamp splitters emitted a segment only after a punctuation ID greater than 1. They now emit a remaining segment when its timestamps are present. The existing surface-text path already emits such a tail.

Type of change

• Bug fix
• Documentation
• Example or demo
• Runtime or deployment
• Benchmark or evaluation
• Model/training change

Validation

• .venv/bin/python -m pytest -q tests/test_timestamp_tools.py tests/test_punc_model_none.py: 25 passed, 8 subtests passed. Restoring timestamp_tools.py from the base commit makes the three new tail cases fail; 22 tests and 8 subtests pass.
• .venv/bin/python -m compileall -q funasr examples tests: exit 0.
• I ran the released iic/punc_ct-transformer_zh-cn-common-vad_realtime-vocab272727 checkpoint on CPU with input hello world. It returned text hello world and punctuation IDs [1, 1]. Feeding that output with fixed ASR timestamps [[0, 100], [100, 300]] through AutoModel.inference_with_vad returned sentence_info=[] on the base source and one hello world sentence spanning 0 to 300 ms with the fix. The existing surface-text splitter returned that same sentence for these values. The ASR and VAD outputs in this check were stubs; I did not run ASR weights.
• The full suite did not finish within a 300-second limit in this environment.

User impact

The tested online punctuation checkpoint can leave a final token with ID 1. When a timestamped ASR result has no word list, AutoModel takes the fallback splitter and previously lost that token from sentence_info. The common ct-punc alias uses a different, offline model; I tested the online checkpoint named above. I measured this input, not how often it occurs in other transcripts.

Notes for reviewers

The new segment is omitted if its timestamp is missing. The change is limited to the fallback timestamp splitters. Completed sentences and top-level ASR text were unchanged in the fixed-input tests.

[LauraGPT]

Independently checked exact head 162ef07eb94fddd2ec0adca4892a6108bed46138 against base 14f12f1dc04418c3a2cd28e29623d1ad810c6d35. No blocking correctness finding in this scoped review.

• Ran the submitted timestamp and AutoModel.inference_with_vad tests against the base implementation: 3 expected failures, 22 passed, 8 subtests passed. The failures reproduce the missing unpunctuated Chinese/English tail and the empty sentence list through the fallback integration path.
• Same submitted tests on the PR implementation: 25 passed, 8 subtests passed.
• An independent deterministic matrix exercised both splitters, raw-text on/off, one to four tokens, and all no-punctuation/period combinations: 120 cases passed. Completed sentence dictionaries remained identical to base; valid token timestamps were preserved exactly once with correct boundaries; raw token separators were retained.
• The submitted missing-tail-timestamp cases also pass. This does not establish correctness for every malformed timestamp payload.

This was CPU-only source/test validation with recording/mocked model outputs; I did not download or run punctuation/ASR weights, independently reproduce the reported real-checkpoint experiment, or run the full repository suite. The current head had no hosted check runs at this review checkpoint, which is not a hosted-CI pass. This is a comment review, not approval or a merge.

[LauraGPT]

Reviewed exact head 162ef07 against current main d98f6ee. The timestamp and AutoModel punctuation suites pass (25 tests and 8 subtests); the combined candidate containing #3758 and #3761 also passes 128 tests and 10 subtests. The earlier independent 120-case splitter review remains applicable to the unchanged implementation. Unpunctuated tails retain their timestamps; incomplete timestamp tails remain omitted and completed prefixes remain unchanged. No blocking findings. This is CPU/parser and recording-double evidence, not checkpoint inference or an accuracy benchmark. No hosted workflow was triggered for this PR.