Repository navigation
Preserve Gemini speaker turns in mixed-audio transcripts - #8127
ilyakarakotov wants to merge 1 commit into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| const legacyGeminiMix = | ||
| transcript.source === "batch_transcription" && | ||
| transcript.provider === "google_generative_ai" && | ||
| words.length > 0 && | ||
| words.every((word) => word.channel === 0) && | ||
| !(transcript.speaker_hints ?? []).some((hint) => { | ||
| const value = parseHintValue(hint.value); | ||
| return ( | ||
| hint.type === "provider_speaker_index" && | ||
| value != null && | ||
| typeof value === "object" && | ||
| (value as { source_channel_restored?: unknown }) | ||
| .source_channel_restored === true | ||
| ); | ||
| }); | ||
| if (legacyGeminiMix) { | ||
| for (const word of words) word.channel = 2; |
There was a problem hiding this comment.
🔴 Legacy speaker turns remain collapsed in summaries
For saved Gemini batches, renderSessionTranscript still treats channel-zero words as personal-mic audio. Its native renderer bypasses this repair, so summaries and chat still lose distinct speaker turns.
Learn more
The desktop renderer now repairs channel-zero words from older Gemini batch transcripts before invoking the shared segment renderer. render_session_transcript instead reads saved words and hints directly and calls render_input_from_stored without checking the transcript's source or provider. buildTranscript consumes that native result for chat; summary generation calls the same command. render_transcripts_markdown also renders stored words without this repair. The UI and these consumers therefore disagree on the same transcript.
Example: A saved Gemini batch contains Hello on channel 0 with speaker 0 and Hi on channel 0 with speaker 1. The desktop view repairs both to mixed capture, but chat and summary requests still identify the channel as DirectMic and can attribute both turns to the owner.
Recommended fix: Share the legacy normalization across native rendering and desktop rendering, preserving verified source-channel restorations and translating channel-scoped assignments. Pass source/provider through the native session-render query and cover its rendered segments with a saved legacy Gemini fixture; apply the same normalization to agent-access exports.
Was this helpful? React with 👍 or 👎 to provide feedback.
| if let Some(mut value) = hint | ||
| .value | ||
| .as_str() | ||
| .and_then(|value| serde_json::from_str::<serde_json::Value>(value).ok()) | ||
| && value.get("channel").and_then(serde_json::Value::as_f64) == Some(0.0) | ||
| { | ||
| value["channel"] = serde_json::json!(2); | ||
| hint.value = serde_json::Value::String(value.to_string()); |
There was a problem hiding this comment.
🟡 Object-valued hints undo mixed-channel repair
When legacy Gemini hints contain JSON objects, materialize_source_row changes words to channel 2 but leaves hint channels at 0. Refinement reapplies those hints, so speaker assignments can be lost during retranscription.
Learn more
Stored speaker hints can hold either a JSON string or a JSON object; parse_stored_speaker_hints accepts both. This branch rewrites the channel only when hint.value.as_str() succeeds. An object-valued provider hint stays on channel 0 while its associated word changes to 2. Later, normalize_speaker_hint overwrites the normalized word channel with the unchanged provider hint; explicit user scopes also remain on channel 0.
Example: An imported legacy Gemini word on channel 0 has an object-valued provider hint { "channel": 0, "speaker_index": 1 }. Before refinement the word becomes channel 2, but the unchanged hint puts it back on channel 0 when speaker labels are reconciled.
Recommended fix: Parse both object and string hint values when detecting restoration markers and remapping channels. Preserve the existing value representation if needed, and test a source row with object-valued provider and user hints.
Was this helpful? React with 👍 or 👎 to provide feedback.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3629f2244d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| hint.type === "provider_speaker_index" && | ||
| value != null && | ||
| typeof value === "object" && | ||
| (value as { source_channel_restored?: unknown }) | ||
| .source_channel_restored === true |
There was a problem hiding this comment.
When a current_capture Gemini refinement returns timestamped words but no speaker labels, restore_refined_source_channels can legitimately restore every word to channel 0, but restore_refined_source_hints has no provider_speaker_index hint on which to write source_channel_restored. This predicate therefore reclassifies the saved words as MixedCapture on the next render (and materialize_source_row repeats it before a later refinement), undoing the verified DirectMic restoration and owner attribution. Persist this provenance independently of optional diarization hints and add a regression for the no-label current-capture path.
AGENTS.md reference: AGENTS.md:L95-L95
Useful? React with 👍 / 👎.
Summary
Intent: Gemini batch words were treated as personal-microphone audio, collapsing distinct speakers into the owner. Preserve speaker turns for new recordings and recover saved Gemini transcripts without retranscription.
Preserve manual speaker assignments and verified source-channel restoration. Normalize legacy source transcripts before batch refinement so retranscription cannot restore the same incorrect personal-microphone attribution.
Demo
Synthetic reproduction:
Speaker 0: Hello.followed bySpeaker 1: Hi.previously rendered as a single owner block on channel 0. The corrected render input uses mixed capture and retains both speaker labels. Regression tests also check that live microphone channels remain unchanged and manual speaker assignments survive.Verification
Summary by cubic
Fixes Gemini batch transcripts being treated as personal-microphone audio, which collapsed all speaker turns into a single owner block and lost speaker labels.
Written for commit 3629f22. Summary will update on new commits.