Fix VibeVoice-ASR streaming via generate() and server mapping#483
Open
JM27616 wants to merge 3 commits into
Open
Fix VibeVoice-ASR streaming via generate() and server mapping#483JM27616 wants to merge 3 commits into
JM27616 wants to merge 3 commits into
Conversation
Blaizzy
reviewed
Feb 5, 2026
Owner
There was a problem hiding this comment.
I have a couple nits:
- This model is multilingual, so hardcoding language to "en" is not optimal.
- This model outputs structured outputs, so we need to think about streaming properly because I know users won't know how to decode the answers and even if they were they would need to instantiate the model on the client side to access the method to decode (see
generate).
This is why I was creating the spec for this feature throughout the week.
Owner
|
Could you also provide the Word Error rate for streaming vs generate on audios of 5min, 10min, 30min and if possible 1h. So we can see if the implementation is accurate. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Context
Streaming transcriptions for VibeVoice‑ASR were not incremental via the OpenAI-compatible endpoint (
stream=true), so users only received output after completion. This aligns VibeVoice with the generate-centric streaming pattern used in #474 and fixes #482.Description
Model.generate(stream=True)the single entrypoint for streaming and non‑streaming, returning an iterator ofStreamingResultobjects when streaming is enabled.stream_transcribe()to emit structured streaming results with token timing estimates and final token counts.stream -> generation_streamfor models that acceptgeneration_streambut notstream.Changes in the codebase
StreamingResultdataclass in VibeVoice‑ASR and exported it in__init__.py.generate()to acceptstreamand return a generator when enabled.stream_transcribe()to yieldStreamingResultobjects instead of raw strings.generation_stream.Changes outside the codebase
None.
Additional information
Manual validation with
curl -Nconfirms multiple NDJSON lines are emitted over time whenstream=true, and non‑streaming behavior remains unchanged.Checklist