Repository navigation
Import heavy ML libraries on demand to speed up startup - #1669
Merged
Merged
Conversation
Opening the main window imported torch, transformers, whisper, faster_whisper, stable_whisper, openai, demucs and numba, although none of them is needed until a transcription, recording, speaker identification or preferences dialog starts. Startup took about 4.2 s and 754 MB before the window appeared. Import them where they are used instead: the transcriber classes when a task starts, the recording and preferences widgets when opened, and the libraries themselves inside the functions that need them. Also check the CUDA_PROMPT_SHOWN setting before is_cuda_torch_installed(), so machines with an NVIDIA GPU no longer import torch on every launch just to skip a prompt that was already shown. The window now appears after about 0.45 s, using 142 MB (repeat launch, offscreen, Linux). Tests that patched the moved names now patch the library attribute instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ozoVtp2JvfpDkZ1SpBgvW
Moving the openai import into Translator.__init__ kept it out of app startup, but every transcription viewer creates a Translator, so opening a transcript still paid about 0.5 s for it. Build the OpenAI client the first time a translation is sent instead, on the translator's worker thread. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WnjChXaA6f83jUFbYb9k4e
yt_dlp is only needed for URL imports, so import it in _download_from_url instead of at app startup. Opening the live recording window imported torch, transformers, whisper, faster_whisper and openai on the UI thread, freezing it for about 4 seconds. Import each backend's libraries where they are used instead; that code runs on the transcription thread and only loads what the selected backend needs. whisper.cpp and the OpenAI API no longer import torch at all. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Closing the live recording window only hid it, so it stayed alive as a child of the main window until the app exited, where PyQt's exit cleanup crashed with a segmentation fault. Every opened window also leaked. Delete the window when it is closed. As it can now go away while the translation thread is still running, defer the close until both the transcription and translation threads have finished, not only the transcription one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The copy button's text was reset by QTimer.singleShot with a lambda, which still fires after the window is closed. Now that the window is deleted on close, that raised "wrapped C/C++ object of type QPushButton has been deleted". Pass a bound method instead, which PyQt drops when the window is deleted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
# Conflicts: # buzz/transcriber/file_transcriber.py # tests/transcriber/file_transcriber_test.py
Collaborator
|
@doqhu Thanks for your contribution! |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1669 +/- ##
==========================================
+ Coverage 84.22% 84.25% +0.03%
==========================================
Files 115 115
Lines 13931 13965 +34
==========================================
+ Hits 11733 11766 +33
- Misses 2198 2199 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Opening the main window imported most of the ML stack — torch, transformers, whisper, faster_whisper, stable_whisper, openai, demucs, numba, pandas, ctranslate2, torchaudio — although none of them is needed until a transcription, recording, speaker identification or the Preferences dialog actually starts. On Linux this took about 4.2 s and 754 MB before the window appeared.
Fix
Import these where they are used instead of at module level:
file_transcriber_queue_worker: the transcriber classes are imported in_create_transcriber, and demucs inside the speech-extraction worker (which already imported torch locally).main_window_toolbar:RecordingTranscriberWidgetis imported when Record is clicked.menu_bar:PreferencesDialogis imported when Preferences opens (the type hint stays viaTYPE_CHECKING).model_loader,translator,transcription_resizer_widget,speaker_identification_widget: whisper, openai, stable_whisper, torch and faster_whisper are imported inside the functions that use them.main_window._maybe_show_cuda_prompt:CUDA_PROMPT_SHOWNis now checked beforeis_cuda_torch_installed(). Previously, on machines with an NVIDIA GPU, every launch imported torch only to then skip a prompt that had already been shown. The outcome of the check is unchanged.Tests that patched a moved module-level name now patch the library attribute instead (e.g.
buzz.translator.OpenAI→openai.OpenAI), which works because the function-level import looks the attribute up at call time.Results
Time until the main window is shown, offscreen, median of 5 runs, Fedora 44:
Heavy modules loaded at startup go from 13 to 2 (yt_dlp, numpy).
Testing
-x: 1364 passed, 15 skipped. The one failure,test_should_run_file_transcription_task, is the existing flake on NVIDIA machines fixed by Fix CUDA prompt firing on a deleted main window #1668; with that fix applied as well, the whole suite passes (1388 passed).buzz add --hide-gui) oftestdata/whisper-french.mp3with Faster Whisper tiny and Whisper tiny both produce the expected text.Independent of #1667 and #1668; it merges cleanly with both.
🤖 Generated with Claude Code
https://claude.ai/code/session_012ozoVtp2JvfpDkZ1SpBgvW