Maintain the CMU course notebooks, and drop ESPnet1 and ESPnet-EZ by sw005320 · Pull Request #54 · espnet/notebook

sw005320 · 2026-09-19T05:37:56Z

The course notebooks from CMU 11492/11692/18495, Speech Technology for Conversational AI, with the graded exercises removed so they are demonstrations rather than assignments. Each one was run end to end before it was proposed here, which is how the list of breakages below was found.

What was broken

None of these were visible from the notebook: a Colab page opens, the cells look right, and the failure appears when somebody runs them.

Three notebooks loaded models from Google Drive. They now load by tag from the ESPnet organisation, which also drops the dependency on git-lfs (a git clone of a Hub repository without it yields a text pointer and torch.load fails on it). The separation model had no usable Hub copy - its repository in the organisation is empty - so that section uses the published DPTNet model instead, and the WSJ ASR evaluation uses espnet/kan-bayashi_wsj_transformer2.
espnet.asr.asr_utils.plot_spectrogram and torch_complex were ESPnet1 imports. ESPnet1 is gone, so the spectrograms are drawn from Stft output.
gdown --id X: the flag was removed in gdown 5.
--use_gpu true: VERSA replaced that flag with --device, and a CPU runtime has to say cpu.
parallel_wavegan 0.6.1 imports scipy.signal.kaiser, which SciPy removed in 1.13. A three-line compatibility shim puts it back before the vocoder is imported.
locale.getpreferredencoding = lambda: "UTF-8", a Colab workaround, raises everywhere else: Python calls it with an argument.
device="cuda" was hardcoded, so a CPU runtime could not run them at all.
input() stopped any attempt to run a notebook rather than type into it. The sentence is a variable to edit now.
Paths were rooted at /content, so nothing ran outside Colab.
Missing extras (espnet[enh], espnet[tts,spk], levenshtein, editdistance) are installed by the notebooks that need them, instead of failing halfway through.

What else changed

Outputs and widget state are cleared; one TensorBoard cell was carrying 9 MB of it.
run_notebook.py runs a notebook outside Colab: it skips the install cells and replaces wget and tar with their Python equivalents.
.github/workflows/run_notebooks.yml runs the inference notebooks weekly, so the next breakage is found here rather than in class. It starts with one notebook and grows as they prove they fit in a runner.
ESPnet1/ and ESPnetEZ/ are deleted, and the index no longer lists them. Nothing in espnet/espnet links to either; its links all point at ESPnet2/.

Not included

The course's two fine-tuning notebooks are held back. They are written against an ESPnet3 data API that has since changed - a Dataset subclass named by _target_ in the config - and both now stop at recipe_dir must be set when data_src is None. Their module renames and the BaseSystem argument are already fixed locally; what remains is a design question for the ESPnet3 authors rather than something to guess at. The ASR one also used TIDIGITS, which is LDC-licensed; it is rewritten to use Google Speech Commands (twelve words, CC-BY 4.0, streamed so only those words are written to disk, split by speaker), and that part is ready when the rest is.

🤖 Generated with Claude Code

The demonstration notebooks from CMU 11492/11692/18495 kept working in class and nowhere else: models fetched from Google Drive, ESPnet1 imports that no longer exist, flags their tools have since removed, paths rooted at /content, and `device="cuda"` hardcoded. Each of the five here was run end to end before being committed, which is how that list was found. The graded exercises are gone: the checkpoints, the points, the screenshots for Gradescope, the empty answer cells, and the sections that record your voice in the browser, which only work in Colab. run_notebook.py runs one of them outside Colab, and a weekly workflow runs the ones that fit a runner, so the next breakage is found here rather than in class. ESPnet1 and ESPnetEZ are deleted. Nothing in espnet/espnet links to either. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

mergify · 2026-09-19T05:38:39Z

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

Queue this pull request

The file was named `.pre.commit-config.yaml` and was empty, so pre-commit.ci failed every pull request with "error during ci config" instead of checking anything. Renamed, with hooks that suit a repository of notebooks: JSON validity catches a notebook that will not open, and no formatter rewrites cells, which would turn every review into a diff of machine output. The fourteen files it touched were missing their final newline. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot

Copilot review overview

🟡 Changes recommended

Runtime correctness, unsafe archive extraction, dependency mismatches, and unresolved Drive-based model downloads remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 · 5 · 2

Open (9)

Install ESPnet before importing espnet2 · New
Safely extract downloaded tar archives · New
Install the same ESPnet source in CI and notebook · New
Accept arguments in getpreferredencoding override · New
Stop enrollment after the first book · New
Replace Google Drive model download with ESPnet Hub · New
Replace simultaneous ST Google Drive download · New
Remove or qualify unavailable ASR notebook documentation · New
Remove grading language from optional exploration · New

What changed in this PR

Maintains five CMU course demonstrations, adds automated execution, and removes obsolete ESPnet1/ESPnet-EZ material.

Changes:

Adds updated speech notebooks with portable model loading and CPU support.
Adds a notebook runner, scheduled GitHub Actions workflow, and pre-commit checks.
Removes legacy notebooks and refreshes repository documentation.

File	Description
`README.md`	Updates the notebook index.
`CMU_11492_11692_18495/README.md`	Documents the maintained course notebooks.
`CMU_11492_11692_18495/run_notebook.py`	Adds non-Colab notebook execution.
`CMU_11492_11692_18495/speaker_verification.ipynb`	Adds speaker verification and diarization demo.
`CMU_11492_11692_18495/speech_enhancement.ipynb`	Adds enhancement and separation demo.
`CMU_11492_11692_18495/neural_codec.ipynb`	Adds neural codec comparison demo.
`CMU_11492_11692_18495/speech_translation.ipynb`	Adds offline and streaming translation demo.
`CMU_11492_11692_18495/text_to_speech.ipynb`	Adds single- and multi-speaker TTS demo.
`.github/workflows/run_notebooks.yml`	Adds scheduled notebook validation.
`.pre-commit-config.yaml`	Adds conservative repository checks.
`.pre.commit-config.yaml`	Removes the misnamed empty configuration.
`ESPnetEZ/ASR/train_from_scratch.ipynb`	Removes obsolete ESPnet-EZ demo.
`ESPnetEZ/ASR/ASR_finetune_owsm.ipynb`	Removes obsolete ESPnet-EZ demo.
`ESPnetEZ/ST/ST_finetune_owsm.ipynb`	Removes obsolete ESPnet-EZ demo.
`ESPnetEZ/ST/integrate_huggingface.ipynb`	Removes obsolete ESPnet-EZ demo.
`ESPnetEZ/SLU/SLU_finetune_owsm.ipynb`	Removes obsolete ESPnet-EZ demo.
`ESPnetEZ/TTS/TTS_finetune_vctk_dump.ipynb`	Removes obsolete ESPnet-EZ demo.
`ESPnetEZ/SVS/SVS_finetune_ace-kising.ipynb`	Removes obsolete ESPnet-EZ demo.
`ESPnet1/asr_library.ipynb`	Removes legacy ESPnet1 material.
`ESPnet1/asr_recipe.ipynb`	Removes legacy ESPnet1 material.
`ESPnet1/pretrained.ipynb`	Removes legacy ESPnet1 material.
`ESPnet1/st_demo.ipynb`	Removes legacy ESPnet1 material.
`ESPnet1/tts_realtime_demo.ipynb`	Removes legacy ESPnet1 material.
`ESPnet1/tts_recipe.ipynb`	Removes legacy ESPnet1 material.
`ESPnet2/Demo/SLU/2pass_slu_demo.ipynb`	Normalizes the final newline.
`ESPnet2/Demo/Others/onnx_conversion_demo.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechRecognition_Fall2022/recipe_tutorial.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechRecognition_Fall2021/general_tutorial.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/s2st_demo.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment8_tts.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment7_se.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment6_slu.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment5_st.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment4_ssl.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment3_spk.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment1_espnet-tutorial.ipynb`	Normalizes the final newline.
`ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment0_data-prep.ipynb`	Normalizes the final newline.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+   "source": [
+    "!git clone https://github.com/wavlab-speech/versa\n",
+    "!cd versa && pip install .\n",
+    "!pip install espnet_model_zoo\n",


+                + f'_u.urlopen("{url}", context=_c).read())'
+            )
+        elif tar:
+            lines.append(f'import tarfile as _t; _t.open("{tar.group(1)}").extractall(".")')


+          python -m pip install espnet nbclient nbformat ipykernel \
+            librosa scikit-learn matplotlib tqdm soundfile


+   "outputs": [],
+   "source": [
+    "import locale\n",
+    "locale.getpreferredencoding = lambda: \"UTF-8\"\n",


+    "      if audio_path.endswith(\"flac\"):\n",
+    "        speaker2wav[spk].append(os.path.join(base_dir, spk, book, audio_path))\n",
+    "    # we will only use the audio from one book\n",
+    "    continue\n",


+   "outputs": [],
+   "source": [
+    "# Download pretrained st model\n",
+    "!gdown 1Sn2rAZXVSm1hrCj5OIlq61EgbjKXNGdq\n",


+   "outputs": [],
+   "source": [
+    "# Download retrained sst model\n",
+    "!gdown 1ekUeMvmaB3ZhAIY_KAb_we1zhIRFZhtu\n",


+- **TIDIGITS is replaced by [Google Speech
+  Commands](https://arxiv.org/abs/1804.03209)** in the ASR notebook. TIDIGITS
+  is licensed by the LDC and cannot be redistributed. The replacement keeps
+  twelve words - *yes*, *no* and the digits - and streams only those out of
+  the archive, and it splits by speaker so the word error rate means


+    "  )\n",
+    "\n",
+    "\n",
+    "# NOTE: please change the config file with your favorite metrics to get full points\n",


sw005320

CMU_11492_11692_18495 --> CMUSpeechTechnology2026S
Remove ESPnet2/Course since they are very old and would not work

sw005320 · 2026-09-19T05:44:29Z

@@ -0,0 +1,64 @@
+# CMU 11492/11692/18495: Speech Technology for Conversational AI


add 2026 spring semester

sw005320 · 2026-09-19T05:48:00Z

Remove ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment0_data-prep.ipynb
This would not be working

- Say which semester the notebooks are from. - Remove ESPnet2/Course/CMU_SpeechProcessing_Spring2023/assignment0_data-prep, which does not work. - The neural codec notebook installed VERSA and espnet_model_zoo but never espnet, so a clean Colab runtime stopped at the espnet2 import. It passed here because the environment already had espnet. - Speaker verification: `continue` was the last statement of a loop, so every book was collected although the comment said one. The verification utterance comes from another book, so this also let test audio into the enrollment average. It is `break` now. - Speaker verification kept the zero-argument locale lambda, which raises outside Colab; the file predated that fix and was never regenerated. - The two English to Spanish translation checkpoints have no copy in the organisation - the Hub has en-de MuST-C models - so they stay on Drive and the notebook says why, rather than the README claiming every model moved. - run_notebook.py extracted archives without checking member paths, one of them fetched over a connection whose certificate it does not verify. filter="data" refuses absolute and ../ members. - The weekly job installed the PyPI release while the notebooks install from git, so a break on master would have passed here and failed in Colab. - Removed the last "to get full points" note, and the README bullet about a notebook this directory does not carry. Speaker verification and the codec notebook were run again after the changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

sw005320 · 2026-09-19T05:58:35Z

Both of your comments are in, along with the review's findings. Pushed as 2dfb6ea.

Yours

The directory README now says Spring 2026.
assignment0_data-prep.ipynb is deleted, and the index no longer lists it. The other eight notebooks in CMU_SpeechProcessing_Spring2023 are from the same term and probably fare no better. Say the word and I will run each one the way I ran these five, and delete or repair them on the evidence rather than by assumption.

From the review, all real

The codec notebook never installed espnet. It installed VERSA and espnet_model_zoo, and a clean Colab runtime stopped at the espnet2 import. It passed my runs because the environment already had espnet. Fixed and run again.
Speaker verification collected every book. continue was the last statement of the loop, so the comment saying "only one book" was not what the code did, and the verification utterance later taken from another book could already be inside the enrollment average. It is break now, and the notebook was run again after the change.
That notebook also kept the zero-argument locale.getpreferredencoding lambda, which raises outside Colab. The file was copied in before that fix and never regenerated.
Two translation models are still on Google Drive, and the README claimed otherwise. They are English to Spanish; the organisation publishes en-de MuST-C models, not these. The Drive copies stay, the notebook says why and points at the recipe that trained them, and the README no longer overstates it.
run_notebook.py extracted archives without checking member paths, one of them fetched over a connection whose certificate the notebook explicitly does not verify. filter="data" now refuses absolute and ../ members.
The weekly job installed the PyPI release while the notebooks install from git, so a break on master would have passed here and failed in class. Both install from git now.
Removed the last "to get full points" note in the codec notebook, and the README bullet describing a notebook this directory does not carry.

A visitor does not need to know that ESPnet2 is a thing to find a demo, and the course directories spelled their term four different ways. So: Demos/{ASR,SE,SLU,TTS,Others} Courses/{CMUSpeechTechnology26S,CMUSpeechProcessing23S, CMUSpeechRecognition22F,CMUSpeechRecognition21F} The ESPnet2 level is gone, and every course is <name><year><term>. Links in espnet/espnet point at the old paths and need the companion pull request there; ci/check_demo_links.py notices the day after this merges. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

That repository is becoming Demos/ and Courses/ (espnet/notebook#54): the ESPnet2 level is gone, since a visitor does not need to know what ESPnet2 is to find a demo, and each course directory is now <name><year><term>. Every link here points at the old paths. This moves them, in the README, in three recipe READMEs, and in the checker's own fixtures. doc/index.md's second card pointed at the ESPnet-EZ section, which that repository no longer has, so it points at the courses instead; doc/notebook2rst.sh no longer deletes an ESPnet1 directory that is gone. Merge this after espnet/notebook#54, not before: until that merges these paths do not exist, and ci/check_demo_links.py checks them daily. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

sw005320 requested a balanced review from Copilot September 19, 2026 05:40

sw005320 mentioned this pull request Sep 19, 2026

The ESPnet3 fine-tuning tutorials no longer run against master espnet/espnet#6724

Closed

Copilot started reviewing on behalf of sw005320 September 19, 2026 05:44

View session

Copilot AI reviewed Sep 19, 2026

View reviewed changes

sw005320 commented Sep 19, 2026

View reviewed changes

sw005320 mentioned this pull request Sep 19, 2026

Follow espnet/notebook's new layout espnet/espnet#6732

Merged

sw005320 merged commit a430161 into master Sep 19, 2026
4 checks passed

Navigation Menu

Repository navigation

Maintain the CMU course notebooks, and drop ESPnet1 and ESPnet-EZ - #54

Maintain the CMU course notebooks, and drop ESPnet1 and ESPnet-EZ#54
sw005320 merged 4 commits into
masterfrom
course/cmu-speech-technology

sw005320 commented Sep 19, 2026

Uh oh!

mergify Bot commented Sep 19, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

sw005320 left a comment

Uh oh!

sw005320 Sep 19, 2026

Uh oh!

sw005320 Sep 19, 2026

Uh oh!

sw005320 commented Sep 19, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		python -m pip install espnet nbclient nbformat ipykernel \
		librosa scikit-learn matplotlib tqdm soundfile

		@@ -0,0 +1,64 @@
		# CMU 11492/11692/18495: Speech Technology for Conversational AI

Conversation

sw005320 commented Sep 19, 2026

What was broken

What else changed

Not included

Uh oh!

mergify Bot commented Sep 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

sw005320 left a comment

Choose a reason for hiding this comment

Uh oh!

sw005320 Sep 19, 2026

Choose a reason for hiding this comment

Uh oh!

sw005320 Sep 19, 2026

Choose a reason for hiding this comment

Uh oh!

sw005320 commented Sep 19, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants