Repository navigation
Run every demo weekly, two new demos, and a front page that matches - #58
Conversation
The six demos were checked by hand and by nothing else: the workflow ran one course notebook, and its pull_request paths did not even mention Demos/, so changing a demo ran nothing. All eight are in the matrix now. What to install is asked of the notebook rather than written in the workflow: `run_notebook.py --print-install` reads its own pip lines, so a demo that starts needing another extra says so in the cell a reader runs and this installs it without being told twice. Only unindented lines count - the model chooser's S3PRL install sits inside an `if`, and what it asks for depends on a choice made when the notebook runs. asr_streaming_demo.ipynb is the old Demos/ASR/streaming_asr_demo.ipynb made to work: same idea, same 40 ms slices, on the release rather than on whatever pip resolved in 2021. It prints the hypothesis as it grows, which is the thing worth seeing and the thing no other demo shows. That needed the naming rule to admit a variant. It admits one, and gains a check that every demo is listed in Demos/README.md with a line saying what it does - so a second page cannot arrive without someone saying how it differs. That check immediately found that the table listed one demo while six existed: an edit lost to a careless `git checkout` earlier today, which nothing would otherwise have noticed. Two more things the work turned up. drop_installs deleted an indented install and left the `if` above it with an empty body. And the course notebook installed S3PRL whichever front-end was chosen, though only the two WavLM ones import it - and those two do not currently work at all, because S3PRL calls a torchaudio function removed in 2.1. The cell now installs it only when it is wanted and says why it will fail there. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each demo carries the workflow's status badge beside its Colab badge, so somebody looking at the page in Colab can see it is run rather than hoped for. One badge for the set: red means one of them broke, not all of them. A badge is a claim, and a claim wants a check. check_layout.py now fails when a demo has no badge, and when a demo is not in the workflow's matrix - which is the way the badge could quietly become a lie, a notebook sitting in the directory wearing the status of runs that never included it. Verified by taking codec_demo out of the matrix and watching it fail. unmaintained/ has no badge, which is the contrast that makes the badge mean something. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
espnet/tedlium2_streaming_transformer, the weights Keqi Deng trained and published, copied into the organization with a card that says whose they are and links the original. Nothing about the model changed; what changed is that the notebook now points at something the organization keeps alive, and the paragraph explaining that it did not is gone. Loaded from the new tag and decoded with it before this commit - same transcript - and the notebook ran again end to end. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The badge belongs where someone arrives, so it is at the top of the root README rather than only inside Demos/. Putting it there showed that page was describing the repository as it was before yesterday: every one of its eight demo links pointed into Demos/ASR, Demos/SE, Demos/SLU, Demos/TTS, Demos/Others. Flattening the directory broke all eight at once, I updated Demos/README.md and espnet's README and not this one, and nothing said so - a moved notebook is a 404 on GitHub and "Notebook not found" in Colab, neither of which reaches anybody here. It now lists the seven demos that exist, and says that unmaintained/ carries no badge and nothing checks it. check_layout.py now resolves every notebook link in both READMEs. Run against the page as it stood a minute ago, it names all eight. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The espnet README's demo table had a row pointing at egs2/TEMPLATE/sds1 - a Gradio app you clone the repository to run. It was the one row in a table of Colab notebooks that was not one. sds_demo.ipynb is the cascade with its parts showing: OWSM-CTC hears a real recording, a 360M instruct model answers it, VITS reads the answer back. Twelve cells, CPU throughout. The recording is a real one and not synthesised. I tried having the notebook speak its own question first, which is tidier: VITS says "What is the capital of France?", OWSM hears "What tis the capital of Frax", and the model duly answers about Frax. A short synthetic utterance is not what a dialogue system meets, and a demo that looks broken is worse than one cell longer. The LLM is plain transformers rather than espnet2.sds.llm, and the notebook says why: that wrapper takes access_token as a required str, so an ungated model needs a token it does not need. Worth fixing in espnet; a demo is not the place to work around it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four ungated instruct models, the same turn, three samples each:
SmolLM2-360M "The hotel sale will be a key component of Holiday's
strategy to focus on property management."
Qwen2.5-0.5B "Yes, that seems like an effective strategy for Holiday
Hotels Group."
SmolLM2-1.7B "The sale of the hotels is part of Holiday's strategy to
sell off assets and concentrate on property management."
Two of the three hand the sentence back reworded, which in a dialogue
demo reads as broken. Qwen2.5-0.5B replies to it - twice in three
samples, against never for the other two. It is 494M against 362M, so
about 1 GB to download instead of 720 MB, and 7 seconds to answer on a
CPU instead of 2.
The five-times-larger SmolLM2-1.7B is the clearest result: it paraphrases
every time, takes 75 seconds to load and 3.4 GB to fetch. Size was not
the axis, which is worth knowing before reaching for a bigger model.
Qwen3-0.6B was tried too and left out: it writes its reasoning into the
reply unless thinking mode is turned off, and a demo should not need that
explained.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Both from review, and the tests found three bugs on their first run.
`--print-install` decides what the weekly run installs, so a wrong answer
there is a green run of the wrong thing. Seventeen cases, and three of
them failed:
- a trailing comment became three packages, because shlex was not
told about comments;
- a command split over lines raised ValueError on the backslash, and
only its first line was read;
- drop_installs removed the first line of such a command and left the
rest in the cell as Python, which does not parse.
Fixed by reading logical lines rather than physical ones, in both
functions, and by splitting with comments=True. The eight notebooks
report the same dependencies as before.
The tests run in pre-commit, so pull requests see them.
And Demos/README.md now says what green means: every cell ran to the end
on a clean CPU runner last Sunday. It does not mean the output was right
- nothing checks a transcript against a reference. Most of what a demo
depends on is outside this repository, and the table lists each: the
pinned release does not move, its dependency tree is not pinned, the
checkpoints are on the Hub, the LLM in sds_demo is not ESPnet at all. A
red badge is a question about which of those changed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Courses section listed four courses as though they were alike. One is maintained and one notebook of it runs weekly; the other three have not been checked in years and nobody runs them. They are worth keeping - they are the record of how these things were taught - and worth saying that about, in a paragraph rather than as three more lists that read like offers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Tick the box to add this pull request to the merge queue (same as
|
GitHub's badge is per workflow and cannot show one job of a matrix, so one shared badge was the most the matrix could offer: red meant "something broke" and you had to go and look. Nine small workflows now, each naming its notebook and calling one reusable file that holds the job. A table with a badge a row, and a pull request that touches one demo runs that one. The tables carry them: Demos in both READMEs, and the course notebooks too, where four of the five say "by hand" instead - which is the honest column and reads better than a footnote. check_layout.py follows: a demo needs a workflow of its own, that workflow has to pass that notebook, and the notebook has to carry that workflow's badge. The second check wanted tightening - the file names its notebook in a comment and in `paths:` as well, so a looser match passed while the workflow ran something else. It matches the input now, and was checked by pointing tts_demo.yml at another notebook. CMUSpeechTechnology26S reads "CMU Speech Technology 26S" in the prose. The directory keeps its name: it is in every link here, in espnet's documentation, and in the workflow paths, and a space in a path is a week of small breakages for a nicer heading. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
From review: --print-install is the interface a notebook author writes against, since the workflow reads the install cell rather than repeating it, so what it supports should be stated rather than inferred from the tests. tools/README.md now has it as two tables - what is read as a dependency, and what is not and why. The `git+` exclusion gets its reason in full: a demo pins a release so that it does the same thing in April, and installing espnet from git in CI would test something other than what a reader gets, which would make a pass mean nothing. And what to do when a notebook needs something the contract cannot express: it is a Courses/ document rather than a demo, because the weekly badge would be promising something it cannot keep. Each row of both tables is a case in tests/test_run_notebook.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Right — it is an interface, and it was only written down in the tests.
Read as a dependency: Not read, each with its reason:
Two more things the section says. How to see what CI will install, which is to And what to do when a notebook needs something the contract cannot express: it Every row of both tables is a case in |
text_to_speech runs. It was listed as blocked on building a vocoder from
source, which turned out to be ParallelWaveGAN's setup.py importing pip -
a package an isolated build does not have, so `--no-build-isolation` is
the whole fix. It has a workflow and a badge like the demos.
Getting there needed CI to install what a notebook clones. VERSA and
ParallelWaveGAN are on nobody's index, so the course notebooks fetch them
with `git clone` and `pip install .`, two shell lines the runner drops.
`--print-git-install` reads that pair and returns `git+<url>`, keeping
the notebook the single source of truth for its dependencies, as
--print-install already does for the rest. espnet is never returned that
way whatever form it is written in.
And the stale pins go: numpy==1.23.5 with tensorflow==2.12.0, from when
Colab ran Python 3.8. numpy 1.23.5 has no wheel for 3.12, so those lines
could only ever have failed - they passed unnoticed because the runner
skips install cells and a reader on Colab gets a resolver error they were
told to ignore.
The three that still do not run say what stops each, which is worth more
than "by hand":
speech_enhancement its noisy sample comes from Google Drive via gdown;
everything else installs and runs
neural_codec VERSA's scorer exits without writing its output
speech_translation not tried since the pins came out
Six more tests for the clone-and-install reading, and tools/README.md
documents it beside the rest of the contract.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#57 never reached master. It was stacked on #55, and #55 merged into master
first — so when #57 merged two hours later it went into
notebooks/pin-released-espnet, a branch master had already taken and stoppedfollowing. GitHub only retargets a stacked pull request when its base branch is
deleted. My mistake for stacking; this is the same work, aimed at master.
It is why
sds_demo.ipynbandasr_streaming_demo.ipynbare not inDemos/, and why thefront page has no badge.
What is in here
Every demo runs weekly. The workflow ran one course notebook, and its
pull_requestpaths did not mentionDemos/at all, so changing a demo rannothing. All nine notebooks are in the matrix, and what to install is asked of
each notebook —
run_notebook.py --print-installreads its own pip lines, so ademo that starts needing another extra says so in the cell a reader runs.
Two more demos, each run end to end before committing:
asr_streaming_demo.ipynb— the old 2021 streaming notebook made to work, onespnet/tedlium2_streaming_transformer, the weights Keqi Deng trained, copiedinto the organization with credit so the link cannot rot.
sds_demo.ipynb— spoken dialogue with the parts showing: OWSM-CTC hears,Qwen2.5-0.5B answers, VITS speaks. It replaces the one row in espnet's demo
table that was a link to a recipe rather than a notebook.
Badges, and the checks that stop them being decoration. Each demo carries
the workflow's status beside its Colab badge.
check_layout.pyfails if a demohas no badge, if it is not in the workflow matrix, if it is not listed in the
README, or if a README links to a notebook that is not there — which caught the
root README pointing at eight files that the flattening had moved.
Tests for the parsing CI trusts (from review): seventeen cases, three of
which were bugs — a trailing comment became four packages, a line-continued
command raised
ValueError, anddrop_installsleft half of one behind asinvalid Python.
What green means is written down: every cell ran on a clean CPU runner last
Sunday; it does not mean the output was right; and a table of what a demo
depends on that lives outside this repository.
The front page now lists the nine demos, says
unmaintained/is unchecked,and stops presenting four courses as though they were alike — one is maintained,
the other three have not been checked in years and are kept as a record.
🤖 Generated with Claude Code