sw005320 · Nov 8, 2026 · Nov 8, 2026 · Nov 8, 2026
diff --git a/.github/workflows/diar_vad_demo.yml b/.github/workflows/diar_vad_demo.yml
@@ -0,0 +1,29 @@
+# Demos/diar_vad_demo.ipynb, every Sunday. The badge on this workflow is what
+# the README shows beside that notebook, which is why it has a file of its own:
+# GitHub's badge is per workflow, and cannot show one job of a matrix.
+
+name: diar_vad_demo
+
+on:
+  schedule:
+    # Sundays, 05:00 UTC
+    - cron: "0 5 * * 0"
+  workflow_dispatch:
+  pull_request:
+    paths:
+      - "Demos/diar_vad_demo.ipynb"
+      - "tools/**"
+      - ".github/workflows/diar_vad_demo.yml"
+      - ".github/workflows/_run_notebook.yml"
+
+permissions:
+  contents: read
+
+jobs:
+  run:
+    uses: ./.github/workflows/_run_notebook.yml
+    with:
+      notebook: Demos/diar_vad_demo.ipynb
+      # the VAD classes in espnet2.bin.diar_inference are not in a release yet: the
+      # notebook pins the one that will carry them
+      allow_unreleased_pin: true
diff --git a/Demos/README.md b/Demos/README.md
@@ -44,6 +44,7 @@ the answer now.
 | [`tts_demo.ipynb`](tts_demo.ipynb) | [![tts_demo](https://github.com/espnet/notebook/actions/workflows/tts_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/tts_demo.yml) | Type a sentence, hear it spoken — one English voice, then 128 of them |
 | [`enh_demo.ipynb`](enh_demo.ipynb) | [![enh_demo](https://github.com/espnet/notebook/actions/workflows/enh_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/enh_demo.yml) | Pull speech out of noise, and measure how much it helped |
 | [`spk_demo.ipynb`](spk_demo.ipynb) | [![spk_demo](https://github.com/espnet/notebook/actions/workflows/spk_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/spk_demo.yml) | Turn a voice into a vector, and score two recordings against each other |
+| [`diar_vad_demo.ipynb`](diar_vad_demo.ipynb) | [![diar_vad_demo](https://github.com/espnet/notebook/actions/workflows/diar_vad_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/diar_vad_demo.yml) | Find where speech is, offline, as it arrives, and over many files at once |
 | [`codec_demo.ipynb`](codec_demo.ipynb) | [![codec_demo](https://github.com/espnet/notebook/actions/workflows/codec_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/codec_demo.yml) | Compress a waveform to a few integers a frame, rebuild it, count the bits |
 | [`sds_demo.ipynb`](sds_demo.ipynb) | [![sds_demo](https://github.com/espnet/notebook/actions/workflows/sds_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/sds_demo.yml) | Speech in, speech out, with a language model thinking in between |
 

diff --git a/Demos/diar_vad_demo.ipynb b/Demos/diar_vad_demo.ipynb
@@ -0,0 +1,310 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Voice activity detection: an online model and an offline one\n",
+    "\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/espnet/notebook/blob/master/Demos/diar_vad_demo.ipynb) [![diar_vad_demo](https://github.com/espnet/notebook/actions/workflows/diar_vad_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/diar_vad_demo.yml)\n",
+    "\n",
+    "Find where someone is speaking, frame by frame (8 ms). Two small temporal\n",
+    "convolutional networks, trained with ESPnet as one-speaker diarization\n",
+    "(`egs2/frontia_vad_v1/diar1`), do it two ways: a causal **online** model that reads\n",
+    "no future audio and so runs on a live stream, and an **offline** model that\n",
+    "reads up to a second ahead and places the boundaries better when the whole\n",
+    "file is there. Here they run offline over a recording, online as audio\n",
+    "arrives, and offline in batches over many files, as for cleaning a corpus.\n",
+    "\n",
+    "CPU, and each model is under 1 MB."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Install"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install -q \"espnet==202612\" espnet_model_zoo librosa matplotlib"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The models\n",
+    "\n",
+    "`VAD_MODEL` (offline) and `VAD_ONLINE_MODEL` (online) take a Hugging Face tag\n",
+    "or the `.zip` that stage 8 of `diar.sh` packs, so checkpoints you trained\n",
+    "yourself load the same way. Each model comes with its own decision rule,\n",
+    "tuned on AMI MH dev."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "\n",
+    "from espnet2.bin.diar_inference import VoiceActivityDetector, format_segments\n",
+    "\n",
+    "offline = VoiceActivityDetector.for_mode(\"offline\", os.environ.get(\"VAD_MODEL\"))\n",
+    "online = VoiceActivityDetector.for_mode(\"online\", os.environ.get(\"VAD_ONLINE_MODEL\"))\n",
+    "fs = offline.fs\n",
+    "for name, vad in ((\"online\", online), (\"offline\", offline)):\n",
+    "    print(f\"{name:8s} reads {vad.offline_look_ahead * vad.frame_shift:.2f} s ahead, \"\n",
+    "          f\"threshold {vad.threshold}, median filter {vad.median} frames\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## A recording with pauses\n",
+    "\n",
+    "Two sentences from two speakers, with silence before, between and after."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import numpy as np\n",
+    "from IPython.display import Audio, display\n",
+    "\n",
+    "from espnet2.bin.diar_inference import read_audio\n",
+    "\n",
+    "!wget -q -O a.wav https://github.com/espnet/espnet/raw/master/test_utils/ctc_align_test.wav\n",
+    "!wget -q -O b.wav https://github.com/espnet/espnet/raw/master/test_utils/st_test.wav\n",
+    "\n",
+    "a, b = read_audio(\"a.wav\", fs), read_audio(\"b.wav\", fs)\n",
+    "\n",
+    "def silence(seconds):\n",
+    "    return np.zeros(int(seconds * fs), dtype=np.float32)\n",
+    "\n",
+    "recording = np.concatenate([silence(0.5), a, silence(2.0), b, silence(1.0)])\n",
+    "print(f\"{len(recording) / fs:.1f} s\")\n",
+    "display(Audio(recording, rate=fs))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Offline\n",
+    "\n",
+    "Each frame's speech probability from the offline model, then the decision\n",
+    "rule the recipe scores with: a frame is speech when its probability is above\n",
+    "the threshold, and a median filter removes short blips."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import matplotlib.pyplot as plt\n",
+    "\n",
+    "probs = offline.probabilities(recording, fs, look_ahead=offline.offline_look_ahead)\n",
+    "segments = offline.segments(probs)\n",
+    "print(format_segments(segments))\n",
+    "\n",
+    "seconds = np.arange(len(recording)) / fs\n",
+    "fig, (top, bottom) = plt.subplots(2, 1, figsize=(10, 4), sharex=True)\n",
+    "top.plot(seconds, recording, linewidth=0.4, color=\"0.3\")\n",
+    "bottom.plot(np.arange(len(probs)) * offline.frame_shift, probs)\n",
+    "bottom.axhline(offline.threshold, color=\"0.5\", linestyle=\"--\")\n",
+    "for s in segments:\n",
+    "    top.axvspan(s.start, s.end, alpha=0.15)\n",
+    "    bottom.axvspan(s.start, s.end, alpha=0.15)\n",
+    "bottom.set_xlabel(\"seconds\")\n",
+    "bottom.set_ylabel(\"speech\")\n",
+    "plt.show()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Online, as it arrives\n",
+    "\n",
+    "Feed the recording to the online model 200 ms at a time, at the pace a\n",
+    "microphone would deliver it, and watch: the waveform grows, the speech the\n",
+    "model has found so far is shaded green, and the red line is now. Each 8 ms\n",
+    "frame is decided as soon as its samples are in; the stream keeps the causal\n",
+    "encoder's past and runs only the new frames."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import time\n",
+    "\n",
+    "from IPython.display import display\n",
+    "\n",
+    "def draw(fig, audio, probs, now, threshold, median):\n",
+    "    \"\"\"The waveform so far with the speech shaded, and the probability.\"\"\"\n",
+    "    from espnet2.bin.diar_inference import frames_to_segments\n",
+    "\n",
+    "    fig.clf()\n",
+    "    wave, prob = fig.subplots(2, 1, sharex=True, gridspec_kw={\"height_ratios\": [2, 1]})\n",
+    "    seconds = np.arange(len(audio)) / fs\n",
+    "    wave.plot(seconds, audio, linewidth=0.4, color=\"0.3\")\n",
+    "    segments = frames_to_segments(probs, online.frame_shift, threshold, median)\n",
+    "    for s in segments:\n",
+    "        wave.axvspan(s.start, s.end, color=\"tab:green\", alpha=0.3)\n",
+    "        prob.axvspan(s.start, s.end, color=\"tab:green\", alpha=0.3)\n",
+    "    prob.plot(np.arange(len(probs)) * online.frame_shift, probs, linewidth=0.8)\n",
+    "    prob.axhline(threshold, color=\"0.5\", linestyle=\"--\", linewidth=0.6)\n",
+    "    for ax in (wave, prob):\n",
+    "        ax.axvline(now, color=\"tab:red\")\n",
+    "    wave.set_xlim(0, len(recording) / fs)\n",
+    "    wave.set_ylim(-1.05 * np.abs(recording).max(), 1.05 * np.abs(recording).max())\n",
+    "    wave.set_yticks([])\n",
+    "    speaking = bool(segments) and segments[-1].end >= len(probs) * online.frame_shift\n",
+    "    wave.set_title(\"speaking\" if speaking else \"silence\", loc=\"left\",\n",
+    "                   color=\"tab:green\" if speaking else \"0.4\", fontweight=\"bold\")\n",
+    "    prob.set_ylim(-0.02, 1.02)\n",
+    "    prob.set_ylabel(\"speech\")\n",
+    "    prob.set_xlabel(\"seconds\")\n",
+    "    fig.tight_layout()\n",
+    "\n",
+    "stream = online.stream()\n",
+    "chunk = fs // 5\n",
+    "fig = plt.figure(figsize=(10, 3.4))\n",
+    "shown = display(fig, display_id=True)\n",
+    "for start in range(0, len(recording), chunk):\n",
+    "    stream.accept(recording[start : start + chunk])\n",
+    "    now = min(start + chunk, len(recording)) / fs\n",
+    "    draw(fig, recording[: start + chunk], stream.all_probabilities(), now,\n",
+    "         online.threshold, online.median)\n",
+    "    shown.update(fig)\n",
+    "    time.sleep(0.2)  # the pace the audio would arrive at\n",
+    "stream.finish()\n",
+    "plt.close(fig)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "streamed = stream.all_probabilities()\n",
+    "whole = online.probabilities(recording, fs)\n",
+    "print(\"frames:\", len(streamed), \"largest difference from the whole recording:\",\n",
+    "      float(np.abs(streamed - whole).max()))\n",
+    "print(format_segments(stream.segments()))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Compare these segments with the offline ones above: the online model has only\n",
+    "the past to go on, which is the price of deciding each frame as it arrives."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Many files\n",
+    "\n",
+    "`detect_files` reads files and runs them through the model in batches of\n",
+    "similar length (`DiarizeSpeech.diarize_batch`). Each file gets the segments\n",
+    "it gets alone."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import soundfile\n",
+    "\n",
+    "paths = []\n",
+    "for i, piece in enumerate([a, b, recording]):\n",
+    "    path = f\"piece{i}.wav\"\n",
+    "    soundfile.write(path, piece, fs)\n",
+    "    paths.append(path)\n",
+    "\n",
+    "for path, found in zip(paths, offline.detect_files(paths, batch_size=3)):\n",
+    "    speech = sum(s.duration for s in found)\n",
+    "    print(f\"{path}: {len(found)} segments, {speech:.2f} s of speech\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## The threshold\n",
+    "\n",
+    "Raising it keeps only the frames the model is sure of; the median filter\n",
+    "then decides how short a pause or a blip may be."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from espnet2.bin.diar_inference import frames_to_segments\n",
+    "\n",
+    "for threshold in (0.3, 0.5, 0.7):\n",
+    "    for median in (1, 11):\n",
+    "        found = frames_to_segments(probs, offline.frame_shift, threshold, median)\n",
+    "        speech = sum(s.duration for s in found)\n",
+    "        print(f\"threshold {threshold}, median {median:2d}: \"\n",
+    "              f\"{len(found)} segments, {speech:.2f} s of speech\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Where next\n",
+    "\n",
+    "- **From a terminal**: `espnet detect recording.wav` (offline), `espnet detect *.wav`\n",
+    "  for a batch, or `espnet detect --live` (online) for the microphone\n",
+    "- **From an agent**: the `detect` tool of `espnet-mcp`\n",
+    "- **In the browser**: the Space in\n",
+    "  [`egs2/frontia_vad_v1/diar1/demo`](https://github.com/espnet/espnet/tree/master/egs2/frontia_vad_v1/diar1/demo),\n",
+    "  with online and offline drawn on the waveform\n",
+    "- **Training your own**: the recipe,\n",
+    "  [`egs2/frontia_vad_v1/diar1`](https://github.com/espnet/espnet/tree/master/egs2/frontia_vad_v1/diar1)"
+   ]
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3",
+   "name": "python3"
+  },
+  "language_info": {
+   "name": "python"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}
diff --git a/README.md b/README.md
@@ -22,6 +22,7 @@ One per task, flat in [`Demos/`](Demos), each short enough to read in a sitting.
 | [`tts_demo.ipynb`](Demos/tts_demo.ipynb) | [![tts_demo](https://github.com/espnet/notebook/actions/workflows/tts_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/tts_demo.yml) | Type a sentence, hear it spoken — one English voice, then 128 of them |
 | [`enh_demo.ipynb`](Demos/enh_demo.ipynb) | [![enh_demo](https://github.com/espnet/notebook/actions/workflows/enh_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/enh_demo.yml) | Pull speech out of noise, and measure how much it helped |
 | [`spk_demo.ipynb`](Demos/spk_demo.ipynb) | [![spk_demo](https://github.com/espnet/notebook/actions/workflows/spk_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/spk_demo.yml) | Turn a voice into a vector, and score two recordings against each other |
+| [`diar_vad_demo.ipynb`](Demos/diar_vad_demo.ipynb) | [![diar_vad_demo](https://github.com/espnet/notebook/actions/workflows/diar_vad_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/diar_vad_demo.yml) | Find where speech is, offline, as it arrives, and over many files at once |
 | [`codec_demo.ipynb`](Demos/codec_demo.ipynb) | [![codec_demo](https://github.com/espnet/notebook/actions/workflows/codec_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/codec_demo.yml) | Compress a waveform to a few integers a frame, rebuild it, count the bits |
 | [`sds_demo.ipynb`](Demos/sds_demo.ipynb) | [![sds_demo](https://github.com/espnet/notebook/actions/workflows/sds_demo.yml/badge.svg)](https://github.com/espnet/notebook/actions/workflows/sds_demo.yml) | Speech in, speech out, with a language model thinking in between |