diff --git a/README.md b/README.md index 69b16949..7470d6a8 100644 --- a/README.md +++ b/README.md @@ -92,6 +92,8 @@ StemDeck is free and **does not accept any money, sponsorship, or funding** fro **Song analysis** including BPM (librosa beat tracker), key, scale, and confidence (Albrecht-Shanahan profiles), integrated LUFS (BS.1770), and sample peak in dBFS. +**Duet split.** On a finished job, split the vocals into two singers. Where the two sit in different vocal ranges this is clean. Where they share a range, a second model handles it and puts the other singer roughly 14 dB down, which is a real split rather than an isolation. A chorus is beyond both, and `docs/models.md` has the measurements behind all of that. + **Cancellable jobs.** Cancel mid-pipeline and the runner terminates the active subprocess immediately, deletes the partial job dir, and returns to ready. **Library panel** with folder-based track organisation, drag-and-drop, search, and trash. @@ -160,7 +162,7 @@ Extract the zip anywhere, run `StemDeck.exe`. FFmpeg, the Demucs model, config,
-StemDeck is built on **[Python 3.12](https://python.org)** managed via **[uv](https://github.com/astral-sh/uv)**, with a **[FastAPI](https://fastapi.tiangolo.com)** backend serving REST and Server-Sent Events. Stem separation uses **[Demucs](https://github.com/facebookresearch/demucs)** (`htdemucs_6s`), Meta AI's open-source 6-stem neural network. The optional on-demand lead/backing vocal split runs the UVR-MDX-NET Karaoke 2 model via **[audio-separator](https://github.com/nomadkaraoke/python-audio-separator)**, trained as part of the **[Ultimate Vocal Remover](https://github.com/Anjok07/ultimatevocalremovergui)** project by Anjok07. YouTube audio is fetched via **[yt-dlp](https://github.com/yt-dlp/yt-dlp)**; transcoding and mixing use **[FFmpeg](https://ffmpeg.org)**. BPM detection and key analysis run on **[librosa](https://librosa.org)**; loudness measurement uses **[pyloudnorm](https://github.com/csteinmetz1/pyloudnorm)** (ITU-R BS.1770). The macOS and Windows desktop shells are **[Tauri v2](https://tauri.app)** (Rust/WKWebView on macOS, Rust/WebView2 on Windows). The frontend is vanilla JS with the Web Audio API, no framework and no build step; waveforms are rendered on `` using min/max sample rendering. +StemDeck is built on **[Python 3.12](https://python.org)** managed via **[uv](https://github.com/astral-sh/uv)**, with a **[FastAPI](https://fastapi.tiangolo.com)** backend serving REST and Server-Sent Events. Stem separation uses **[Demucs](https://github.com/facebookresearch/demucs)** (`htdemucs_6s`), Meta AI's open-source 6-stem neural network. The optional on-demand lead/backing vocal split runs the UVR-MDX-NET Karaoke 2 model via **[audio-separator](https://github.com/nomadkaraoke/python-audio-separator)**, trained as part of the **[Ultimate Vocal Remover](https://github.com/Anjok07/ultimatevocalremovergui)** project by Anjok07. The duet split runs the same project's BS-Roformer chorus model, and for two singers who share a vocal range it falls back to a **[MedleyVox](https://github.com/jeonchangbin49/MedleyVox)** Conv-TasNet, using weights trained and released under CC BY 4.0 by Carson Evans; the model itself is built from **[asteroid](https://github.com/asteroid-team/asteroid)** components (MIT, Pariente Manuel), vendored into `app/pipeline/_convtasnet.py` with its copyright notice rather than pulled in as a dependency. YouTube audio is fetched via **[yt-dlp](https://github.com/yt-dlp/yt-dlp)**; transcoding and mixing use **[FFmpeg](https://ffmpeg.org)**. BPM detection and key analysis run on **[librosa](https://librosa.org)**; loudness measurement uses **[pyloudnorm](https://github.com/csteinmetz1/pyloudnorm)** (ITU-R BS.1770). The macOS and Windows desktop shells are **[Tauri v2](https://tauri.app)** (Rust/WKWebView on macOS, Rust/WebView2 on Windows). The frontend is vanilla JS with the Web Audio API, no framework and no build step; waveforms are rendered on `` using min/max sample rendering. *Thanks to the creators and maintainers of all the open-source libraries that make StemDeck possible.* diff --git a/app/_vendor/LICENSE.asteroid b/app/_vendor/LICENSE.asteroid new file mode 100644 index 00000000..0708f118 --- /dev/null +++ b/app/_vendor/LICENSE.asteroid @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2019 Pariente Manuel + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/app/api/config.py b/app/api/config.py index 21cfe854..ed5ab088 100644 --- a/app/api/config.py +++ b/app/api/config.py @@ -2,7 +2,7 @@ from fastapi import APIRouter -from app.core.config import EXTRA_STEM_NAMES, STEM_NAMES +from app.core.config import DUET_STEM_NAMES, EXTRA_STEM_NAMES, STEM_NAMES router = APIRouter() @@ -14,4 +14,8 @@ def get_config() -> dict: # existing clients that assume "every job produces exactly these stems" # are unaffected; new clients merge it into their lane vocab (see # syncStemNamesFromAPI in static/js/constants.js). - return {"stem_names": list(STEM_NAMES), "extra_stem_names": list(EXTRA_STEM_NAMES)} + return { + "stem_names": list(STEM_NAMES), + "extra_stem_names": list(EXTRA_STEM_NAMES), + "duet_stem_names": list(DUET_STEM_NAMES), + } diff --git a/app/api/jobs.py b/app/api/jobs.py index c27775fd..849f667e 100644 --- a/app/api/jobs.py +++ b/app/api/jobs.py @@ -40,6 +40,7 @@ from app.pipeline import jobqueue from app.pipeline.collect import merge_stem_peaks, presence_for_split from app.pipeline.download import InvalidYouTubeURL, validate_youtube_url +from app.pipeline.duet_split import split_duet from app.pipeline.errors import classify_failure from app.pipeline.runner import _pipeline_lock from app.pipeline.vocal_split import split_vocals @@ -408,15 +409,15 @@ def cancel_job(job_id: str) -> dict: return job.to_state() -def _write_vocal_split_error(stems_dir: Path, cause: str, tail: list[str]) -> None: - """Best-effort error record for the on-demand vocal split (#275). The job - itself stays "done" -- this is diagnostic-only, not the quarantine path - (which would delete the job's base stems).""" +def _write_split_error(stems_dir: Path, filename: str, cause: str, tail: list[str]) -> None: + """Best-effort error record for an on-demand split (#275 and the duet + split). The job itself stays "done" -- this is diagnostic-only, not the + quarantine path (which would delete the job's base stems).""" try: lines = [f"cause: {cause}", "", "--- stderr tail ---", *tail] - (stems_dir / "vocal_split_error.txt").write_text("\n".join(lines) + "\n", encoding="utf-8") + (stems_dir / filename).write_text("\n".join(lines) + "\n", encoding="utf-8") except OSError: - logger.warning("could not write vocal_split_error.txt in %s", stems_dir, exc_info=True) + logger.warning("could not write %s in %s", filename, stems_dir, exc_info=True) @router.post("/{job_id}/vocal-split") @@ -448,7 +449,9 @@ async def start_vocal_split(job_id: str) -> Response: except Exception as e: cause = classify_failure("\n".join([*(getattr(e, "tail", None) or []), str(e)])) logger.warning("[%s] vocal split failed: %s", job_id, e, exc_info=True) - _write_vocal_split_error(stems_dir, cause, getattr(e, "tail", None) or [str(e)]) + _write_split_error( + stems_dir, "vocal_split_error.txt", cause, getattr(e, "tail", None) or [str(e)] + ) job.vocal_split = "error" _set(job, stage="Done") registry_persist(JOBS_DIR) @@ -471,6 +474,61 @@ async def start_vocal_split(job_id: str) -> Response: return JSONResponse(_job_state(job)) +@router.post("/{job_id}/duet-split") +async def start_duet_split(job_id: str) -> Response: + """Trigger the on-demand duet split for a completed job: a second model + pass over the existing vocals.wav, producing voice_1.wav + + voice_2.wav. Idempotent once done -- calling again returns 202 with the + existing result rather than re-running the (expensive) model. + + Mirrors start_vocal_split above; the two are alternatives, so a job that + already ran one can still run the other.""" + if not JOB_ID_RE.match(job_id): + raise HTTPException(status_code=404, detail="job not found") + job = registry_get(job_id) + if job is None or job.status != "done": + raise HTTPException(status_code=404, detail="job not found") + if job.duet_split == "running": + raise HTTPException(status_code=409, detail="duet split already running") + if job.duet_split == "done": + return JSONResponse(_job_state(job), status_code=202) + + stems_dir = (JOBS_DIR / job_id / "stems").resolve() + if not stems_dir.is_relative_to(JOBS_DIR.resolve()): + raise HTTPException(status_code=404, detail="job not found") + + job.duet_split = "running" + _set(job, stage="Splitting duet voices...") + try: + async with _pipeline_lock: + new_names = await asyncio.to_thread(split_duet, job, stems_dir) + except Exception as e: + cause = classify_failure("\n".join([*(getattr(e, "tail", None) or []), str(e)])) + logger.warning("[%s] duet split failed: %s", job_id, e, exc_info=True) + _write_split_error( + stems_dir, "duet_split_error.txt", cause, getattr(e, "tail", None) or [str(e)] + ) + job.duet_split = "error" + _set(job, stage="Done") + registry_persist(JOBS_DIR) + raise HTTPException(status_code=500, detail="duet split failed") from e + + existing = {s["name"] for s in job.stems} + for name in new_names: + if name not in existing: + job.stems.append({"name": name, "url": f"/api/jobs/{job_id}/stems/{name}.wav"}) + # "vocals" rides along so presence_for_split can recover the scale the base + # stems were normalised against (see start_vocal_split). + rms_values = merge_stem_peaks(stems_dir, ["vocals", *new_names]) + extra_presence = presence_for_split(rms_values, job.stem_presence) + if extra_presence: + job.stem_presence = {**(job.stem_presence or {}), **extra_presence} + job.duet_split = "done" + _set(job, stage="Done") + registry_persist(JOBS_DIR) + return JSONResponse(_job_state(job)) + + _SECTION_ID_RE = re.compile(r"^[a-zA-Z0-9_\-]{1,64}$") _COLOR_RE = re.compile(r"^#[0-9a-fA-F]{3,8}$") _SECTIONS_WRITE_LOCK = threading.Lock() diff --git a/app/api/stems.py b/app/api/stems.py index 0f800c9b..ff916607 100644 --- a/app/api/stems.py +++ b/app/api/stems.py @@ -20,6 +20,7 @@ from app.core.config import ( CACHE_DIR, + DUET_STEM_NAMES, EXTRA_STEM_NAMES, JOB_ID_RE, JOBS_DIR, @@ -41,19 +42,26 @@ # (#275, EXTRA_STEM_NAMES) when a job has requested it. "original" is the # re-encoded source song (added when the user picked a strict subset), "mix" # is the ffmpeg amix of the user's selected stems. -_ALLOWED_NAMES = frozenset(STEM_NAMES) | frozenset(EXTRA_STEM_NAMES) | {"original", "mix"} +_ALLOWED_NAMES = ( + frozenset(STEM_NAMES) + | frozenset(EXTRA_STEM_NAMES) + | frozenset(DUET_STEM_NAMES) + | {"original", "mix"} +) # Lanes the dynamic mixdown may sum: the 6 stems plus "original" (the complement # track shown when the user picked a subset) plus lead/backing vocals when a # job has split them. "mix" is excluded -- it is the static pre-render this # endpoint replaces. Gains are linear; the studio caps a lane at 2.0, so this # generous bound just rejects abusive values. -_MIXDOWN_NAMES = frozenset(STEM_NAMES) | frozenset(EXTRA_STEM_NAMES) | {"original"} +_MIXDOWN_NAMES = ( + frozenset(STEM_NAMES) | frozenset(EXTRA_STEM_NAMES) | frozenset(DUET_STEM_NAMES) | {"original"} +) _MIXDOWN_MAX_GAIN = 4.0 # lead_vocals/backing_vocals are a decomposition of vocals, not an independent # signal -- summing vocals alongside either would double-count the vocal # energy in the mix (#275). -_VOCAL_DECOMPOSITION_NAMES = frozenset(EXTRA_STEM_NAMES) +_VOCAL_DECOMPOSITION_NAMES = frozenset(EXTRA_STEM_NAMES) | frozenset(DUET_STEM_NAMES) # Output encoders by container/extension, shared by the dynamic mixdown and the # stems zip. WAV is lossless PCM, FLAC is lossless compressed, MP3 is VBR ~190 kbps, @@ -811,7 +819,23 @@ async def _ensure_cached_mp3(src: Path) -> Path: @router.get("/jobs/{job_id}/stems/peaks.json") async def get_stem_peaks(job_id: str) -> Response: - """Return pre-computed waveform peaks for all stems.""" + """Return pre-computed waveform peaks for all stems. + + This used to be served `immutable` for a year, on the reasoning that a job's + stems never change once it is done. That stopped being true when the + on-demand splits arrived: both the lead/backing split and the duet split add + stems to a finished job and rewrite this file through `merge_stem_peaks`. + + `immutable` is a promise the browser takes literally. It will not revalidate, + not on a reload and not on a hard reload, so a user who ran a split saw the + waveform from before it for as long as the entry survived. The audio has no + such header, so the two disagreed: the right stems playing under the wrong + picture. + + `must-revalidate` with `max-age=0` keeps the file out of the response body on + a repeat visit -- FileResponse's ETag still answers 304 -- while guaranteeing + the browser asks. + """ if not JOB_ID_RE.match(job_id): raise HTTPException(status_code=404, detail="job not found") job = registry_get(job_id) @@ -823,7 +847,7 @@ async def get_stem_peaks(job_id: str) -> Response: return FileResponse( path, media_type="application/json", - headers={"Cache-Control": "public, max-age=31536000, immutable"}, + headers={"Cache-Control": "public, max-age=0, must-revalidate"}, ) @@ -1364,7 +1388,7 @@ async def get_all_stems_zip( raise HTTPException(status_code=404, detail="job not ready") # Resolve the requested subset (whitelisted) or fall back to all stems. - all_names = (*STEM_NAMES, *EXTRA_STEM_NAMES) + all_names = (*STEM_NAMES, *EXTRA_STEM_NAMES, *DUET_STEM_NAMES) if stems: requested = {s for s in stems.split(",") if s} if not requested <= set(all_names): diff --git a/app/core/config.py b/app/core/config.py index fe9f86ea..98e0c93e 100644 --- a/app/core/config.py +++ b/app/core/config.py @@ -65,6 +65,11 @@ def detect_torch_device() -> str: # GET /api/config contract all assume "the 6 Demucs stems" and must not change # just because a job happened to request this extra pass. EXTRA_STEM_NAMES: tuple[str, ...] = ("lead_vocals", "backing_vocals") +# The on-demand duet split's two stems. Deliberately a separate tuple rather +# than more EXTRA_STEM_NAMES: the two splits are alternatives, and the lane +# ordering swaps "vocals" for a pair only when every name in a pair is +# present, which a single four-name tuple would never satisfy. +DUET_STEM_NAMES: tuple[str, ...] = ("voice_1", "voice_2") JOB_ID_RE = re.compile(r"^[a-f0-9]{12}$") @@ -312,6 +317,28 @@ def js_solver_available() -> bool: # A first run downloads the checkpoint (hundreds of MB); generous default so a # slow connection isn't mistaken for a stall. TIMEOUT_VOCAL_SPLIT = _env_int("STEMDECK_TIMEOUT_VOCAL_SPLIT", 1800) +# On-demand duet split. BS-Roformer trained on male/female vocal separation: +# it splits a duet by vocal weight/register, which works when the two singers +# sit in different registers and does nothing when they do not. Measured on +# "Shallow": 30-40 dB rejection of the idle stem. Measured on two sopranos +# ("What Is This Feeling"): 93/7 energy, no split. STEMDECK_DUET_MODEL lets a +# deployment swap the checkpoint without a code change; see docs/models.md. +DUET_SPLIT_MODEL = ( + os.environ.get("STEMDECK_DUET_MODEL", "").strip() + or "model_chorus_bs_roformer_ep_267_sdr_24.1275.ckpt" +) +# Same rationale as TIMEOUT_VOCAL_SPLIT, and this checkpoint is larger (527 MB). +TIMEOUT_DUET_SPLIT = _env_int("STEMDECK_TIMEOUT_DUET_SPLIT", 1800) +# Force the same-register duet path (app/pipeline/duet_medleyvox.py) instead of +# letting the register model try first. The register model is the better answer +# when the two singers sit in different ranges, and produces a confident, wrong +# one when they do not -- and which of those happened cannot be measured without +# knowing the right answer, so it is offered as a choice rather than guessed. +DUET_SAME_REGISTER = os.environ.get("STEMDECK_DUET_SAME_REGISTER", "").strip().lower() in ( + "1", + "true", + "yes", +) # Beat-grid stage decodes the whole drums stem (not the 180 s analyze window), # so it gets its own, larger budget. TIMEOUT_BEATGRID = _env_int("STEMDECK_TIMEOUT_BEATGRID", 300) diff --git a/app/core/model_cache.py b/app/core/model_cache.py index 0de34568..5e7eecd3 100644 --- a/app/core/model_cache.py +++ b/app/core/model_cache.py @@ -132,3 +132,20 @@ def vocal_split_artifacts(models_dir: Path, model_file: str) -> list[Path]: root / "vr_model_data.json", root / "mdx_model_data.json", ] + + +def duet_split_artifacts(models_dir: Path, model_file: str) -> list[Path]: + """The duet-split checkpoint, its yaml config, and the metadata indexes. + + Same reasoning as vocal_split_artifacts, plus one extra file: the MDXC + roformer family is distributed as a checkpoint *and* a separate yaml + config, and audio-separator needs both. A truncated yaml fails the same way + a truncated checkpoint does, so healing has to be able to remove either. + """ + root = models_dir / "audio-separator" + return [ + root / model_file, + root / "config_chorus_male_female_bs_roformer.yaml", + root / "vr_model_data.json", + root / "mdx_model_data.json", + ] diff --git a/app/core/models.py b/app/core/models.py index 0220e195..579566af 100644 --- a/app/core/models.py +++ b/app/core/models.py @@ -98,6 +98,10 @@ class Job: # app/pipeline/vocal_split.py) and is recorded in stems/vocal_split_error.txt, # not job.error_detail, since the job itself did not fail. vocal_split: Literal["none", "running", "done", "error"] = "none" + # On-demand duet split -- the same shape of post-hoc action as vocal_split + # above, and mutually exclusive with it in the UI, but tracked separately + # so a job can have run one, the other, or neither. + duet_split: Literal["none", "running", "done", "error"] = "none" # When the user put this job in the Trash, or None if they have not. # # Server-side on purpose. The Trash used to live only in the browser's @@ -162,6 +166,7 @@ def to_state(self) -> dict[str, Any]: "gpu_fallback": self.gpu_fallback, "stage_timings": self.stage_timings, "vocal_split": self.vocal_split, + "duet_split": self.duet_split, "trashed_at": self.trashed_at, "created_at": self.created_at, } diff --git a/app/core/registry.py b/app/core/registry.py index 9cc07c87..48a7ce54 100644 --- a/app/core/registry.py +++ b/app/core/registry.py @@ -9,7 +9,7 @@ import uuid from pathlib import Path -from app.core.config import DEMUCS_MODEL, EXTRA_STEM_NAMES, JOB_ID_RE, STEM_NAMES +from app.core.config import DEMUCS_MODEL, DUET_STEM_NAMES, EXTRA_STEM_NAMES, JOB_ID_RE, STEM_NAMES from app.core.models import Job logger = logging.getLogger("stemdeck.registry") @@ -311,7 +311,7 @@ def _recover_done_job(job_dir: Path) -> Job | None: return None stems = [ {"name": name, "url": f"/api/jobs/{job_dir.name}/stems/{name}.wav"} - for name in ("original", *STEM_NAMES, *EXTRA_STEM_NAMES) + for name in ("original", *STEM_NAMES, *EXTRA_STEM_NAMES, *DUET_STEM_NAMES) if (stems_dir / f"{name}.wav").is_file() ] if not stems: @@ -326,6 +326,8 @@ def _recover_done_job(job_dir: Path) -> Job | None: # regress (#275). has_split = all((stems_dir / f"{name}.wav").is_file() for name in EXTRA_STEM_NAMES) vocal_split = "done" if has_split else "none" + has_duet = all((stems_dir / f"{name}.wav").is_file() for name in DUET_STEM_NAMES) + duet_split = "done" if has_duet else "none" meta_path = job_dir / "metadata.json" meta: dict = {} if meta_path.is_file(): @@ -369,6 +371,7 @@ def _recover_done_job(job_dir: Path) -> Job | None: sections_source=meta.get("sections_source"), tags=meta.get("tags"), vocal_split=vocal_split, + duet_split=duet_split, ) diff --git a/app/pipeline/_convtasnet.py b/app/pipeline/_convtasnet.py new file mode 100644 index 00000000..a4d04080 --- /dev/null +++ b/app/pipeline/_convtasnet.py @@ -0,0 +1,500 @@ +"""Conv-TasNet masker, vendored from asteroid (MIT), plus the StemDeck wrapper. + +The duet separator needs four things out of `asteroid`: a Conv-TasNet masker, a +global layer norm, and two shape helpers. Depending on the package for them +costs 23 transitive packages including pytorch-lightning, pandas and pesq, which +is not a reasonable thing to put inside a packaged desktop app, and any of it +landing in uv.lock sends every existing install through a full re-download. + +So the pieces are copied here instead. `asteroid` is MIT licensed, and this is +its code, unmodified apart from removing branches nothing on this path uses. + + asteroid -- https://github.com/asteroid-team/asteroid + Copyright (c) 2019 Pariente Manuel + MIT, full text in app/_vendor/LICENSE.asteroid + +MIT asks for the permission notice to travel with the code, not just the +copyright line, so the licence sits beside the yt-dlp-ejs one rather than being +paraphrased here. Everything below MixtureConsistent is StemDeck's own and +Apache-2.0 like the rest of the project. + +`asteroid-filterbanks` is still a real dependency: it supplies the STFT encoder +and decoder, and unlike `asteroid` it pulls in nothing of its own. + +Do not hand-edit the vendored classes. They must stay byte-identical to the +version the published checkpoint was trained against or the weights stop +loading. Regenerate instead. +""" + +import functools +import inspect +from typing import List, Optional + +import torch +from torch import nn +from torch.jit import is_tracing + + +# The checkpoint's config asks for norm_type "gLN" and mask_act "linear"; these +# stand in for asteroid.masknn.{norms,activations}.get without the registries. +class _Norms: + @staticmethod + def get(identifier): + if identifier in (None, "gLN", "GlobLN"): + return GlobLN + raise ValueError(f"vendored build supports gLN only, got {identifier!r}") + + +class _Activations: + @staticmethod + def get(identifier): + if identifier in (None, "linear"): + return nn.Identity + if identifier == "relu": + return nn.ReLU + if identifier == "sigmoid": + return nn.Sigmoid + raise ValueError(f"vendored build does not support mask_act {identifier!r}") + + +norms = _Norms() +activations = _Activations() + + +def script_if_tracing(fn): + # Taken for pytorch for compat in 1.6.0 + """ + Compiles ``fn`` when it is first called during tracing. ``torch.jit.script`` + has a non-negligible start up time when it is first called due to + lazy-initializations of many compiler builtins. Therefore you should not use + it in library code. However, you may want to have parts of your library work + in tracing even if they use control flow. In these cases, you should use + ``@torch.jit.script_if_tracing`` to substitute for + ``torch.jit.script``. + + Arguments: + fn: A function to compile. + + Returns: + If called during tracing, a :class:`ScriptFunction` created by ` + `torch.jit.script`` is returned. Otherwise, the original function ``fn`` is returned. + """ + + @functools.wraps(fn) + def wrapper(*args, **kwargs): + if not is_tracing(): + # Not tracing, don't do anything + return fn(*args, **kwargs) + + compiled_fn = torch.jit.script(wrapper.__original_fn) # type: ignore + return compiled_fn(*args, **kwargs) + + wrapper.__original_fn = fn # type: ignore + wrapper.__script_if_tracing_wrapper = True # type: ignore + + return wrapper + + +@script_if_tracing +def pad_x_to_y(x: torch.Tensor, y: torch.Tensor, axis: int = -1) -> torch.Tensor: + """Right-pad or right-trim first argument to have same size as second argument + + Args: + x (torch.Tensor): Tensor to be padded. + y (torch.Tensor): Tensor to pad `x` to. + axis (int): Axis to pad on. + + Returns: + torch.Tensor, `x` padded to match `y`'s shape. + """ + if axis != -1: + raise NotImplementedError + inp_len = y.shape[axis] + output_len = x.shape[axis] + return nn.functional.pad(x, [0, inp_len - output_len]) + + +@script_if_tracing +def jitable_shape(tensor): + """Gets shape of ``tensor`` as ``torch.Tensor`` type for jit compiler + + .. note:: + Returning ``tensor.shape`` of ``tensor.size()`` directly is not torchscript + compatible as return type would not be supported. + + Args: + tensor (torch.Tensor): Tensor + + Returns: + torch.Tensor: Shape of ``tensor`` + """ + return torch.tensor(tensor.shape) + + +def has_arg(fn, name): + """Checks if a callable accepts a given keyword argument. + + Args: + fn (callable): Callable to inspect. + name (str): Check if ``fn`` can be called with ``name`` as a keyword + argument. + + Returns: + bool: whether ``fn`` accepts a ``name`` keyword argument. + """ + signature = inspect.signature(fn) + parameter = signature.parameters.get(name) + if parameter is None: + return False + return parameter.kind in ( + inspect.Parameter.POSITIONAL_OR_KEYWORD, + inspect.Parameter.KEYWORD_ONLY, + ) + + +def z_norm(x, dims: List[int], eps: float = 1e-8): + mean = x.mean(dim=dims, keepdim=True) + var2 = torch.var(x, dim=dims, keepdim=True, unbiased=False) + value = (x - mean) / torch.sqrt((var2 + eps)) + return value + + +@script_if_tracing +def _glob_norm(x, eps: float = 1e-8): + dims: List[int] = torch.arange(1, len(x.shape)).tolist() + return z_norm(x, dims, eps) + + +class _LayerNorm(nn.Module): + """Layer Normalization base class.""" + + def __init__(self, channel_size): + super(_LayerNorm, self).__init__() + self.channel_size = channel_size + self.gamma = nn.Parameter(torch.ones(channel_size), requires_grad=True) + self.beta = nn.Parameter(torch.zeros(channel_size), requires_grad=True) + + def apply_gain_and_bias(self, normed_x): + """Assumes input of size `[batch, chanel, *]`.""" + return (self.gamma * normed_x.transpose(1, -1) + self.beta).transpose(1, -1) + + +class GlobLN(_LayerNorm): + """Global Layer Normalization (globLN).""" + + def forward(self, x, EPS: float = 1e-8): + """Applies forward pass. + + Works for any input size > 2D. + + Args: + x (:class:`torch.Tensor`): Shape `[batch, chan, *]` + + Returns: + :class:`torch.Tensor`: gLN_x `[batch, chan, *]` + """ + value = _glob_norm(x, eps=EPS) + return self.apply_gain_and_bias(value) + + +class _Chop1d(nn.Module): + """To ensure the output length is the same as the input.""" + + def __init__(self, chop_size): + super().__init__() + self.chop_size = chop_size + + def forward(self, x): + return x[..., : -self.chop_size].contiguous() + + +class Conv1DBlock(nn.Module): + """One dimensional convolutional block, as proposed in [1]. + + Args: + in_chan (int): Number of input channels. + hid_chan (int): Number of hidden channels in the depth-wise + convolution. + skip_out_chan (int): Number of channels in the skip convolution. + If 0 or None, `Conv1DBlock` won't have any skip connections. + Corresponds to the the block in v1 or the paper. The `forward` + return res instead of [res, skip] in this case. + kernel_size (int): Size of the depth-wise convolutional kernel. + padding (int): Padding of the depth-wise convolution. + dilation (int): Dilation of the depth-wise convolution. + norm_type (str, optional): Type of normalization to use. To choose from + + - ``'gLN'``: global Layernorm. + - ``'cLN'``: channelwise Layernorm. + - ``'cgLN'``: cumulative global Layernorm. + - Any norm supported by :func:`~.norms.get` + causal (bool, optional) : Whether or not the convolutions are causal + + + References + [1] : "Conv-TasNet: Surpassing ideal time-frequency magnitude masking + for speech separation" TASLP 2019 Yi Luo, Nima Mesgarani + https://arxiv.org/abs/1809.07454 + """ + + def __init__( + self, + in_chan, + hid_chan, + skip_out_chan, + kernel_size, + padding, + dilation, + norm_type="gLN", + causal=False, + ): + super(Conv1DBlock, self).__init__() + self.skip_out_chan = skip_out_chan + conv_norm = norms.get(norm_type) + in_conv1d = nn.Conv1d(in_chan, hid_chan, 1) + depth_conv1d = nn.Conv1d( + hid_chan, hid_chan, kernel_size, padding=padding, dilation=dilation, groups=hid_chan + ) + if causal: + depth_conv1d = nn.Sequential(depth_conv1d, _Chop1d(padding)) + self.shared_block = nn.Sequential( + in_conv1d, + nn.PReLU(), + conv_norm(hid_chan), + depth_conv1d, + nn.PReLU(), + conv_norm(hid_chan), + ) + self.res_conv = nn.Conv1d(hid_chan, in_chan, 1) + if skip_out_chan: + self.skip_conv = nn.Conv1d(hid_chan, skip_out_chan, 1) + + def forward(self, x): + r"""Input shape $(batch, feats, seq)$.""" + shared_out = self.shared_block(x) + res_out = self.res_conv(shared_out) + if not self.skip_out_chan: + return res_out + skip_out = self.skip_conv(shared_out) + return res_out, skip_out + + +class TDConvNet(nn.Module): + """Temporal Convolutional network used in ConvTasnet. + + Args: + in_chan (int): Number of input filters. + n_src (int): Number of masks to estimate. + out_chan (int, optional): Number of bins in the estimated masks. + If ``None``, `out_chan = in_chan`. + n_blocks (int, optional): Number of convolutional blocks in each + repeat. Defaults to 8. + n_repeats (int, optional): Number of repeats. Defaults to 3. + bn_chan (int, optional): Number of channels after the bottleneck. + hid_chan (int, optional): Number of channels in the convolutional + blocks. + skip_chan (int, optional): Number of channels in the skip connections. + If 0 or None, TDConvNet won't have any skip connections and the + masks will be computed from the residual output. + Corresponds to the ConvTasnet architecture in v1 or the paper. + conv_kernel_size (int, optional): Kernel size in convolutional blocks. + norm_type (str, optional): To choose from ``'BN'``, ``'gLN'``, + ``'cLN'``. + mask_act (str, optional): Which non-linear function to generate mask. + causal (bool, optional) : Whether or not the convolutions are causal. + + References + [1] : "Conv-TasNet: Surpassing ideal time-frequency magnitude masking + for speech separation" TASLP 2019 Yi Luo, Nima Mesgarani + https://arxiv.org/abs/1809.07454 + """ + + def __init__( + self, + in_chan, + n_src, + out_chan=None, + n_blocks=8, + n_repeats=3, + bn_chan=128, + hid_chan=512, + skip_chan=128, + conv_kernel_size=3, + norm_type="gLN", + mask_act="relu", + causal=False, + ): + super(TDConvNet, self).__init__() + self.in_chan = in_chan + self.n_src = n_src + out_chan = out_chan if out_chan else in_chan + self.out_chan = out_chan + self.n_blocks = n_blocks + self.n_repeats = n_repeats + self.bn_chan = bn_chan + self.hid_chan = hid_chan + self.skip_chan = skip_chan + self.conv_kernel_size = conv_kernel_size + self.norm_type = norm_type + self.mask_act = mask_act + self.causal = causal + + layer_norm = norms.get(norm_type)(in_chan) + bottleneck_conv = nn.Conv1d(in_chan, bn_chan, 1) + self.bottleneck = nn.Sequential(layer_norm, bottleneck_conv) + # Succession of Conv1DBlock with exponentially increasing dilation. + self.TCN = nn.ModuleList() + for r in range(n_repeats): + for x in range(n_blocks): + if not causal: + padding = (conv_kernel_size - 1) * 2**x // 2 + else: + padding = (conv_kernel_size - 1) * 2**x + self.TCN.append( + Conv1DBlock( + bn_chan, + hid_chan, + skip_chan, + conv_kernel_size, + padding=padding, + dilation=2**x, + norm_type=norm_type, + causal=causal, + ) + ) + mask_conv_inp = skip_chan if skip_chan else bn_chan + mask_conv = nn.Conv1d(mask_conv_inp, n_src * out_chan, 1) + self.mask_net = nn.Sequential(nn.PReLU(), mask_conv) + # Get activation function. + mask_nl_class = activations.get(mask_act) + # For softmax, feed the source dimension. + if has_arg(mask_nl_class, "dim"): + self.output_act = mask_nl_class(dim=1) + else: + self.output_act = mask_nl_class() + + def forward(self, mixture_w): + r"""Forward. + + Args: + mixture_w (:class:`torch.Tensor`): Tensor of shape $(batch, nfilters, nframes)$ + + Returns: + :class:`torch.Tensor`: estimated mask of shape $(batch, nsrc, nfilters, nframes)$ + """ + batch, _, n_frames = mixture_w.size() + output = self.bottleneck(mixture_w) + skip_connection = torch.tensor([0.0], device=output.device) + for layer in self.TCN: + # Common to w. skip and w.o skip architectures + tcn_out = layer(output) + if self.skip_chan: + residual, skip = tcn_out + skip_connection = skip_connection + skip + else: + residual = tcn_out + output = output + residual + # Use residual output when no skip connection + mask_inp = skip_connection if self.skip_chan else output + score = self.mask_net(mask_inp) + score = score.view(batch, self.n_src, self.out_chan, n_frames) + est_mask = self.output_act(score) + return est_mask + + def get_config(self): + config = { + "in_chan": self.in_chan, + "out_chan": self.out_chan, + "bn_chan": self.bn_chan, + "hid_chan": self.hid_chan, + "skip_chan": self.skip_chan, + "conv_kernel_size": self.conv_kernel_size, + "n_blocks": self.n_blocks, + "n_repeats": self.n_repeats, + "n_src": self.n_src, + "norm_type": self.norm_type, + "mask_act": self.mask_act, + "causal": self.causal, + } + return config + + +@script_if_tracing +def _unsqueeze_to_3d(x): + """Normalize shape of `x` to [batch, n_chan, time].""" + if x.ndim == 1: + return x.reshape(1, 1, -1) + elif x.ndim == 2: + return x.unsqueeze(1) + else: + return x + + +@script_if_tracing +def _shape_reconstructed(reconstructed, size): + """Reshape `reconstructed` to have same size as `size` + + Args: + reconstructed (torch.Tensor): Reconstructed waveform + size (torch.Tensor): Size of desired waveform + + Returns: + torch.Tensor: Reshaped waveform + + """ + if len(size) == 1: + return reconstructed.squeeze(0) + return reconstructed + + +class MixtureConsistent(nn.Module): + """Encoder / masker / decoder whose two outputs are made to sum to the input. + + Wisdom et al., "Differentiable consistency constraints for improved deep + speech enhancement", ICASSP 2019: share whatever the estimates failed to + account for equally between them. This is why the two stems add back up to + the recording, so soloing and muting behave. + + Replaces asteroid's BaseEncoderMaskerDecoder, whose base class reaches for + huggingface_hub at import time. + """ + + def __init__(self, encoder, masker, decoder, encoder_activation=None): + super().__init__() + self.encoder = encoder + self.masker = masker + self.decoder = decoder + self.enc_activation = activations.get(encoder_activation or "linear")() + + def forward(self, wav): + shape = jitable_shape(wav) + wav = _unsqueeze_to_3d(wav) + tf = self.enc_activation(self.encoder(wav)) + masked = self.masker(tf) * tf.unsqueeze(1) + out = _shape_reconstructed(pad_x_to_y(self.decoder(masked), wav), shape) + return out + (wav - out.sum(dim=1, keepdim=True)) / out.shape[1] + + +def build_from_config(cfg): + """Build the separator a checkpoint's own config describes.""" + from asteroid_filterbanks import make_enc_dec + + enc, dec = make_enc_dec( + "torch_stft", + n_filters=cfg["nfft"], + kernel_size=cfg["nfft"], + stride=cfg["nhop"], + sample_rate=cfg["sample_rate"], + ) + masker = TDConvNet( + in_chan=enc.n_feats_out, + n_src=cfg["n_src"], + out_chan=None, + n_blocks=cfg["n_blocks"], + n_repeats=cfg["n_repeats"], + bn_chan=cfg["bn_chan"], + hid_chan=cfg["hid_chan"], + skip_chan=cfg["skip_chan"], + mask_act=cfg["mask_act"], + ) + return MixtureConsistent(enc, masker, dec, encoder_activation=cfg["encoder_activation"]) diff --git a/app/pipeline/duet_medleyvox.py b/app/pipeline/duet_medleyvox.py new file mode 100644 index 00000000..bd2ea65b --- /dev/null +++ b/app/pipeline/duet_medleyvox.py @@ -0,0 +1,257 @@ +"""Separate two singers of the same vocal range. + +The register model in `duet_split_worker.py` splits a duet by vocal weight. That +works when the two singers sit in different ranges and does not when they do +not: measured on "What Is This Feeling" (Wicked, two sopranos) it led with one +stem in 76% of frames and the other in 7%, which is one voice with gaps rather +than a duet. This is the path for that case. + +It runs the MedleyVox Conv-TasNet, which is trained on multi-singer mixtures +rather than on singing-versus-speech, and is the only open model measured to +separate simultaneous same-register singing at all. + +Measured against ground-truth duets built from real material, with an ideal-mask +ceiling of +13.3 dB on the same audio: + + the other singer ends up 14.5 dB down (no split is 0 dB, ideal mask is 24.5) + +7.3 dB SI-SDRi, roughly 55% of what any masking method could reach + around 7x realtime on CPU, faster on GPU + +What it does *not* do is choruses. The model has two outputs, so once an +ensemble joins, both stems contain everyone. That cannot be detected from the +audio, before or after -- see `register_split_collapsed` for the five measures +tried and what each one missed -- so it is a documented limit rather than a +guard, and STEMDECK_DUET_SAME_REGISTER lets someone who has listened choose. + +Weights are Carson Evans' training run, published CC-BY-4.0, fetched on first +use the way Demucs already is. The model code is `_convtasnet.py`, vendored from +asteroid (MIT); the upstream MedleyVox inference repo carries no licence at all +and is deliberately not used. +""" + +from __future__ import annotations + +import json +import logging +from pathlib import Path + +import numpy as np + +logger = logging.getLogger("stemdeck.pipeline") + +# The model's own rate. Everything below decides *which singer* at this rate and +# then applies the decision at the file's rate, so the output keeps its top end. +MODEL_SR = 24000 + +_REPO = "https://huggingface.co/Cyru5/MedleyVox/resolve/main" +# Two checkpoints of the same run at different epochs. They fail differently on +# the same audio, so averaging their masks is worth +0.1 to +0.2 dB for one +# extra forward pass. Both are from the multi-singing training; the +# singing_librispeech ones in the same repo score less than half as well here +# because they were trained to pull singing away from *speech*. +CHECKPOINTS = ("multi_singing_librispeech", "multi_singing_librispeech_138") + +_NPER, _NOV = 4096, 3072 + + +def artifacts(models_dir: Path) -> list[Path]: + """Every file this model owns, for `load_or_heal` to clear and re-fetch.""" + root = models_dir / "medleyvox" + return [root / name / fn for name in CHECKPOINTS for fn in ("vocals.json", "vocals.pth")] + + +def _fetch(models_dir: Path) -> list[tuple[Path, Path]]: + """Download both checkpoints if absent. Returns (config, weights) pairs.""" + import requests + + out = [] + for name in CHECKPOINTS: + d = models_dir / "medleyvox" / name + d.mkdir(parents=True, exist_ok=True) + for fn in ("vocals.json", "vocals.pth"): + dest = d / fn + if dest.is_file() and dest.stat().st_size > 1024: + continue + url = f"{_REPO}/{name}/{fn}" + logger.info("duet: fetching %s/%s", name, fn) + # .part first: a download cut short must not leave a file that looks + # present and fails to load for good, which is the shape of #502. + part = dest.with_suffix(dest.suffix + ".part") + with requests.get(url, stream=True, timeout=120) as r: + r.raise_for_status() + with open(part, "wb") as f: + for chunk in r.iter_content(1 << 20): + f.write(chunk) + part.replace(dest) + out.append((d / "vocals.json", d / "vocals.pth")) + return out + + +def load_models(models_dir: Path, device: str = "cpu"): + import torch + + from app.pipeline._convtasnet import build_from_config + + models = [] + for cfg_path, pth_path in _fetch(models_dir): + cfg = json.loads(cfg_path.read_text(encoding="utf-8"))["args"] + model = build_from_config(cfg) + # weights_only, because a checkpoint is a pickle and an unrestricted + # load executes whatever it contains. This one is fetched over the + # network at first use, so the file is not something the app has seen + # before it opens it. The contents are plain tensors either way. + raw = torch.load(pth_path, map_location="cpu", weights_only=True) + want = model.state_dict() + # Training kept an exponential moving average beside the live weights. + # The EMA copy is the one that was evaluated, and it is the better one. + got = { + k.replace("ema_model.module.", ""): v + for k, v in raw.items() + if k.startswith("ema_model.module.") and k.replace("ema_model.module.", "") in want + } + if len(got) != len(want): + raise ValueError( + f"duet checkpoint {cfg_path.parent.name} covers {len(got)} of " + f"{len(want)} tensors; the file is probably truncated" + ) + model.load_state_dict(got) + model.eval() + models.append(model.to(device)) + return models + + +def _separate_once(model, mix, device): + import pyloudnorm as pyln + import torch + + meter = pyln.Meter(MODEL_SR) + gain = -24.0 - meter.integrated_loudness(mix.astype(np.float32)) + x = torch.as_tensor( + (mix * 10 ** (gain / 20)).reshape(1, 1, -1), dtype=torch.float32, device=device + ) + with torch.no_grad(): + y = model(x) + return [ + y[0, i, : len(mix)].cpu().numpy().astype(np.float64) * 10 ** (-gain / 20) for i in (0, 1) + ] + + +def _masks(est, sr, nper, nov): + from scipy.signal import stft + + S = [stft(e, sr, nperseg=nper, noverlap=nov)[2] for e in est] + power = np.abs(S[0]) ** 2 + np.abs(S[1]) ** 2 + 1e-12 + return [np.abs(S[k]) ** 2 / power for k in (0, 1)] + + +def separate(models, wav, sr, device="cpu"): + """Two stems at the input's own sample rate, summing back to the input. + + Each model sees the audio forwards and reversed, which is a genuinely + different view for a convolutional network and worth about +0.2 dB. Every + view becomes a soft mask, the masks are aligned and averaged, and the + average is applied to the *original-rate* spectrogram. + + That last step matters. The model is 24 kHz, so separating and resampling + back would throw away everything above 12 kHz -- on a vocal, the air and + most of the sibilance. The mask is resampled onto the real grid instead, in + Hz and seconds rather than by bin index: bin 40 is 234 Hz at 24 kHz and + 431 Hz at 44.1 kHz, and frames are 85 ms against 46 ms, so index-wise + copying shears the mask across both axes and costs about 7.8 dB. + """ + from scipy.signal import istft, resample_poly, stft + + mono = wav.mean(1) if wav.ndim > 1 else wav + lo = resample_poly(mono.astype(np.float64), MODEL_SR, sr) + + ref, acc, n = None, None, 0 + for model in models: + rev = _separate_once(model, lo[::-1].copy(), device) + for view in (_separate_once(model, lo, device), [r[::-1].copy() for r in rev]): + m = _masks(view, MODEL_SR, _NPER, _NOV) + if ref is None: + ref, acc = m, [a.copy() for a in m] + else: + # Which output is which singer is arbitrary per view. Average + # them unaligned and both stems become both singers. + keep = sum(np.mean(np.abs(m[i] - ref[i])) for i in (0, 1)) + swap = sum(np.mean(np.abs(m[1 - i] - ref[i])) for i in (0, 1)) + if swap < keep: + m = m[::-1] + acc = [a + b for a, b in zip(acc, m, strict=True)] + n += 1 + avg = [a / n for a in acc] + + f_lo, t_lo, _ = stft(lo, MODEL_SR, nperseg=_NPER, noverlap=_NOV) + f_hi, t_hi, M = stft(mono, sr, nperseg=_NPER, noverlap=_NOV) + grids = [] + for k in (0, 1): + a = np.empty((len(f_lo), len(t_hi))) + for i in range(len(f_lo)): + a[i] = np.interp(t_hi, t_lo, avg[k][i]) + b = np.empty((len(f_hi), len(t_hi))) + for j in range(len(t_hi)): + # np.interp clamps past the end of f_lo, which is exactly the wanted + # "hold the highest band the model saw" behaviour above 12 kHz. + b[:, j] = np.interp(f_hi, f_lo, a[:, j]) + grids.append(b) + + total = grids[0] + grids[1] + 1e-12 + # One mask per singer, decided on the downmix, applied to every channel. + # Deciding per channel would let the two sides disagree about who is + # singing and smear the image; applying to only the downmix would hand back + # mono stems while every other stem in the job is stereo, which breaks pan + # and width in the mixer. + channels = wav.shape[1] if wav.ndim > 1 else 1 + stems = [] + for k in (0, 1): + mask = grids[k] / total + out = np.empty((len(mono), channels), dtype=np.float64) + for ch in range(channels): + src = wav[:, ch].astype(np.float64) if wav.ndim > 1 else mono + _, _, C = stft(src, sr, nperseg=_NPER, noverlap=_NOV) + cols = min(mask.shape[1], C.shape[1]) + y = np.real(istft(C[:, :cols] * mask[:, :cols], sr, nperseg=_NPER, noverlap=_NOV)[1]) + y = y[: len(mono)] + if len(y) < len(mono): + y = np.pad(y, (0, len(mono) - len(y))) + out[:, ch] = y + stems.append(out[:, 0] if channels == 1 else out) + return stems + + +def register_split_collapsed(path_a: Path, path_b: Path, min_share: float = 0.12) -> bool: + """Did the register model put essentially everything in one stem? + + This is the only property of a split that can be judged without knowing the + right answer, and it is deliberately the *only* test made here. Four richer + ones were tried and all of them passed a split that had not happened: + + energy share -- misses two copies of one performance at 40/60 + share spread -- a stem holding almost everything, with gaps, swings + between extremes and scores 0.287, higher than some + real splits + who leads -- measures whether a song has singers taking turns, + not whether separation worked; scored 0.000 on + ground-truth stems of a simultaneous duet + stem correlation -- the bad split scored 0.09, better than the model's + and decisiveness own correct output at 0.17-0.29, because it is + confidently wrong rather than confused + + The lesson is that a wrong separation looks exactly like a right one unless + you already know the answer. So the cascade turns only on the unambiguous + case -- one stem is essentially silent -- and anything subtler is the user's + call via STEMDECK_DUET_SAME_REGISTER rather than a guess made on their + behalf. + """ + import soundfile as sf + + def energy(p: Path) -> float: + x, _ = sf.read(str(p), dtype="float32", always_2d=True) + return float(np.sum(x.astype(np.float64) ** 2)) + + ea, eb = energy(path_a), energy(path_b) + total = ea + eb + if total <= 0: + return True + return min(ea, eb) / total < min_share diff --git a/app/pipeline/duet_split.py b/app/pipeline/duet_split.py new file mode 100644 index 00000000..b52f647e --- /dev/null +++ b/app/pipeline/duet_split.py @@ -0,0 +1,79 @@ +"""On-demand duet split: separate two singers out of the vocals stem. + +A post-hoc action on an already-"done" job, the same shape as the lead/backing +vocal split in `vocal_split.py`, sharing its subprocess supervision via +`split_runner.run_split`. + +What this can and cannot do +--------------------------- +The model separates voices by **vocal weight/register**, not by singer +identity. It splits a duet cleanly when the two singers sit in different +registers -- measured on "Shallow" (Cooper/Gaga), the quiet stem is suppressed +by 30-40 dB through the other singer's verse, and the stems alternate with the +song (per-frame energy share std 0.41 across the excerpt). + +It does **not** split two singers of the same register. On "What Is This +Feeling" (Wicked, two sopranos) it routes 93% of the energy to one stem and +alternates not at all. Same-register duets are an unsolved problem, not a +tuning gap: a speech separator (SepFormer) and the purpose-built singing +separator (MedleyVox iSRNet) were both measured on the same material and +neither produced any split at all. See docs/models.md. + +Stems are therefore named for what the model actually keys on, so the feature +does not promise per-singer separation it cannot deliver. +""" + +from __future__ import annotations + +import logging +import sys +from pathlib import Path + +from app.core.config import TIMEOUT_DUET_SPLIT +from app.core.models import Job +from app.core.settings import get_demucs_device +from app.pipeline.errors import SeparationError +from app.pipeline.split_runner import run_split + +logger = logging.getLogger("stemdeck.pipeline") + +DUET_STEMS = ["voice_1", "voice_2"] + + +def _spawn_cmd(device: str, vocals_path: Path, out_dir: Path) -> list[str]: + """The worker argv. Kept as its own function so tests can swap in a stub + worker and still exercise the real subprocess machinery.""" + return [ + sys.executable, + "-m", + "app.pipeline.duet_split_worker", + device, + str(vocals_path), + str(out_dir), + ] + + +def split_duet(job: Job, stems_dir: Path) -> list[str]: + """Split stems/vocals.wav into voice_1.wav + voice_2.wav in stems_dir. + + Returns the two new stem names on success. Raises SeparationError (carrying + the stderr tail) on failure. + + Best-effort by design: the caller is responsible for catching failures and + leaving the job's base stems untouched -- this never mutates anything but + stems_dir's contents, and never touches vocals.wav itself. + """ + vocals_path = stems_dir / "vocals.wav" + if not vocals_path.is_file(): + raise SeparationError("vocals.wav not found -- job has no vocals stem to split") + + device = get_demucs_device() + return run_split( + job, + argv=_spawn_cmd(device, vocals_path, stems_dir), + device=device, + timeout=TIMEOUT_DUET_SPLIT, + label="duet split", + produces=DUET_STEMS, + out_dir=stems_dir, + ) diff --git a/app/pipeline/duet_split_worker.py b/app/pipeline/duet_split_worker.py new file mode 100644 index 00000000..77c42b75 --- /dev/null +++ b/app/pipeline/duet_split_worker.py @@ -0,0 +1,149 @@ +"""On-demand duet split worker. + +Run as its own process: `python -m app.pipeline.duet_split_worker + `. Like vocal_split_worker.py and unlike demucs_worker.py +this is **not** persistent -- the split is an occasional, user-triggered action +on an already-done job, so there is no repeat model-load cost worth amortising. + +Runs the BS-Roformer male/female model (or STEMDECK_DUET_MODEL's override) via +`audio-separator` on an already-isolated vocals stem, producing voice_1.wav + +voice_2.wav in out_dir. + +Protocol, matching demucs_worker.py's contract so the parent's stderr reader +can be reused: writes progress/log lines to stderr, then exactly one of: + "@@DONE@@" -- success, both files written + "@@ERROR@@" -- failure; process exits(1) +""" + +from __future__ import annotations + +import json +import os +import sys + +from app.core.config import DUET_SPLIT_MODEL +from app.core.process import arm_parent_watchdog + + +def _run(device: str, vocals_path: str, out_dir: str) -> None: + """Split by register, unless the singers share one. + + The two models answer different questions. The register model separates by + vocal weight and gives 30-40 dB of rejection when the singers sit in + different ranges, which is the better result whenever it applies. The + same-register model reaches about 14.5 dB but does not care about range. + + Picking between them automatically turns out not to be possible from the + audio alone: a wrong separation is confident and uncorrelated, exactly like + a right one, and every blind measure tried scored the register model's + failure on this project's test song as healthy (see + duet_medleyvox.register_split_collapsed for the four that were tried and + what each missed). + + So the automatic path handles only the unambiguous failure -- one stem + essentially silent -- and STEMDECK_DUET_SAME_REGISTER is how someone who has + listened says which one they want. + """ + from pathlib import Path + + from app.core.config import DUET_SAME_REGISTER + from app.pipeline.duet_medleyvox import register_split_collapsed + + if DUET_SAME_REGISTER: + sys.stderr.write("duet: same-register model requested\n") + _run_same_register(device, vocals_path, out_dir) + return + + _run_register(device, vocals_path, out_dir) + + a, b = Path(out_dir) / "voice_1.wav", Path(out_dir) / "voice_2.wav" + if not (a.is_file() and b.is_file()): + raise RuntimeError("register split produced no stems") + if not register_split_collapsed(a, b): + return + + sys.stderr.write("duet: register split collapsed into one stem, trying same-register\n") + _run_same_register(device, vocals_path, out_dir) + + +def _run_same_register(device: str, vocals_path: str, out_dir: str) -> None: + """Separate two singers of the same range, overwriting the register stems.""" + from pathlib import Path + + import soundfile as sf + + from app.core.config import MODELS_DIR + from app.pipeline.duet_medleyvox import load_models, separate + + wav, sr = sf.read(vocals_path, dtype="float32", always_2d=True) + models = load_models(MODELS_DIR, device) + stems = separate(models, wav, sr, device) + + for name, y in zip(("voice_1.wav", "voice_2.wav"), stems, strict=True): + sf.write(str(Path(out_dir) / name), y.astype("float32"), sr) + e = [float((s.astype(float) ** 2).sum()) for s in stems] + total = sum(e) or 1.0 + sys.stderr.write( + f"duet: same-register split, energy {100 * e[0] / total:.0f}/{100 * e[1] / total:.0f}\n" + ) + + +def _run_register(device: str, vocals_path: str, out_dir: str) -> None: + if device == "cpu": + # audio-separator/onnxruntime pick their execution provider mostly on + # their own; hiding CUDA devices is the reliable way to force the CPU + # provider when the caller explicitly resolved to "cpu" (matches the + # separation stage's own device policy in app.core.settings). + os.environ.setdefault("CUDA_VISIBLE_DEVICES", "") + + from audio_separator.separator import Separator + + from app.core.config import MODELS_DIR + from app.core.model_cache import duet_split_artifacts, load_or_heal + + separator = Separator( + log_level=40, # logging.ERROR -- only the lines we emit ourselves matter + model_file_dir=str(MODELS_DIR / "audio-separator"), + output_dir=out_dir, + output_format="WAV", + ) + # This is the only path Docker ever takes: there is no warmup step there, so + # the model is fetched here on first use. A download cut short leaves a file + # that audio-separator will keep accepting as present and keep failing to + # load, which is #502 with no setup step to have caught it earlier. + load_or_heal( + lambda: separator.load_model(model_filename=DUET_SPLIT_MODEL), + lambda: duet_split_artifacts(MODELS_DIR, DUET_SPLIT_MODEL), + ) + # The model labels its two outputs "Male" and "Female", which is the label + # set it was trained on rather than a claim about who is singing: what it + # actually keys on is vocal weight/register. Renaming at the library + # boundary avoids depending on audio-separator's default filename format. + separator.separate( + vocals_path, + {"Male": "voice_1", "Female": "voice_2"}, + ) + + +def main() -> None: + # A Force-Quit of the app otherwise orphans this process holding the GPU: + # onnxruntime reads nothing from stdin mid-inference, so EOF never arrives + # and nothing else bounds it (#519). + arm_parent_watchdog() + if len(sys.argv) < 4: + sys.stderr.write("@@ERROR@@usage: duet_split_worker \n") + sys.stderr.flush() + sys.exit(1) + device, vocals_path, out_dir = sys.argv[1], sys.argv[2], sys.argv[3] + try: + _run(device, vocals_path, out_dir) + except Exception as e: + sys.stderr.write(f"@@ERROR@@{json.dumps(str(e))}\n") + sys.stderr.flush() + sys.exit(1) + sys.stderr.write("@@DONE@@\n") + sys.stderr.flush() + + +if __name__ == "__main__": + main() diff --git a/app/pipeline/split_runner.py b/app/pipeline/split_runner.py new file mode 100644 index 00000000..7a8e6b95 --- /dev/null +++ b/app/pipeline/split_runner.py @@ -0,0 +1,154 @@ +"""Shared supervision for the on-demand vocal-split worker subprocesses. + +Both the lead/backing split (#275) and the duet split run the same shape of +job: a fresh, non-persistent subprocess that loads one `audio-separator` model, +writes a fixed set of wav files next to the job's stems, and reports over the +stderr line protocol that `demucs_worker.py` established. Only the argv, the +timeout and the expected output names differ, so the process lifecycle, +stall watchdog and error hygiene live here once rather than per split. + +Deliberately not a persistent worker: these are occasional, user-triggered +actions rather than the hot path every job takes, so there is no repeat +model-load cost worth amortising (see ml-pipeline.md / #309 for why that +reasoning applies to Demucs and not here). +""" + +from __future__ import annotations + +import json +import logging +import os +import subprocess +import threading +import time +from pathlib import Path + +from app.core.models import Job +from app.core.registry import set_proc +from app.pipeline.errors import SeparationError + +logger = logging.getLogger("stemdeck.pipeline") + +_WATCHDOG_POLL_SECONDS = 30 +_TAIL_LIMIT = 40 + + +def _child_env() -> dict[str, str]: + env = os.environ.copy() + # Pin the child's stdio encoding to match what the parent decodes with. + # Without it a Windows child writes cp1252 while the parent reads utf-8, so + # the mismatch simply moves rather than being fixed. audio-separator emits + # progress bars and can echo track metadata, neither of which is + # guaranteed to be cp1252-safe. + env["PYTHONIOENCODING"] = "utf-8:replace" + # The worker arms a watchdog on this and hard-exits when we disappear, so a + # kill that runs no cleanup (SIGKILL, Force Quit, Task Manager, a crash) + # cannot leave it running with nobody to collect the result (#519). + env["STEMDECK_PARENT_PID"] = str(os.getpid()) + try: + import certifi + + env.setdefault("SSL_CERT_FILE", certifi.where()) + env.setdefault("REQUESTS_CA_BUNDLE", certifi.where()) + except ModuleNotFoundError: + pass + return env + + +def run_split( + job: Job, + *, + argv: list[str], + device: str, + timeout: int, + label: str, + produces: list[str], + out_dir: Path, +) -> list[str]: + """Run a split worker to completion and return `produces` on success. + + Raises SeparationError (carrying the stderr tail) if the worker fails, + stalls past `timeout` with no output, or exits without writing every file + in `produces`. Never mutates anything outside `out_dir`. + """ + proc = subprocess.Popen( + argv, + stdout=subprocess.DEVNULL, + stderr=subprocess.PIPE, + # utf-8/replace explicitly. text=True alone decodes with the locale + # encoding, which on Windows is cp1252, and a single byte outside it + # in a child's output kills the whole job with a UnicodeDecodeError + # ("'charmap' codec can't decode byte 0x8f"). This is diagnostic text: + # a byte we cannot read is never a reason to fail a separation. + text=True, + encoding="utf-8", + errors="replace", + bufsize=1, + env=_child_env(), + ) + set_proc(job.id, proc) + + tail: list[str] = [] + last_output = [time.monotonic()] + done_evt = threading.Event() + + def _watchdog() -> None: + while not done_evt.wait(timeout=_WATCHDOG_POLL_SECONDS): + if proc.poll() is not None: + return + if time.monotonic() - last_output[0] > timeout: + logger.warning( + "%s stalled for %ss with no output, terminating job %s", + label, + timeout, + job.id, + ) + proc.terminate() + return + + wt = threading.Thread(target=_watchdog, daemon=True) + wt.start() + job_ok = False + try: + assert proc.stderr is not None + for raw_line in proc.stderr: + last_output[0] = time.monotonic() + line = raw_line.strip() + if not line: + continue + if line == "@@DONE@@": + job_ok = True + break + if line.startswith("@@ERROR@@"): + msg = line[len("@@ERROR@@") :] + try: + msg = json.loads(msg) + except json.JSONDecodeError: + pass + tail.append(str(msg)) + break + tail.append(line) + if len(tail) > _TAIL_LIMIT: + tail.pop(0) + finally: + done_evt.set() + set_proc(job.id, None) + wt.join(timeout=2) + if proc.poll() is None: + proc.terminate() + try: + proc.wait(timeout=5) + except subprocess.TimeoutExpired: + proc.kill() + proc.communicate() + + if not job_ok: + detail = "\n".join(tail[-15:]) if tail else "(no stderr captured)" + logger.warning("[%s] %s failed: %s", job.id, label, detail) + last = tail[-1] if tail else f"{label} failed" + raise SeparationError(f"{label} failed: {last}", tail=tail[-_TAIL_LIMIT:], device=device) + + for name in produces: + if not (out_dir / f"{name}.wav").is_file(): + raise SeparationError(f"{label} did not produce {name}.wav", device=device) + return list(produces) diff --git a/app/pipeline/vocal_split.py b/app/pipeline/vocal_split.py index a3d1ebf8..c42032ff 100644 --- a/app/pipeline/vocal_split.py +++ b/app/pipeline/vocal_split.py @@ -8,29 +8,31 @@ occasional user-triggered action, not the hot path every job takes (the persistent-worker pattern in ml-pipeline.md exists specifically to amortize a cost every job pays; that reasoning doesn't apply here, #309). + +The subprocess lifecycle, stall watchdog and error hygiene live in +split_runner.py, shared with the duet split. """ from __future__ import annotations -import json import logging -import os -import subprocess import sys -import threading -import time from pathlib import Path from app.core.config import TIMEOUT_VOCAL_SPLIT from app.core.models import Job -from app.core.registry import set_proc from app.core.settings import get_demucs_device from app.pipeline.errors import SeparationError +from app.pipeline.split_runner import run_split logger = logging.getLogger("stemdeck.pipeline") +VOCAL_SPLIT_STEMS = ["lead_vocals", "backing_vocals"] + def _spawn_cmd(device: str, vocals_path: Path, out_dir: Path) -> list[str]: + """The worker argv. Kept as its own function so tests can swap in a stub + worker and still exercise the real subprocess machinery.""" return [ sys.executable, "-m", @@ -55,102 +57,12 @@ def split_vocals(job: Job, stems_dir: Path) -> list[str]: raise SeparationError("vocals.wav not found -- job has no vocals stem to split") device = get_demucs_device() - env = os.environ.copy() - # Pin the child's stdio encoding to match what the parent now decodes with. - # Without it a Windows child writes cp1252 while the parent reads utf-8, so - # the mismatch simply moves rather than being fixed. Demucs and audio- - # separator both emit progress bars and can echo track metadata, neither of - # which is guaranteed to be cp1252-safe. - # The worker arms a watchdog on this and hard-exits when we disappear, so a - # kill that runs no cleanup (SIGKILL, Force Quit, Task Manager, a crash) - # cannot leave it running with nobody to collect the result (#519). - env["STEMDECK_PARENT_PID"] = str(os.getpid()) - env["PYTHONIOENCODING"] = "utf-8:replace" - try: - import certifi - - env.setdefault("SSL_CERT_FILE", certifi.where()) - env.setdefault("REQUESTS_CA_BUNDLE", certifi.where()) - except ModuleNotFoundError: - pass - - proc = subprocess.Popen( - _spawn_cmd(device, vocals_path, stems_dir), - stdout=subprocess.DEVNULL, - stderr=subprocess.PIPE, - # utf-8/replace explicitly. text=True alone decodes with the locale - # encoding, which on Windows is cp1252, and a single byte outside it - # in a child's output kills the whole job with a UnicodeDecodeError - # ("'charmap' codec can't decode byte 0x8f"). This is diagnostic text: - # a byte we cannot read is never a reason to fail a separation. - text=True, - encoding="utf-8", - errors="replace", - bufsize=1, - env=env, + return run_split( + job, + argv=_spawn_cmd(device, vocals_path, stems_dir), + device=device, + timeout=TIMEOUT_VOCAL_SPLIT, + label="vocal split", + produces=VOCAL_SPLIT_STEMS, + out_dir=stems_dir, ) - set_proc(job.id, proc) - - tail: list[str] = [] - last_output = [time.monotonic()] - done_evt = threading.Event() - - def _watchdog() -> None: - while not done_evt.wait(timeout=30): - if proc.poll() is not None: - return - if time.monotonic() - last_output[0] > TIMEOUT_VOCAL_SPLIT: - logger.warning( - "vocal split stalled for %ss with no output, terminating job %s", - TIMEOUT_VOCAL_SPLIT, - job.id, - ) - proc.terminate() - return - - wt = threading.Thread(target=_watchdog, daemon=True) - wt.start() - job_ok = False - try: - assert proc.stderr is not None - for raw_line in proc.stderr: - last_output[0] = time.monotonic() - line = raw_line.strip() - if not line: - continue - if line == "@@DONE@@": - job_ok = True - break - if line.startswith("@@ERROR@@"): - msg = line[len("@@ERROR@@") :] - try: - msg = json.loads(msg) - except json.JSONDecodeError: - pass - tail.append(str(msg)) - break - tail.append(line) - if len(tail) > 40: - tail.pop(0) - finally: - done_evt.set() - set_proc(job.id, None) - wt.join(timeout=2) - if proc.poll() is None: - proc.terminate() - try: - proc.wait(timeout=5) - except subprocess.TimeoutExpired: - proc.kill() - proc.communicate() - - if not job_ok: - detail = "\n".join(tail[-15:]) if tail else "(no stderr captured)" - logger.warning("[%s] vocal split failed: %s", job.id, detail) - last = tail[-1] if tail else "vocal split failed" - raise SeparationError(f"vocal split failed: {last}", tail=tail[-40:], device=device) - - for name in ("lead_vocals", "backing_vocals"): - if not (stems_dir / f"{name}.wav").is_file(): - raise SeparationError(f"vocal split did not produce {name}.wav", device=device) - return ["lead_vocals", "backing_vocals"] diff --git a/docs/duet-training.md b/docs/duet-training.md new file mode 100644 index 00000000..86ded67f --- /dev/null +++ b/docs/duet-training.md @@ -0,0 +1,225 @@ +# Training a singer-conditioned duet separator + +A plan, not a record. Everything here is untried; the measured results it builds +on are in `models.md`, and the eval harness it depends on already exists. + +The goal is the thing the shipped feature does not do: tell you *which* singer is +in which stem, reliably, for a whole song. + +## Why conditioning, rather than a better separator + +The shipped model separates blindly and leaves identity to be worked out +afterwards. That second step is where it fails, measured three ways: + +- output order is arbitrary per chunk -- **11 of 25 chunks** had the singers + swapped on a real song +- re-assigning them afterwards with ECAPA moved speaker distance from **0.977 to + 0.973**, which is nothing +- because separated audio is degraded enough that a speaker embedder cannot read + it reliably, which is the same reason the first two are true + +A conditioned model has no identification step. Give it a recording of a singer, +get that singer back. Stem 1 is whoever was enrolled, for the entire song, by +construction. There is no permutation to fix and nothing to infer. + +[IWAENC 2026](https://arxiv.org/abs/2608.14516) demonstrates the idea, taking +target-singer SI-SDR from 0.33 to 5.58 dB. What matters is how weak their setup +is: an Open-Unmix backbone and a **two-layer GRU with hidden size 32** as the +singer encoder. They reached most of what this project measured (+7.3 dB) from a +far worse starting point, which says the conditioning is carrying the result and +nobody has yet put it on a strong backbone. + +Their checkpoints cannot be used: they are trained on DAMP-VSEP, which is +research-only and reserves derivatives to Smule. + +## The licensing position, which is what makes this possible + +StemDeck is free, open source and never monetised. That is not incidental here, +it decides the corpus. + +**NonCommercial is satisfied.** Every CC BY-NC corpus below is usable. + +**ShareAlike is satisfied by publishing the weights the same way.** Trained +weights are data, not code. StemDeck already keeps them out of the repository and +fetches them at first use, which is how Demucs and the models in `models.md` +work. Publishing a checkpoint under CC BY-NC-SA next to an Apache-2.0 codebase +changes nothing about the licence of the code. + +This is the difference between 20 singers and 200. Singer count is what governs +whether a separator generalises, and it is why the VocalSet-only attempt +recorded in `models.md` overfit. + +**Research-only stays out.** DAMP-VSEP reserves derivatives to Smule, and no +amount of being open source changes that. It is the one corpus everyone else +trains on, and skipping it is the cost of being able to ship the result. + +| corpus | singers | licence | note | +|---|---|---|---| +| OpenSinger | 93 | CC BY-NC-SA 2.0 | pop, 24 kHz | +| JVS-MuSiC | 100 | free, check terms before use | studio, 24 kHz | +| M4Singer | 20 | CC BY-NC-SA 4.0 | **labelled SATB**, 700 songs | +| VocalSet | 20 | CC BY 4.0 | exercises, not songs | +| vocadito | ~40 | CC BY 4.0 | short, real singing | +| CSD | 1 | CC BY-NC-SA 4.0 | two keys per song | +| GTSinger | 20 languages | CC BY-NC-SA 4.0 | 2024, multi-technique | + +M4Singer earns its place despite being small: it is **labelled by voice type**, +so same-register pairs can be drawn deliberately rather than hoped for. Everywhere +else, register has to be estimated from F0. + +## Architecture + +Keep the shipped separator and add conditioning to it. + +- **Backbone**: the Conv-TasNet in `app/pipeline/_convtasnet.py`, already loading + the CC-BY-4.0 checkpoint as a starting point. A stronger backbone is the + obvious follow-up, but changing two things at once makes the result + uninterpretable. +- **Singer encoder**: ECAPA, which this project already runs and which is + Apache-2.0. It is the single largest difference from the paper above. +- **Injection**: FiLM on the masker's bottleneck -- `gamma(z) * h + beta(z)`, + with `gamma` and `beta` small MLPs over the 192-dimensional embedding. FiLM + over concatenation because the paper found little between them and FiLM leaves + the backbone's shapes untouched. +- **Output**: one stem, the enrolled singer. The second stem is the mixture minus + the first, which keeps mixture consistency for free and halves what the model + has to learn. + +## Data + +Mixtures are built, not found. Two solo recordings from **different singers of +the same register**, summed. Ground truth is exact because the sum was +constructed. + +- **Pairing**: same voice type. Use M4Singer's labels where they exist, median F0 + elsewhere. Cross-register pairs are what the register model already handles, so + they are worth at most a small fraction of the batch. +- **Enrolment**: a separate passage by the same singer, never overlapping the one + in the mixture. The model must key on the voice, not on the passage. +- **Levels**: random, +/- 5 dB. Neither singer should be recoverable by being + the loud one. + +### Reverb is not the problem, and measuring it properly is why + +The first attempt used exponentially-decaying noise as a room. It raised the +held-out *produced* score from +6.3 to +10.0 dB while real music fell from +6.9 +to +4.9, and the obvious reading was that the model had learned a fake room and +that better rooms were the fix. + +That reading was wrong, and rebuilding the eval set with **real measured rooms** +(OpenSLR 28, Apache-2.0: RWCP, REVERB, Aachen) showed why: + +| condition | synthetic rooms | real rooms | +|---|---|---| +| VocalSet held-out, produced | +6.55 dB | **+10.31 dB** | + +The synthetic reverb was far harsher than any real room. The "production gap" it +appeared to reveal was an artefact of the augmentation, and against real rooms +the model gives up only 1.5 dB against dry. **Reverb robustness is close to a +non-problem, and training harder on it buys almost nothing.** + +Use real IRs anyway, since they are free and Apache-2.0, as a wet/dry send rather +than convolving outright -- those IRs are 16 kHz and convolving the signal would +teach the model that duets have no top end. But do not expect a result from it. + +### Measured: training on VocalSet makes real music worse + +Run to completion with real rooms, 8,000 steps, four singers held out: + +| step | produced | real music | +|---|---|---| +| 0 | +8.69 | **+6.93** | +| 1500 | +9.89 | +6.06 | +| 4000 | +9.42 | +5.52 | +| 8000 | +9.37 | **+5.52** | + +Real music falls **1.4 dB and flatlines**, never recovering across the whole +run, while the in-domain score drifts up by less than a decibel. Real rooms +halved the damage against the synthetic-reverb attempt but did not change its +direction. + +This is the clean version of the experiment: one variable, a held-out eval, and +an out-of-domain check. **VocalSet cannot close the gap and actively hurts.** The +stock checkpoint remains the one to ship, and the next attempt needs sung songs +rather than better augmentation of exercises. + +### The gap that is real: exercises against songs + +What the corrected numbers leave is this: + +| | | +|---|---| +| VocalSet in a real room | +10.31 dB | +| actual produced music | +6.93 dB | + +**3.4 dB, and none of it acoustic.** VocalSet is scales, arpeggios and sustained +vowels. Real material is sung phrases with consonants, phrasing, vibrato that +means something, doubling, compression, and two people reacting to each other. + +That is a training-data problem, not an augmentation one, and it is the reason +the corpus table above matters more than anything else on this page. M4Singer is +700 songs; OpenSinger is 50 hours of pop from 93 singers. Both are sung music +rather than exercises, and both are reachable only because NonCommercial is +acceptable here. + +## Evaluation + +The harness exists and is the reason the failed run was caught in five minutes +rather than eight hours. Rebuild it from `models.md` if it has been lost: + +- **60 mixtures per condition**, four singers held out of training entirely +- **two conditions**, dry and produced, because the gap between them is the + finding +- **real out-of-domain audio**, produced music the model never saw +- **95% confidence intervals**, so a sub-dB change is readable + +Baselines to beat, measured on the shipped model: + +| | | +|---|---| +| held-out dry | +11.78 +/- 1.38 dB | +| held-out produced, real rooms | +10.31 +/- 1.57 dB | +| real produced music | +6.93 +/- 1.60 dB | +| masking ceiling | +13.3 dB | + +The produced figure is against real measured rooms. The +6.55 dB recorded +earlier was against synthetic reverb and says more about that reverb than about +the model. + +For a conditioned model, add the measure the whole exercise is for: **does stem 1 +hold the enrolled singer for the whole song**. Speaker distance between the +output and the enrolment, per 30-second window, across a full track. The shipped +model scores 0.977 there, where the true sources score 0.153. + +## Cost + +Measured on an RTX 3080, 10 GB, training only the masker (20.7M parameters): + +| batch | ms/step | steps/hour | VRAM | +|---|---|---|---| +| 16 | 130 | 27,800 | 2.9 GB | +| 32 | 206 | 17,500 | 5.5 GB | +| 58 | 14,200 | 254 | 9.5 GB, thrashing | + +Batch 16 is the working point; 58, the original recipe's batch size, does not fit +in 10 GB. A first run of 30-60k steps is one to two hours. A serious one is days, +not weeks. + +Data preparation is the part that will surprise you. The first attempt ran at 3% +GPU because reverb used `np.convolve`: **2.6 s per step against a 130 ms step**. +FFT convolution and a precomputed bank of rooms fixed it. + +## What would say this worked + +1. Real-world produced audio improves, **not just the produced eval set**. The + first attempt failed exactly here. +2. Enrolment identity holds across a whole song: well under 0.5 speaker distance, + against 0.977 today. +3. It degrades honestly on a chorus rather than returning two confident stems + that both contain everyone. + +## What this does not fix + +Ensembles. Extracting one enrolled singer from twelve is a harder problem than +the two-source case, and nothing measured here suggests it comes for free. Expect +to need more than two sources at the output and a great deal more data. diff --git a/docs/models.md b/docs/models.md index 317e6d27..f568bce7 100644 --- a/docs/models.md +++ b/docs/models.md @@ -65,3 +65,199 @@ not intend it for free redistribution at all, and its already-dead HF mirror makes it an unreliable thing to depend on regardless of the licensing question. `STEMDECK_KARAOKE_MODEL` remains available as an env override for a deployment that wants to accept that risk itself. + +## BS-Roformer Chorus Male-Female (on-demand duet split) + +- **File**: `model_chorus_bs_roformer_ep_267_sdr_24.1275.ckpt` plus its + `config_chorus_male_female_bs_roformer.yaml`. +- **Distributed via**: `audio-separator`, the same MIT + credit-to-UVR channel + as the karaoke model above. Community-trained (credited to Sucial); no + separate LICENSE file of its own was found, which puts it in the same + "unlicensed but freely and openly distributed" bucket the karaoke default + sits in -- not the Boosty-paywall bucket that was rejected. +- **Credit**: Ultimate Vocal Remover (Anjok07) -- https://github.com/Anjok07/ultimatevocalremovergui + +`DUET_SPLIT_MODEL` in `app/core/config.py`, overridable via +`STEMDECK_DUET_MODEL`. + +### What it actually separates, measured + +The model keys on **vocal weight/register**, not on singer identity. Measured +on the job's own vocals stem: + +| material | result | +|---|---| +| "Shallow" (Cooper/Gaga, different registers) | clean split: 30-40 dB rejection of the idle stem through each singer's verse, per-frame energy share std 0.41 (real alternation), sum-to-mix residual 0.028 | +| "What Is This Feeling" (Wicked, two sopranos) | no split: leads with one stem in 76% of frames and the other in 7%, which is one voice with gaps. An earlier run of the same material recorded 93/7 by energy; a fresh run gave 24/76, so the totals move and who leads is the stable signal | + +So this is a duet splitter for singers in different registers, and does +nothing for same-register duets. The stem names (`voice_1` / `voice_2`) +say that rather than promising per-singer separation. + +### Same-register duets (two sopranos): what was tried + +Splitting two singers of the *same* register -- the "Glinda and Elphaba" case -- +is the thing the register model above cannot do. What follows is the record of +what failed, kept so the dead ends are not re-attempted, and then the model that +works, which is MedleyVox Conv-TasNet in the next section. + +**What does not work.** Measured against an ideal-ratio-mask ceiling of +**+13.4 dB SI-SDRi**, so there is real headroom. + +| approach | result | +|---|---| +| SepFormer (speech PIT separation) | two anti-correlated outputs whose sum does not reconstruct the input; out of domain on singing | +| supervised NMF (bases per singer from their own solo passages) | -2.3 dB on held-out audio: with enough components the two dictionaries explain each other | +| harmonic-comb masking from estimated multi-F0 | -2.7 dB. With *oracle* F0s it reaches **+8.2 dB**, so the idea is sound and estimation is what fails: multi-F0 recall 0.44-0.54, contour-to-singer assignment 0.57 (chance 0.5) | +| unsupervised 2-way diarization of the vocals stem | fails on any song with a chorus: the dominant split is leads-vs-ensemble, not lead-vs-lead | +| ECAPA enrolment (embed each singer from a solo passage, route windows) | routes lines well on alternating material but does not separate simultaneous singing, and did not improve a separated result: speaker distance 0.977 to 0.973. Removed. | +| zero-shot diffusion prior (`iamycy/duet-svs-diffusion`, MIT) | **-0.91 dB SI-SDRi**, worse than not splitting, at 60x slower than realtime. The paper's 11.75 dB uses oracle retry and ground-truth conditioning | + +**A correction.** An earlier version of this file recorded MedleyVox as tested +and failing -- "no split on any real duet; also splits a *solo* voice across +both outputs". That was a mistake in how it was run, twice over. iSRNet is a +*refinement* stage cascaded after a Conv-TasNet separator, not a separator, so +running it alone does nothing. And the checkpoint used was from the +`singing_librispeech` training, which separates singing from *speech*. The +`multi_singing_librispeech` checkpoint is the one for duets and had not been +downloaded. + +## MedleyVox Conv-TasNet (same-register duet split) + +`app/pipeline/duet_medleyvox.py`, with the model code vendored into +`app/pipeline/_convtasnet.py`. + +**What it does.** Separates two singers who share a vocal range, which the +register model above cannot do at all. Measured on ground-truth duets built from +real production audio, against a +13.3 dB ideal-mask ceiling on the same clips: + +| | result | +|---|---| +| other singer suppressed by | **14.5 dB** (no split is 0; the ideal mask reaches 24.5) | +| SI-SDRi | **+7.3 dB**, about 55% of the masking ceiling | +| speed | 5-7x realtime on CPU, faster on GPU | +| output | full input rate and channel count, stems sum back to the recording | + +Inference details that are worth their cost, each measured: two checkpoints of +the same run ensembled (+0.1 to +0.2 dB), each also run on the reversed signal +(+0.2 dB), single-pass rather than overlap-add (+0.6 dB), and masks applied to +the original-rate spectrogram rather than resampling the output (keeps +everything above 12 kHz, which the model itself cannot see). + +Things tried that did not help: shift augmentation (-0.1 dB), multi-resolution +mask averaging (+0.04 dB), and fine-tuning on the song's own solo passages, +which overfits within 10 steps. + +**Training on VocalSet made it worse.** Fine-tuning on same-register pairs from +VocalSet (CC BY 4.0) with synthetic reverb raised the held-out *produced* score +from +6.3 to +10.0 dB while the real-world test set fell from +6.9 to +4.9. The +model learned the synthetic reverb rather than production. Real impulse +responses are the open lead here. + +**Choosing between the two models cannot be automated.** Five blind measures +were tried and every one scored the register model's failure as healthy; the +detail is in `duet_medleyvox.register_split_collapsed`. The short version is +that a wrong separation is confident and uncorrelated, exactly like a right one: +on the failed split, stem correlation scored 0.09 against the correct output's +0.17-0.29, and mask decisiveness scored 0.96, higher than anything else +measured. Only the unambiguous case -- one stem essentially silent -- is +detected automatically. `STEMDECK_DUET_SAME_REGISTER=1` forces this model for +anyone who has listened and knows which they want. + +**Limits.** The checkpoints have two outputs, so an ensemble is beyond them: +stems from two-singer passages sit at 0.32 speaker distance, and across a full +musical number with a chorus they smear to 0.97. There is no reliable way to +detect that from the audio in advance. + +### It does not work on real duets, and the eval that said otherwise was flawed + +Tested on "Shallow" through the running app, then checked by ear: both singers +audible in the same stem. That report was right and the measurements agreeing +with it were not. + +Speaker identity, which is the only measure that has ever caught a wrong split +here: + +| | cross-stem cosine | +|---|---| +| clustering the **unsplit** vocals, confident windows | **0.08** | +| clustering the unsplit vocals, all windows | 0.567 | +| register model's stems | **0.917** | +| same-register model's stems | 0.958 | + +**Naively clustering the raw vocals separates the two singers better than either +model does.** Split by section it is no kinder: 0.952 through the solo verses, +0.883 through the overlaps. It fails everywhere, not just where they sing at +once. + +**Why the +7.3 dB and 14.5 dB figures did not predict this.** Those ground-truth +mixtures were built by summing two *unrelated* passages -- one singer from one +part of the song, the other from somewhere else. That is a much easier problem +than a duet. Real duet voices are musically correlated: harmonised pitch, +synchronised consonants, the same vowels at the same time, and a shared reverb +and compression bus. The model separates two arbitrary voices that happen to +overlap and does not separate two people singing together. + +So the SI-SDR numbers above are real but measure the wrong task. An honest eval +needs genuine multitrack duets, where each singer's recorded track is available +separately, rather than constructed sums. + +**Do not ship this as a duet splitter.** The register model's 30-40 dB figure +recorded earlier came from rejection measured through single-singer verses, +which is not the same claim as separating the two people. + +### Routing beats separating, measured + +Prompted by the obvious question after the failure above: if the singers can be +told apart well enough to *decide* who is singing, why separate at all? Cut the +vocal into windows, decide whose it is, and route it. Where both sing, say so. + +On "Shallow", cross-stem speaker cosine, lower being further apart: + +| approach | coverage | cosine | +|---|---|---| +| same-register model | 100% | 0.958 | +| register model | 100% | 0.917 | +| **routing, overlap kept in both stems** | 100% | **0.756** | +| routing, overlap silenced | 73% | 0.626 | +| routing, conservative | 51% | 0.467 | +| routing, very conservative | 33% | **0.351** | + +**Every routing setting beats both separators**, and purity is a dial: the more +of the song you are willing to leave out, the cleaner what remains. It is also +far cheaper -- an ECAPA pass over the vocal, no 460 MB checkpoint, seconds +rather than nine minutes on CPU. + +The reason is not subtle in hindsight. A separator has to *reconstruct* two +signals and destroys identity doing it. Routing never touches the audio; it only +decides where each second goes, so whatever identity was in the mixture survives +intact. + +**What it still does not do**, and cannot: two people singing at once are in the +same samples and no routing extracts them. Routing's advantage is that it can be +honest about that, which the separators are not -- they return two confident +stems and nothing signals that both contain everyone. + +**One caveat on these numbers.** At high confidence the two clusters come out +lopsided, 2% against 31%, which suggests the clustering finds one distinctive +voice and a remainder rather than a balanced two-singer partition. The 0.08 +"floor" quoted earlier has the same asymmetry, 92 windows against 18, so it +flatters the ceiling. Treat the ordering above as solid and the absolute values +as optimistic. + +### Licensing + +- **MedleyVox weights** (`Cyru5/MedleyVox`) are **CC-BY 4.0**, trained by Carson + Evans. Fetched at first use rather than bundled, the same pattern as Demucs. +- **The MedleyVox inference code** (`jeonchangbin49/MedleyVox`) carries no + LICENSE and is **not used**. The model is four classes from `asteroid` (MIT) + plus one published formula, vendored into `app/pipeline/_convtasnet.py` with + attribution. Depending on `asteroid` directly would cost 23 transitive + packages including pytorch-lightning and pandas. +- **`asteroid-filterbanks`** (MIT) is a real dependency and pulls in nothing of + its own. +- **SepFormer** and the ECAPA speaker embedder (`speechbrain/*`) are Apache-2.0; + neither is used any more. +- **DAMP-VSEP**, behind the singer-conditioned models in the literature, is + research-only and reserves derivatives to Smule. Those checkpoints are + unusable here regardless of quality. diff --git a/pyproject.toml b/pyproject.toml index ad60d5e6..161bbfcb 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -88,6 +88,12 @@ dependencies = [ # today, but pin it explicitly so this survives an eventual librosa 1.x move # (#407) -- same platform gate as audio-separator itself. "audioread>=3.0; sys_platform != 'darwin' or platform_machine != 'x86_64'", + # STFT encoder/decoder for the same-register duet split + # (app/pipeline/duet_medleyvox.py). MIT, and unlike the full `asteroid` + # package it has no dependencies of its own -- that one costs 23 transitive + # packages including pytorch-lightning and pandas, so the handful of model + # classes needed from it are vendored in app/pipeline/_convtasnet.py. + "asteroid-filterbanks>=0.4", ] [project.optional-dependencies] @@ -124,7 +130,18 @@ line-length = 100 # app/_vendor is a third-party payload copied verbatim from PyPI. Linting or # reformatting it would create a diff against upstream that the refresh script # would then fight every time it runs. -exclude = ["jobs", ".run", "static/vendor", "app/_version.py", "app/_vendor"] +# app/pipeline/_convtasnet.py is the same deal: model classes copied from +# asteroid (MIT) so the published checkpoint keeps loading. Modernising the +# annotations there would be harmless today and a silent trap the next time it +# is regenerated. +exclude = [ + "jobs", + ".run", + "static/vendor", + "app/_version.py", + "app/_vendor", + "app/pipeline/_convtasnet.py", +] [tool.ruff.lint] select = ["E", "F", "W", "I", "UP", "B", "SIM"] diff --git a/static/css/daw.css b/static/css/daw.css index 938979bb..77d3223a 100644 --- a/static/css/daw.css +++ b/static/css/daw.css @@ -107,7 +107,10 @@ input, textarea { font-family: inherit; } gap: 10px; padding: 0 14px; flex: 1; - min-width: 0; + /* A floor, not 0: below about 140px the input holds no visible text and the + drop-refusal message renders at zero width, which reads as no answer at + all. */ + min-width: 168px; cursor: text; } /* Hide decorative SVG icons in the topbar */ @@ -216,13 +219,34 @@ input, textarea { font-family: inherit; } } /* Stem section */ +/* Shrinks, and the URL zone does not below its floor. The composer is one + fixed-height row with a hard width budget, and everything the controls ask + for used to come out of the URL zone, which is the only flexible item in the + row. A third vocal mode button took the zone to nothing but its own padding + at 1280px: no room for the input, and none for the refusal message #584 puts + in the same place, so dropping an unsupported file explained itself to a + zero-width box. The stem chips give the space up instead, by ellipsising + their labels next to a colour dot that already identifies them. */ .daw-stem-section { display: flex; align-items: center; gap: 10px; padding: 0 14px; - flex-shrink: 0; + flex-shrink: 1; + min-width: 0; } +.daw-stem-chips { min-width: 0; } +.stem-choice { min-width: 0; } +/* The dot keeps its size; only the word gives way. */ +.stem-choice > span:not(.stem-dot) { + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} +/* Exempt: its buttons are a mode each, and truncating "Lead + Backing" to + "Lead" would name a different thing. The experimental ribbon also overhangs + the last button, so this must not become a clipping context. */ +.vocal-mode-toggle { flex-shrink: 0; } .daw-extract-label { font-size: 10px; font-weight: 600; @@ -303,13 +327,15 @@ input, textarea { font-family: inherit; } color: var(--accent); } -/* Vocals: All / Lead + Backing (on-demand split, #275) */ +/* Vocals: All / Lead + Backing / Duets (on-demand splits, #275) */ .vocal-mode-toggle { display: inline-flex; height: 30px; border: 1px solid var(--border); border-radius: 6px; - overflow: hidden; + /* Not overflow:hidden. The experimental ribbon on the last button has to + reach outside this box to fit its word, so the rounded ends are clipped by + the buttons themselves below. */ } .vocal-mode-toggle.hidden { display: none; } .vocal-mode-btn { @@ -325,10 +351,67 @@ input, textarea { font-family: inherit; } transition: background var(--t-fast), color var(--t-fast); } .vocal-mode-btn + .vocal-mode-btn { border-left: 1px solid var(--border); } +.vocal-mode-btn:first-child { border-radius: 5px 0 0 5px; } +.vocal-mode-btn:last-child { border-radius: 0 5px 5px 0; } .vocal-mode-btn[aria-pressed="true"] { background: color-mix(in srgb, var(--vocals) 16%, transparent); color: var(--vocals); } +/* Diagonal tag marking a mode as experimental. + + The text comes from data-badge, which applyTranslations fills from + data-i18n-badge. It cannot be a child element: the button carries data-i18n, + and that assigns textContent, so any child inside it is destroyed on every + language change. + + Fitting the word is the whole difficulty. The control is 30px tall, and three + layouts were tried against it: a corner ribbon has a 42px chord at 45 degrees + and rendered "RIMEN"; running the word along the button's diagonal fitted it + but laid it over the label so neither could be read; letting a 45-degree + ribbon escape the control put it across the neighbouring stem button. + + What works is a shallow angle. At 9 degrees the word needs almost no vertical + room, so it sits on the button's top edge, inside the button's own width, in + the spare height .daw-composer has above the toggle. */ +.vocal-mode-btn[data-badge] { + position: relative; +} +.vocal-mode-btn[data-badge]::after { + content: attr(data-badge); + position: absolute; + top: -7px; + left: 50%; + transform: translateX(-50%) rotate(-9deg); + padding: 1px 4px; + border-radius: 3px; + background: color-mix(in srgb, var(--vocals) 80%, transparent); + color: #0d1016; + font-size: 6px; + font-weight: 800; + line-height: 1.5; + letter-spacing: 0.02em; + text-transform: uppercase; + white-space: nowrap; + /* Sits over the button, so it must never swallow the click. */ + pointer-events: none; +} +.vocal-mode-btn[data-badge][aria-pressed="true"]::after, +.vocal-mode-btn[data-badge]:hover::after, +.vocal-mode-btn[data-badge]:focus-visible::after { + background: var(--vocals); +} +/* Pointing at it lifts the ribbon slightly, so the warning registers before the + tooltip has appeared. Suppressed where motion is unwelcome. */ +.vocal-mode-btn[data-badge]:hover::after { + transform: translateX(-50%) rotate(-9deg) translateY(-1px) scale(1.06); + transition: transform var(--t-fast), background var(--t-fast); +} +@media (prefers-reduced-motion: reduce) { + .vocal-mode-btn[data-badge]:hover::after { + transform: translateX(-50%) rotate(-9deg); + transition: background var(--t-fast); + } +} /* Process button */ .daw-process-btn { diff --git a/static/index.html b/static/index.html index f432726d..c126cacb 100644 --- a/static/index.html +++ b/static/index.html @@ -83,12 +83,18 @@ - + diff --git a/static/js/catalog.js b/static/js/catalog.js index 889643b6..f890051a 100644 --- a/static/js/catalog.js +++ b/static/js/catalog.js @@ -721,7 +721,14 @@ async function loadTrackIntoStudio(trackId) { // Start peaks fetch immediately — runs in parallel with job-data fetch so it // resolves before wireUpAudio calls Multitrack.create. This prevents peaks.json // from competing with stem WAV fetches for Safari's 6-connection-per-origin limit. - const peaksPromise = fetch(`/api/jobs/${trackId}/stems/peaks.json`) + // The cache buster is not belt-and-braces. This file used to be served + // `immutable` for a year, which a browser takes at its word and will not + // revalidate even on a hard reload. The header is fixed, but any browser that + // cached it under the old one keeps serving that copy until the year is up, + // and an on-demand split rewrites this file: the new stems play under the old + // waveform with no way for the user to refresh out of it. A changing URL is + // the only thing that reaches an entry already poisoned. + const peaksPromise = fetch(`/api/jobs/${trackId}/stems/peaks.json?v=${Date.now()}`) .then((r) => (r.ok ? r.json() : {})) .catch(() => ({})); diff --git a/static/js/constants.js b/static/js/constants.js index 5de23c60..a9b8a311 100644 --- a/static/js/constants.js +++ b/static/js/constants.js @@ -10,6 +10,11 @@ export let TRACK_NAMES = ["original", ...STEM_NAMES]; // never run this split -- but the player/mixer render them as real lanes, // swapped in for "vocals", via effectiveStemOrder() below. export let EXTRA_STEM_NAMES = ["lead_vocals", "backing_vocals"]; +// Stems from the on-demand duet split (DUET_STEM_NAMES in +// app/core/config.py). A separate pair, not more EXTRA_STEM_NAMES: the two +// splits are alternatives and effectiveStemOrder() swaps "vocals" for a pair +// only when every name in that pair is present. +export let DUET_STEM_NAMES = ["voice_1", "voice_2"]; export async function syncStemNamesFromAPI() { try { @@ -23,6 +28,9 @@ export async function syncStemNamesFromAPI() { if (Array.isArray(data.extra_stem_names) && data.extra_stem_names.length > 0) { EXTRA_STEM_NAMES = data.extra_stem_names; } + if (Array.isArray(data.duet_stem_names) && data.duet_stem_names.length > 0) { + DUET_STEM_NAMES = data.duet_stem_names; + } } catch (e) { console.warn("[constants] failed to sync stem names from API:", e); } @@ -34,8 +42,14 @@ export async function syncStemNamesFromAPI() { // has (typically from job.stems). Order matters -- callers use this both to // decide what to render and in what sequence (waveform stacking, mixer rows). export function effectiveStemOrder(presentNames) { - const splitDone = presentNames.has("lead_vocals") && presentNames.has("backing_vocals"); - return STEM_NAMES.flatMap((n) => (n === "vocals" && splitDone ? EXTRA_STEM_NAMES : [n])); + // Only one decomposition can be in play for a job, but check both: a job + // that ran the lead/backing split and was later re-run as a duet has all + // four on disk, and lead/backing keeps precedence so the lane order does + // not silently change under a user who never asked for the duet split. + const pair = [EXTRA_STEM_NAMES, DUET_STEM_NAMES].find((names) => + names.every((n) => presentNames.has(n)) + ); + return STEM_NAMES.flatMap((n) => (n === "vocals" && pair ? pair : [n])); } // A Proxy, not a plain object, so every lookup resolves through the CURRENT @@ -65,6 +79,8 @@ export const STEM_COLORS = { original: "#a8b0bd", lead_vocals: "#e8748a", backing_vocals: "#c98fe0", + voice_1: "#e8748a", + voice_2: "#7fc7e8", }; export const PROGRESS_COLOR = "#3a3a3a"; diff --git a/static/js/i18n.js b/static/js/i18n.js index cae25a97..ad271fad 100644 --- a/static/js/i18n.js +++ b/static/js/i18n.js @@ -189,6 +189,13 @@ export function applyTranslations(root) { scope.querySelectorAll("[data-i18n-placeholder]").forEach((el) => { el.placeholder = t(el.getAttribute("data-i18n-placeholder")); }); + // Text a pseudo-element draws, via content: attr(data-badge). An element + // carrying data-i18n has its textContent rewritten above, so a real child + // element inside one is wiped on every language change; this is how such an + // element gets a second translated string. + scope.querySelectorAll("[data-i18n-badge]").forEach((el) => { + el.dataset.badge = t(el.getAttribute("data-i18n-badge")); + }); const titleKey = document.documentElement.getAttribute("data-i18n-doctitle"); if (titleKey) document.title = t(titleKey); applyStemRowAriaLabels(scope); @@ -233,7 +240,7 @@ const en = { "sections.clearConfirm": "Confirm?", "structure.toggle": "Song structure", "structure.toggleTitle": "Experimental Song Structure Extraction: automatically label intro, verse and chorus after a split", - "structure.experimental": "Experimental", + "common.experimental": "EXPERIMENTAL", "doc.title": "StemDeck — split any track into stems", "topbar.urlPlaceholder": "Search or drop an audio file…", @@ -247,6 +254,8 @@ const en = { "stem.vocals": "Vocals", "stem.lead_vocals": "Lead Vocals", "stem.backing_vocals": "Backing Vocals", + "stem.voice_1": "Voice 1", + "stem.voice_2": "Voice 2", "stem.drums": "Drums", "stem.bass": "Bass", "stem.guitar": "Guitar", @@ -257,6 +266,8 @@ const en = { "vocalMode.groupAria": "Vocals mode", "vocalMode.all": "All", "vocalMode.split": "Lead + Backing", + "vocalMode.duet": "Duets", + "vocalMode.duetTitle": "EXPERIMENTAL. Splits the vocals into two singers. Works best when they trade lines and their voices differ. Where both sing at once, some of the other voice remains.", "aria.mute": "Mute {name}", "aria.solo": "Solo {name}", @@ -379,6 +390,8 @@ const en = { "wave.loadingAria": "Loading waveform", "job.cancelAria": "Cancel job", "job.cancel": "Cancel", + "job.splittingVocals": "Splitting lead/backing vocals…", + "job.splittingDuet": "Splitting duet voices…", "footer.stemsPlaceholder": "— Stems", "footer.stemsCount": "{count} stems", @@ -850,7 +863,7 @@ const pl = { "sections.clearConfirm": "Potwierdzić?", "structure.toggle": "Struktura utworu", "structure.toggleTitle": "Eksperymentalne wykrywanie struktury utworu: automatycznie oznacza intro, zwrotkę i refren po podziale", - "structure.experimental": "Eksperymentalne", + "common.experimental": "EKSPERYMENTALNE", "doc.title": "StemDeck — rozdziel dowolny utwór na ścieżki", "topbar.urlPlaceholder": "Szukaj albo upuść plik audio…", @@ -864,6 +877,8 @@ const pl = { "stem.vocals": "Wokal", "stem.lead_vocals": "Wokal główny", "stem.backing_vocals": "Chórki", + "stem.voice_1": "Głos 1", + "stem.voice_2": "Głos 2", "stem.drums": "Perkusja", "stem.bass": "Bas", "stem.guitar": "Gitara", @@ -874,6 +889,8 @@ const pl = { "vocalMode.groupAria": "Tryb wokalu", "vocalMode.all": "Wszystko", "vocalMode.split": "Wokal + Chórki", + "vocalMode.duet": "Duety", + "vocalMode.duetTitle": "EKSPERYMENTALNE. Dzieli wokal na dwoje śpiewaków. Działa najlepiej, gdy śpiewają na zmianę i mają różne głosy. Gdy śpiewają razem, słychać część drugiego głosu.", "aria.mute": "Wycisz: {name}", "aria.solo": "Solo: {name}", @@ -996,6 +1013,8 @@ const pl = { "wave.loadingAria": "Wczytywanie przebiegu fali", "job.cancelAria": "Anuluj zadanie", "job.cancel": "Anuluj", + "job.splittingVocals": "Dzielenie wokalu i chórków…", + "job.splittingDuet": "Dzielenie głosów duetu…", "footer.stemsPlaceholder": "— Ścieżek", "footer.stemsCount.one": "{count} ścieżka", @@ -1457,7 +1476,7 @@ const ja = { "sections.clearConfirm": "確認?", "structure.toggle": "曲の構成", "structure.toggleTitle": "実験的な楽曲構造抽出: 分離後にイントロ、Aメロ、サビを自動でラベル付けします", - "structure.experimental": "試験的", + "common.experimental": "試験的", "doc.title": "StemDeck — トラックをパートごとに分離", "topbar.urlPlaceholder": "検索するか、音声ファイルをドロップ…", @@ -1471,6 +1490,8 @@ const ja = { "stem.vocals": "ボーカル", "stem.lead_vocals": "リードボーカル", "stem.backing_vocals": "コーラス", + "stem.voice_1": "ボイス 1", + "stem.voice_2": "ボイス 2", "stem.drums": "ドラム", "stem.bass": "ベース", "stem.guitar": "ギター", @@ -1481,6 +1502,8 @@ const ja = { "vocalMode.groupAria": "ボーカルモード", "vocalMode.all": "すべて", "vocalMode.split": "リード+コーラス", + "vocalMode.duet": "デュエット", + "vocalMode.duetTitle": "実験的な機能です。ボーカルを2人の歌手に分割します。交互に歌い、声質が異なるときに最も有効です。同時に歌う部分では、もう一方の声が残ります。", "aria.mute": "{name}をミュート", "aria.solo": "{name}をソロ", @@ -1603,6 +1626,8 @@ const ja = { "wave.loadingAria": "波形を読み込み中", "job.cancelAria": "ジョブをキャンセル", "job.cancel": "キャンセル", + "job.splittingVocals": "リードとコーラスを分離中…", + "job.splittingDuet": "デュエットの声を分離中…", "footer.stemsPlaceholder": "— パート", "footer.stemsCount.other": "{count}パート", @@ -2039,7 +2064,7 @@ const zhHans = { "sections.clearConfirm": "确认?", "structure.toggle": "歌曲结构", "structure.toggleTitle": "实验性歌曲结构提取: 分离后自动标记前奏、主歌和副歌", - "structure.experimental": "实验性", + "common.experimental": "实验性", "doc.title": "StemDeck — 将任意曲目分离为音轨", "topbar.urlPlaceholder": "搜索,或拖放音频文件…", @@ -2053,6 +2078,8 @@ const zhHans = { "stem.vocals": "人声", "stem.lead_vocals": "主唱", "stem.backing_vocals": "和声", + "stem.voice_1": "声部 1", + "stem.voice_2": "声部 2", "stem.drums": "鼓组", "stem.bass": "贝斯", "stem.guitar": "吉他", @@ -2063,6 +2090,8 @@ const zhHans = { "vocalMode.groupAria": "人声模式", "vocalMode.all": "全部", "vocalMode.split": "主唱+和声", + "vocalMode.duet": "二重唱", + "vocalMode.duetTitle": "实验性功能。将人声分成两位歌手。当他们轮流演唱且音色不同时效果最佳。两人同时演唱时,仍会听到另一个声音。", "aria.mute": "静音{name}", "aria.solo": "独奏{name}", @@ -2185,6 +2214,8 @@ const zhHans = { "wave.loadingAria": "正在加载波形", "job.cancelAria": "取消任务", "job.cancel": "取消", + "job.splittingVocals": "正在分离主唱与和声…", + "job.splittingDuet": "正在分离二重唱声部…", "footer.stemsPlaceholder": "— 音轨", "footer.stemsCount.other": "{count} 条音轨", @@ -2621,7 +2652,7 @@ const de = { "sections.clearConfirm": "Sicher?", "structure.toggle": "Songstruktur", "structure.toggleTitle": "Experimentelle Songstruktur-Erkennung: beschriftet Intro, Strophe und Refrain nach dem Trennen automatisch", - "structure.experimental": "Experimentell", + "common.experimental": "EXPERIMENTELL", "doc.title": "StemDeck — jeden Track in Stems zerlegen", "topbar.urlPlaceholder": "Suchen oder eine Audiodatei ablegen…", @@ -2635,6 +2666,8 @@ const de = { "stem.vocals": "Gesang", "stem.lead_vocals": "Lead-Gesang", "stem.backing_vocals": "Backing-Gesang", + "stem.voice_1": "Stimme 1", + "stem.voice_2": "Stimme 2", "stem.drums": "Schlagzeug", "stem.bass": "Bass", "stem.guitar": "Gitarre", @@ -2645,6 +2678,8 @@ const de = { "vocalMode.groupAria": "Gesangsmodus", "vocalMode.all": "Alle", "vocalMode.split": "Lead + Backing", + "vocalMode.duet": "Duette", + "vocalMode.duetTitle": "EXPERIMENTELL. Teilt den Gesang in zwei Stimmen auf. Funktioniert am besten, wenn sie sich abwechseln und unterschiedlich klingen. Wo beide gleichzeitig singen, bleibt etwas der anderen Stimme.", "aria.mute": "{name} stummschalten", "aria.solo": "{name} solo", @@ -2767,6 +2802,8 @@ const de = { "wave.loadingAria": "Wellenform wird geladen", "job.cancelAria": "Auftrag abbrechen", "job.cancel": "Abbrechen", + "job.splittingVocals": "Lead/Backing werden getrennt…", + "job.splittingDuet": "Duett-Stimmen werden getrennt…", "footer.stemsPlaceholder": "— Stems", "footer.stemsCount": "{count} Stems", @@ -3214,7 +3251,7 @@ const pt = { "sections.clearConfirm": "Confirmar?", "structure.toggle": "Estrutura da música", "structure.toggleTitle": "Extração experimental da estrutura da música: marca intro, verso e refrão automaticamente após a separação", - "structure.experimental": "Experimental", + "common.experimental": "EXPERIMENTAL", "doc.title": "StemDeck — separe qualquer faixa em stems", "topbar.urlPlaceholder": "Pesquise ou solte um arquivo de áudio…", @@ -3228,6 +3265,8 @@ const pt = { "stem.vocals": "Vocais", "stem.lead_vocals": "Vocal principal", "stem.backing_vocals": "Vocal de apoio", + "stem.voice_1": "Voz 1", + "stem.voice_2": "Voz 2", "stem.drums": "Bateria", "stem.bass": "Baixo", "stem.guitar": "Guitarra", @@ -3238,6 +3277,8 @@ const pt = { "vocalMode.groupAria": "Modo de vocais", "vocalMode.all": "Todos", "vocalMode.split": "Principal + Apoio", + "vocalMode.duet": "Duetos", + "vocalMode.duetTitle": "EXPERIMENTAL. Divide as vozes em dois cantores. Funciona melhor quando alternam e têm vozes diferentes. Onde cantam ao mesmo tempo, fica alguma da outra voz.", "aria.mute": "Mudo {name}", "aria.solo": "Solo {name}", @@ -3360,6 +3401,8 @@ const pt = { "wave.loadingAria": "Carregando forma de onda", "job.cancelAria": "Cancelar tarefa", "job.cancel": "Cancelar", + "job.splittingVocals": "A separar vocal principal e apoio…", + "job.splittingDuet": "A separar as vozes do dueto…", "footer.stemsPlaceholder": "— Stems", "footer.stemsCount": "{count} stems", @@ -3809,7 +3852,7 @@ const id = { "sections.clearConfirm": "Yakin?", "structure.toggle": "Struktur lagu", "structure.toggleTitle": "Ekstraksi struktur lagu eksperimental: menandai intro, bait, dan refrein secara otomatis setelah pemisahan", - "structure.experimental": "Eksperimental", + "common.experimental": "EKSPERIMENTAL", "doc.title": "StemDeck — pisahkan trek apa pun menjadi stem", "topbar.urlPlaceholder": "Cari atau seret file audio…", @@ -3823,6 +3866,8 @@ const id = { "stem.vocals": "Vokal", "stem.lead_vocals": "Vokal Utama", "stem.backing_vocals": "Vokal Latar", + "stem.voice_1": "Suara 1", + "stem.voice_2": "Suara 2", "stem.drums": "Drum", "stem.bass": "Bass", "stem.guitar": "Gitar", @@ -3833,6 +3878,8 @@ const id = { "vocalMode.groupAria": "Mode vokal", "vocalMode.all": "Semua", "vocalMode.split": "Utama + Latar", + "vocalMode.duet": "Duet", + "vocalMode.duetTitle": "EKSPERIMENTAL. Memisahkan vokal menjadi dua penyanyi. Paling baik bila mereka bergantian dan suaranya berbeda. Bila keduanya bernyanyi bersamaan, sebagian suara lain tetap terdengar.", "aria.mute": "Bisukan {name}", "aria.solo": "Solo {name}", @@ -3955,6 +4002,8 @@ const id = { "wave.loadingAria": "Memuat gelombang", "job.cancelAria": "Batalkan tugas", "job.cancel": "Batal", + "job.splittingVocals": "Memisahkan vokal utama/latar…", + "job.splittingDuet": "Memisahkan suara duet…", "footer.stemsPlaceholder": "— Stem", "footer.stemsCount": "{count} stem", @@ -4391,7 +4440,7 @@ const fr = { "sections.clearConfirm": "Confirmer ?", "structure.toggle": "Structure du morceau", "structure.toggleTitle": "Extraction expérimentale de la structure: étiquette automatiquement intro, couplet et refrain après la séparation", - "structure.experimental": "Expérimental", + "common.experimental": "EXPÉRIMENTAL", "doc.title": "StemDeck — séparez n'importe quel morceau en pistes", "topbar.urlPlaceholder": "Recherchez ou déposez un fichier audio…", @@ -4405,6 +4454,8 @@ const fr = { "stem.vocals": "Voix", "stem.lead_vocals": "Voix principale", "stem.backing_vocals": "Chœurs", + "stem.voice_1": "Voix 1", + "stem.voice_2": "Voix 2", "stem.drums": "Batterie", "stem.bass": "Basse", "stem.guitar": "Guitare", @@ -4415,6 +4466,8 @@ const fr = { "vocalMode.groupAria": "Mode voix", "vocalMode.all": "Tout", "vocalMode.split": "Principale + chœurs", + "vocalMode.duet": "Duos", + "vocalMode.duetTitle": "EXPÉRIMENTAL. Sépare le chant en deux interprètes. Fonctionne mieux quand ils alternent et que leurs voix diffèrent. Là où ils chantent ensemble, une partie de l'autre voix subsiste.", "aria.mute": "Couper {name}", "aria.solo": "Solo {name}", @@ -4537,6 +4590,8 @@ const fr = { "wave.loadingAria": "Chargement de la forme d'onde", "job.cancelAria": "Annuler la tâche", "job.cancel": "Annuler", + "job.splittingVocals": "Séparation voix principale/chœurs…", + "job.splittingDuet": "Séparation des voix du duo…", "footer.stemsPlaceholder": "— Pistes", "footer.stemsCount": "{count} pistes", @@ -5102,7 +5157,7 @@ const es = { "sections.clearConfirm": "¿Confirmar?", "structure.toggle": "Estructura de la canción", "structure.toggleTitle": "Extracción experimental de la estructura: etiqueta automáticamente intro, verso y estribillo tras la separación", - "structure.experimental": "Experimental", + "common.experimental": "EXPERIMENTAL", "doc.title": "StemDeck — separa cualquier pista en stems", "topbar.urlPlaceholder": "Busca o suelta un archivo de audio…", @@ -5116,6 +5171,8 @@ const es = { "stem.vocals": "Voces", "stem.lead_vocals": "Voz principal", "stem.backing_vocals": "Coros", + "stem.voice_1": "Voz 1", + "stem.voice_2": "Voz 2", "stem.drums": "Batería", "stem.bass": "Bajo", "stem.guitar": "Guitarra", @@ -5126,6 +5183,8 @@ const es = { "vocalMode.groupAria": "Modo de voces", "vocalMode.all": "Todo", "vocalMode.split": "Principal + coros", + "vocalMode.duet": "Dúos", + "vocalMode.duetTitle": "EXPERIMENTAL. Divide la voz en dos cantantes. Funciona mejor cuando se alternan y sus voces son distintas. Donde cantan a la vez, queda algo de la otra voz.", "aria.mute": "Mute {name}", "aria.solo": "Solo {name}", @@ -5248,6 +5307,8 @@ const es = { "wave.loadingAria": "Cargando forma de onda", "job.cancelAria": "Cancelar tarea", "job.cancel": "Cancelar", + "job.splittingVocals": "Separando voz principal y coros…", + "job.splittingDuet": "Separando las voces del dúo…", "footer.stemsPlaceholder": "— Stems", "footer.stemsCount": "{count} stems", @@ -5718,7 +5779,7 @@ const ko = { "sections.clearConfirm": "정말요?", "structure.toggle": "곡 구조", "structure.toggleTitle": "실험적 곡 구조 분석: 분리한 뒤 인트로, 벌스, 코러스를 자동으로 표시해요", - "structure.experimental": "실험적", + "common.experimental": "실험적", "doc.title": "StemDeck, 어떤 곡이든 스템으로 분리", "topbar.urlPlaceholder": "검색하거나 오디오 파일을 끌어다 놓으세요…", @@ -5732,6 +5793,8 @@ const ko = { "stem.vocals": "보컬", "stem.lead_vocals": "리드 보컬", "stem.backing_vocals": "백 보컬", + "stem.voice_1": "목소리 1", + "stem.voice_2": "목소리 2", "stem.drums": "드럼", "stem.bass": "베이스", "stem.guitar": "기타", @@ -5742,6 +5805,8 @@ const ko = { "vocalMode.groupAria": "보컬 모드", "vocalMode.all": "전체", "vocalMode.split": "리드 + 백", + "vocalMode.duet": "듀엣", + "vocalMode.duetTitle": "실험적 기능입니다. 보컬을 두 명의 가수로 나눕니다. 번갈아 부르고 목소리가 다를 때 가장 잘 작동합니다. 함께 부르는 부분에서는 다른 목소리가 일부 남습니다.", "aria.mute": "{name} 음소거", "aria.solo": "{name} 솔로", @@ -5864,6 +5929,8 @@ const ko = { "wave.loadingAria": "파형 불러오는 중", "job.cancelAria": "작업 취소", "job.cancel": "취소", + "job.splittingVocals": "리드/백 보컬 분리 중…", + "job.splittingDuet": "듀엣 목소리 분리 중…", "footer.stemsPlaceholder": "— 스템", "footer.stemsCount.other": "스템 {count}개", diff --git a/static/js/job.js b/static/js/job.js index 8ec73301..f337faf9 100644 --- a/static/js/job.js +++ b/static/js/job.js @@ -45,14 +45,21 @@ const splitAutoTriggered = new Set(); // than assuming their own `state` is still current. export async function runVocalSplitIfWanted(state) { if (state.status !== "done" || splitAutoTriggered.has(state.job_id)) return state; - const wantsSplit = jobVocalSplitModes.get(state.job_id) === "split"; + const mode = jobVocalSplitModes.get(state.job_id); + // The two decompositions are alternatives, so one table drives both: which + // endpoint to call, which job field says it already ran, and what to show + // while it runs. + const SPLITS = { + split: { path: "vocal-split", done: state.vocal_split, busy: t("job.splittingVocals") }, + duet: { path: "duet-split", done: state.duet_split, busy: t("job.splittingDuet") }, + }; + const split = SPLITS[mode]; const hasVocals = (state.stems || []).some((s) => s.name === "vocals"); - const alreadySplit = state.vocal_split === "done"; - if (!wantsSplit || !hasVocals || alreadySplit) return state; + if (!split || !hasVocals || split.done === "done") return state; splitAutoTriggered.add(state.job_id); - if (state.job_id === foregroundJobId) setWaveformLoading(true, "Splitting lead/backing vocals…"); + if (state.job_id === foregroundJobId) setWaveformLoading(true, split.busy); try { - const res = await fetch(`/api/jobs/${state.job_id}/vocal-split`, { method: "POST" }); + const res = await fetch(`/api/jobs/${state.job_id}/${split.path}`, { method: "POST" }); if (!(res.ok || res.status === 202)) return state; const r2 = await fetch(`/api/jobs/${state.job_id}`); if (!r2.ok) return state; diff --git a/static/js/mixer.js b/static/js/mixer.js index 10b84314..b159186a 100644 --- a/static/js/mixer.js +++ b/static/js/mixer.js @@ -1,16 +1,18 @@ import { - STEM_NAMES, TRACK_NAMES, EXTRA_STEM_NAMES, STEM_COLORS, STEM_DISPLAY, LANE_VOLUME_MAX, + STEM_NAMES, TRACK_NAMES, EXTRA_STEM_NAMES, DUET_STEM_NAMES, STEM_COLORS, STEM_DISPLAY, + LANE_VOLUME_MAX, } from "./constants.js"; // Every lane name this session might ever need mixer state for, including -// the on-demand lead/backing vocal split (#275). Safe to iterate broadly: +// both on-demand vocal splits (#275 lead/backing, and the duet pair). Safe +// to iterate broadly: // callers below all guard on the row/trackIndex entry actually existing, so // touching state for a lane a given job doesn't have is a no-op. A function // (not a snapshot) because STEM_NAMES/EXTRA_STEM_NAMES are reassigned once // syncStemNamesFromAPI() resolves -- a const array here would freeze the // fallback values from before that happens. function allTrackNames() { - return [...TRACK_NAMES, ...EXTRA_STEM_NAMES]; + return [...TRACK_NAMES, ...EXTRA_STEM_NAMES, ...DUET_STEM_NAMES]; } import { mixerState, mixerEl, stemListEl, currentJobId, multitrack, trackIndex, diff --git a/static/js/playbackStems.js b/static/js/playbackStems.js index a0342dee..29343481 100644 --- a/static/js/playbackStems.js +++ b/static/js/playbackStems.js @@ -5,6 +5,8 @@ // any drums inside it. Current jobs retain the individual Demucs stems, so the // engine can rebuild that lane as a control group and route drums separately. +import { EXTRA_STEM_NAMES, DUET_STEM_NAMES } from "./constants.js"; + function playable(stem) { return !!stem?.name && !!stem?.url; } @@ -36,10 +38,11 @@ export function buildPlaybackStems(rawStems, visibleStems, baseStemNames) { if (!visible.some((stem) => stem.name === "original")) return result; const dedicatedNames = new Set(dedicated.map((stem) => stem.name)); - // A completed lead/backing split replaces the base vocals source. Treat the - // base stem as selected so it is not added to the complement as well. - if (dedicatedNames.has("lead_vocals") && dedicatedNames.has("backing_vocals")) { - dedicatedNames.add("vocals"); + // A completed split (lead/backing, or the duet pair) replaces the base + // vocals source. Treat the base stem as selected so it is not added to the + // complement as well and counted twice. + for (const pair of [EXTRA_STEM_NAMES, DUET_STEM_NAMES]) { + if (pair.every((name) => dedicatedNames.has(name))) dedicatedNames.add("vocals"); } const complementNames = baseStemNames.filter((name) => !dedicatedNames.has(name)); diff --git a/static/js/player.js b/static/js/player.js index be2e32bc..120336e2 100644 --- a/static/js/player.js +++ b/static/js/player.js @@ -6,7 +6,7 @@ import { fmtTime } from "./utils.js"; // job.js state. import { showPlaybackError, clearPlaybackError, resolvePlaybackSuccess } from "./job.js"; import { - STEM_NAMES, TRACK_NAMES, EXTRA_STEM_NAMES, STEM_COLORS, PROGRESS_COLOR, + STEM_NAMES, TRACK_NAMES, EXTRA_STEM_NAMES, DUET_STEM_NAMES, STEM_COLORS, PROGRESS_COLOR, LOOP_DEFAULT_START_FRAC, LOOP_DEFAULT_END_FRAC, LANE_VOLUME_MAX, effectiveStemOrder, } from "./constants.js"; @@ -95,14 +95,18 @@ const _STEM_ROW_SELECTORS = [ ]; function applyStemSelectionFilter(presentNames) { - // The order for THIS job: STEM_NAMES, with "vocals" swapped for - // lead_vocals + backing_vocals when the on-demand split (#275) produced - // both. Rows for a name outside `order` (the vocals-family row not in - // play for this job) are hidden outright, not just grayed "unavailable" -- - // unlike the base 6, lead_vocals/backing_vocals aren't part of every job's - // contract, so there's no "not selected this time" state for them to be in. + // The order for THIS job: STEM_NAMES, with "vocals" swapped for whichever + // on-demand split produced a complete pair -- lead_vocals + backing_vocals + // (#275), or the duet pair. Rows for a name outside `order` (the + // vocals-family rows not in play for this job) are hidden outright, not + // just grayed "unavailable" -- unlike the base 6, the split stems aren't + // part of every job's contract, so there's no "not selected this time" + // state for them to be in. Derived from the constants rather than listed + // here, so a new split pair does not silently render as an always-visible + // "unavailable" lane on every job. const order = effectiveStemOrder(presentNames); - const isVocalFamily = (s) => s === "vocals" || s === "lead_vocals" || s === "backing_vocals"; + const vocalFamily = new Set(["vocals", ...EXTRA_STEM_NAMES, ...DUET_STEM_NAMES]); + const isVocalFamily = (s) => vocalFamily.has(s); // Waveform rows: original hides if absent; order rows always show, grayed if absent for (const el of document.querySelectorAll(".stem-waveform-row[data-stem]")) { @@ -904,10 +908,11 @@ export function renderEmptyShell() { stopStemVuLoop(); ensureMixerStateDefaults(); mixerEl.innerHTML = ""; - // lead_vocals/backing_vocals (#275) get rows too, built once here like the - // base 6 -- applyStemSelectionFilter (below) hides them by default since no - // job is loaded yet, and shows them in place of "vocals" once one is. - for (const name of ["original", ...STEM_NAMES, ...EXTRA_STEM_NAMES]) { + // The split lanes (#275 lead/backing, and the duet pair) get rows too, + // built once here like the base 6 -- applyStemSelectionFilter (below) hides + // them by default since no job is loaded yet, and shows them in place of + // "vocals" once one is. + for (const name of ["original", ...STEM_NAMES, ...EXTRA_STEM_NAMES, ...DUET_STEM_NAMES]) { const { row } = renderMixerRow({ name, url: "#" }); mixerEl.appendChild(row); } @@ -1118,17 +1123,24 @@ export function wireUpAudio(jobId, stems, duration, thumbnail, mixUrl = null, ti // original.wav, so it's simply not in `stems` and the mixer/sidebar // rows for it stay hidden.) // - // vocals/lead_vocals/backing_vocals are mutually exclusive (#275): once a - // job's on-demand split has produced both, show those two in place of the - // plain Vocals lane rather than all three -- vocals.wav is never deleted by - // the split, so it would otherwise still show up here too. + // The vocals family is mutually exclusive (#275): once an on-demand split + // has produced a complete pair, show that pair in place of the plain Vocals + // lane rather than all of them -- vocals.wav is never deleted by a split, so + // it would otherwise still show up here too. The pair is whichever split + // ran, so it has to be looked up rather than named: a duet job's stems are + // not in selectedStems (which holds "vocals") and would be dropped by the + // final clause otherwise. const rawPresent = new Set(stems.map((s) => s.name)); - const splitDone = rawPresent.has("lead_vocals") && rawPresent.has("backing_vocals"); + const splitPair = [EXTRA_STEM_NAMES, DUET_STEM_NAMES].find((names) => + names.every((n) => rawPresent.has(n)) + ); const wantsVocals = selectedStems.has("vocals"); stems = stems.filter((s) => { if (s.name === "original") return true; - if (s.name === "vocals") return wantsVocals && !splitDone; - if (s.name === "lead_vocals" || s.name === "backing_vocals") return wantsVocals && splitDone; + if (s.name === "vocals") return wantsVocals && !splitPair; + if (EXTRA_STEM_NAMES.includes(s.name) || DUET_STEM_NAMES.includes(s.name)) { + return wantsVocals && !!splitPair && splitPair.includes(s.name); + } return selectedStems.has(s.name); }); const playbackStems = buildPlaybackStems(rawStems, stems, STEM_NAMES); diff --git a/static/js/state.js b/static/js/state.js index e181d58d..e9a0a44c 100644 --- a/static/js/state.js +++ b/static/js/state.js @@ -158,20 +158,22 @@ export function setStemSelected(name, selected) { saveSelectedStems(); } -// On-demand lead/backing vocal split (#275): "all" (default, plain Vocals -// lane) or "split" (auto-run the split once the next import finishes). -// A page-level setting like selectedStems, applied at whatever moment the -// user submits -- not stored per-job. +// On-demand vocal decomposition (#275): "all" (default, plain Vocals lane), +// "split" (lead + backing) or "duet" (two voices by register). The last two +// are alternatives -- one extra model pass, not both. A page-level setting +// like selectedStems, applied at whatever moment the user submits -- not +// stored per-job. const _VOCAL_SPLIT_MODE_KEY = "stemdeck:vocal-split-mode"; +const _VOCAL_SPLIT_MODES = ["split", "duet"]; export let vocalSplitMode = "all"; export const vocalSplitModeReady = (async () => { try { const v = await storeGet(_VOCAL_SPLIT_MODE_KEY, null); - if (v === "split") vocalSplitMode = v; + if (_VOCAL_SPLIT_MODES.includes(v)) vocalSplitMode = v; } catch (e) { console.warn("[state] failed to load vocal split mode:", e); } })(); export function setVocalSplitMode(mode) { - vocalSplitMode = mode === "split" ? "split" : "all"; + vocalSplitMode = _VOCAL_SPLIT_MODES.includes(mode) ? mode : "all"; storeSet(_VOCAL_SPLIT_MODE_KEY, vocalSplitMode).catch((e) => console.warn("[state] failed to save vocal split mode:", e) ); diff --git a/tests/e2e/file-drop.spec.mjs b/tests/e2e/file-drop.spec.mjs index fed2f314..59d1a4e0 100644 --- a/tests/e2e/file-drop.spec.mjs +++ b/tests/e2e/file-drop.spec.mjs @@ -170,6 +170,37 @@ test.describe("file drop", () => { await expect(page.locator("#url")).toBeVisible(); }); + test("the refusal still has room to show on a small window", async ({ page }) => { + // The composer is one fixed-height row sharing a hard width budget, and the + // URL zone is the only item in it that flexes, so every pixel the stem + // controls ask for comes out of the zone. Adding a third vocal mode button + // took it to nothing but its own padding at 1280px: the input had no room, + // and neither did the message that sits in its place, so refusing a file + // answered into a zero-width box. Asserted at the narrowest width the + // desktop shell allows rather than at Playwright's default, because the + // default is what made this visible by luck. + await seedLibrary(page); + await stubTauri(page); + await stubUpdateCheck(page); + await page.setViewportSize({ width: 1024, height: 700 }); + await page.goto("/", { waitUntil: "domcontentloaded" }); + await page.waitForSelector(".url-wrap"); + + await dropOn(page, "body", "notes.txt", Buffer.from("hello")); + const err = page.locator("#urlDropError"); + await expect(err).toBeVisible(); + expect((await err.boundingBox()).width).toBeGreaterThan(40); + + // And nothing was pushed out of the pill to pay for it: the button that + // starts the job is the one thing that must never be unreachable. + const fits = await page.evaluate(() => { + const c = document.querySelector(".daw-composer").getBoundingClientRect(); + const b = document.querySelector(".daw-process-btn").getBoundingClientRect(); + return b.right <= c.right + 1 && b.width > 0; + }); + expect(fits, "the process button is inside the composer").toBe(true); + }); + test("a library drag is not mistaken for a file drop", async ({ page }) => { // The library's own drags carry no file list. If the document handler // stopped guarding on that, dragging a track would arm the importer. diff --git a/tests/test_cancellation_reach.py b/tests/test_cancellation_reach.py index 8f0a79b8..09c303ec 100644 --- a/tests/test_cancellation_reach.py +++ b/tests/test_cancellation_reach.py @@ -103,7 +103,8 @@ def test_every_worker_spawn_exports_the_parent_pid(): for path in ( "app/pipeline/separate.py", - "app/pipeline/vocal_split.py", + # The on-demand splits (lead/backing, duet) share one supervisor. + "app/pipeline/split_runner.py", "app/pipeline/sections.py", ): src = pathlib.Path(path).read_text() @@ -116,6 +117,7 @@ def test_every_worker_arms_the_watchdog(): for path in ( "app/pipeline/demucs_worker.py", "app/pipeline/vocal_split_worker.py", + "app/pipeline/duet_split_worker.py", "app/pipeline/section_worker.py", ): src = pathlib.Path(path).read_text() diff --git a/tests/test_duet_medleyvox.py b/tests/test_duet_medleyvox.py new file mode 100644 index 00000000..3643e34b --- /dev/null +++ b/tests/test_duet_medleyvox.py @@ -0,0 +1,164 @@ +"""Tests for the same-register duet split. + +Two things are worth pinning down. The collapse check, because it is the one +property of a split that can be judged without knowing the right answer and it +decides whether a second, slower model runs. And the separator's output +contract: stems that sum back to the recording, at the recording's own rate. + +The separator is exercised through a stub model, so the mask maths and the +sample-rate transfer are covered without a 460 MB checkpoint in CI. +""" + +from __future__ import annotations + +import numpy as np +import pytest +import soundfile as sf + +from app.pipeline.duet_medleyvox import MODEL_SR, register_split_collapsed, separate + +SR = 44100 + + +def _write(path, x, sr=SR): + sf.write(str(path), np.asarray(x, dtype=np.float32), sr) + return path + + +def _tone(f, n, sr=SR, amp=1.0): + t = np.arange(n) / sr + return amp * np.sin(2 * np.pi * f * t) + + +def test_collapse_into_one_stem_is_a_failure(tmp_path): + """The shape the register model takes on two singers of the same range.""" + n = SR * 6 + a = _write(tmp_path / "a.wav", _tone(220, n)) + b = _write(tmp_path / "b.wav", _tone(220, n, amp=0.02)) + assert register_split_collapsed(a, b) + + +def test_alternating_split_passes(tmp_path): + """Two stems that trade the lead, as a separated duet does.""" + n = SR * 8 + gate = np.zeros(n) + for i in range(0, n, SR * 2): + gate[i : i + SR] = 1.0 + a = _write(tmp_path / "a.wav", _tone(220, n) * gate) + b = _write(tmp_path / "b.wav", _tone(330, n) * (1.0 - gate)) + assert not register_split_collapsed(a, b) + + +def test_silence_counts_as_failure(tmp_path): + n = SR * 4 + a = _write(tmp_path / "a.wav", np.zeros(n)) + b = _write(tmp_path / "b.wav", np.zeros(n)) + assert register_split_collapsed(a, b) + + +class _StubModel: + """Stands in for the checkpoint: splits by frequency, nothing learned. + + Lets the mask averaging and the 24 kHz-to-file-rate transfer be tested + without downloading 460 MB, which is the part with real arithmetic in it. + """ + + def to(self, device): + return self + + def eval(self): + return self + + def __call__(self, x): + import torch + + wav = x[0, 0].numpy().astype(np.float64) + spec = np.fft.rfft(wav) + freqs = np.fft.rfftfreq(len(wav), 1 / MODEL_SR) + lo = np.fft.irfft(spec * (freqs < 400), n=len(wav)) + hi = wav - lo + return torch.as_tensor(np.stack([lo, hi])[None, :, :], dtype=torch.float32) + + +def test_separate_preserves_the_recording_and_its_rate(): + """Whatever the model says, the two stems must add back to the input. + + Mixture consistency plus the mask normalisation is what makes solo and mute + behave in the mixer, so it is worth asserting rather than assuming. + """ + n = SR * 3 + mix = (_tone(180, n) + _tone(900, n, amp=0.7)).reshape(-1, 1) + mix = mix / np.max(np.abs(mix)) + + stems = separate([_StubModel()], mix, SR, "cpu") + + assert len(stems) == 2 + for s in stems: + assert len(s) == n, "stems must come back at the input length" + assert np.all(np.isfinite(s)) + + mono = mix[:, 0] + residual = mono - (stems[0] + stems[1]) + rel = np.sqrt(np.mean(residual**2)) / np.sqrt(np.mean(mono**2)) + assert rel < 1e-6, f"stems do not sum to the input (relative residual {rel:.2e})" + + +def test_separate_keeps_content_above_the_models_ceiling(): + """The model is 24 kHz; the output must not be band-limited to 12 kHz. + + Separating at 24 kHz and resampling back would silently drop the air and + most of the sibilance from a vocal. The mask is transferred to the file's + own grid instead, so energy above the model's ceiling has to survive. + """ + n = SR * 3 + mix = (_tone(300, n) + _tone(16000, n, amp=0.3)).reshape(-1, 1) + mix = mix / np.max(np.abs(mix)) + + stems = separate([_StubModel()], mix, SR, "cpu") + recombined = stems[0] + stems[1] + + spec = np.abs(np.fft.rfft(recombined)) + freqs = np.fft.rfftfreq(len(recombined), 1 / SR) + above = float(np.sum(spec[freqs > 12000] ** 2)) + total = float(np.sum(spec**2)) + 1e-12 + assert above / total > 0.01, "content above 12 kHz was lost" + + +@pytest.mark.parametrize("channels", [1, 2]) +def test_separate_keeps_the_input_channel_count(channels): + """Stereo in, stereo out. + + The model is mono, so the obvious implementation hands back mono stems while + every other stem in the job is stereo. That is not a cosmetic difference: + a mono stem among stereo ones loses its pan position and width in the mixer. + The mask is decided on the downmix and applied to each channel. + """ + n = SR * 2 + left = _tone(220, n) + _tone(660, n, amp=0.5) + right = _tone(220, n, amp=0.4) + _tone(660, n) + wav = left.reshape(-1, 1) if channels == 1 else np.stack([left, right], axis=1) + + stems = separate([_StubModel()], wav, SR, "cpu") + + for stem in stems: + assert len(stem) == n + if channels == 1: + assert stem.ndim == 1 + else: + assert stem.shape == (n, 2) + + # Per channel, the two stems must still add back to that channel. + for ch in range(channels): + src = wav[:, ch] + got = stems[0] + stems[1] + got = got if channels == 1 else got[:, ch] + rel = np.sqrt(np.mean((src - got) ** 2)) / np.sqrt(np.mean(src**2)) + assert rel < 1e-6, f"channel {ch} does not reconstruct (residual {rel:.2e})" + + if channels == 2: + # A real stereo image must survive: the two channels of a stem must not + # collapse to the same signal. + for stem in stems: + if np.sqrt(np.mean(stem[:, 0] ** 2)) > 1e-6: + diff = np.mean(np.abs(stem[:, 0] - stem[:, 1])) + assert diff > 1e-6, "stereo stem collapsed to dual mono" diff --git a/tests/test_jobs_api.py b/tests/test_jobs_api.py index 3a48d042..078e182b 100644 --- a/tests/test_jobs_api.py +++ b/tests/test_jobs_api.py @@ -821,3 +821,116 @@ def test_submit_captures_the_song_structure_setting_on_the_job(client, monkeypat r = client.post("/api/jobs", json={"url": "https://youtu.be/oHg5SJYRHA0"}) assert r.status_code == 200 assert registry.get(r.json()["job_id"]).auto_sections is False + + +# ─── POST /jobs/{id}/duet-split ────────────────────────────────────────────── +# +# Mirrors the vocal-split cases above. The two splits are alternatives, so the +# last test here is the one that matters most: running one must not make the +# other unavailable. + + +def test_duet_split_404_unknown_job(client): + r = client.post("/api/jobs/000000000000/duet-split") + assert r.status_code == 404 + + +def test_duet_split_404_job_not_done(client, tmp_path, monkeypatch): + import app.api.jobs as jobs_mod + + monkeypatch.setattr(jobs_mod, "JOBS_DIR", tmp_path) + job = Job(id="abcdefabc630", status="separating", title="In progress") + _jobs[job.id] = job + + r = client.post(f"/api/jobs/{job.id}/duet-split") + assert r.status_code == 404 + + +def _fake_duet(job_arg, stems_dir_arg): + (stems_dir_arg / "voice_1.wav").write_bytes(b"RIFF") + (stems_dir_arg / "voice_2.wav").write_bytes(b"RIFF") + return ["voice_1", "voice_2"] + + +def test_duet_split_success(client, tmp_path, monkeypatch): + import app.api.jobs as jobs_mod + + job, _ = _make_done_job_with_vocals(tmp_path, monkeypatch, job_id="abcdefabc630") + monkeypatch.setattr(jobs_mod, "split_duet", _fake_duet) + + r = client.post(f"/api/jobs/{job.id}/duet-split") + assert r.status_code == 200 + body = r.json() + assert body["duet_split"] == "done" + names = {s["name"] for s in body["stems"]} + assert {"voice_1", "voice_2"} <= names + assert job.duet_split == "done" + + +def test_duet_split_409_when_already_running(client, tmp_path, monkeypatch): + job, _ = _make_done_job_with_vocals(tmp_path, monkeypatch, job_id="abcdefabc630") + job.duet_split = "running" + + r = client.post(f"/api/jobs/{job.id}/duet-split") + assert r.status_code == 409 + + +def test_duet_split_202_idempotent_when_already_done(client, tmp_path, monkeypatch): + import app.api.jobs as jobs_mod + + job, _ = _make_done_job_with_vocals(tmp_path, monkeypatch, job_id="abcdefabc630") + job.duet_split = "done" + calls = [] + + def should_not_run(job_arg, stems_dir_arg): + calls.append(1) + return ["voice_1", "voice_2"] + + monkeypatch.setattr(jobs_mod, "split_duet", should_not_run) + + r = client.post(f"/api/jobs/{job.id}/duet-split") + assert r.status_code == 202 + assert calls == [], "the expensive model must not re-run once the split is done" + + +def test_duet_split_failure_keeps_job_done_with_base_stems(client, tmp_path, monkeypatch): + import app.api.jobs as jobs_mod + from app.pipeline.errors import SeparationError + + job, stems_dir = _make_done_job_with_vocals(tmp_path, monkeypatch, job_id="abcdefabc630") + + def boom(job_arg, stems_dir_arg): + raise SeparationError("duet split failed: model download failed") + + monkeypatch.setattr(jobs_mod, "split_duet", boom) + + r = client.post(f"/api/jobs/{job.id}/duet-split") + assert r.status_code == 500 + assert job.status == "done", "a failed split must not fail the job" + assert job.duet_split == "error" + assert (stems_dir / "duet_split_error.txt").is_file() + assert (stems_dir / "vocals.wav").is_file(), "base stems must be untouched" + + +def test_duet_split_and_vocal_split_are_independent(client, tmp_path, monkeypatch): + """Running one split must not block or overwrite the other.""" + import app.api.jobs as jobs_mod + + job, _ = _make_done_job_with_vocals(tmp_path, monkeypatch, job_id="abcdefabc630") + + def fake_vocals(job_arg, stems_dir_arg): + (stems_dir_arg / "lead_vocals.wav").write_bytes(b"RIFF") + (stems_dir_arg / "backing_vocals.wav").write_bytes(b"RIFF") + return ["lead_vocals", "backing_vocals"] + + monkeypatch.setattr(jobs_mod, "split_vocals", fake_vocals) + monkeypatch.setattr(jobs_mod, "split_duet", _fake_duet) + + assert client.post(f"/api/jobs/{job.id}/vocal-split").status_code == 200 + assert client.post(f"/api/jobs/{job.id}/duet-split").status_code == 200 + + body = client.get(f"/api/jobs/{job.id}").json() + names = {s["name"] for s in body["stems"]} + assert {"lead_vocals", "backing_vocals", "voice_1", "voice_2"} <= names + assert body["vocal_split"] == "done" + assert body["duet_split"] == "done" diff --git a/tests/test_pipeline_duet_split.py b/tests/test_pipeline_duet_split.py new file mode 100644 index 00000000..caa4c781 --- /dev/null +++ b/tests/test_pipeline_duet_split.py @@ -0,0 +1,106 @@ +"""Tests for the on-demand duet split. + +Mirrors test_pipeline_vocal_split.py: the real subprocess machinery (Popen, +stderr streaming, watchdog, cleanup) runs end-to-end against a stub worker +script swapped in via the _spawn_cmd seam, so no real ML model or GPU is +needed. +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +import pytest + +from app.core.models import Job +from app.pipeline import duet_split as ds_mod +from app.pipeline.errors import SeparationError + +_SUCCESS_WORKER = """ +import sys, os +out_dir = sys.argv[3] +open(os.path.join(out_dir, "voice_1.wav"), "wb").write(b"RIFF") +open(os.path.join(out_dir, "voice_2.wav"), "wb").write(b"RIFF") +sys.stderr.write("@@DONE@@\\n") +sys.stderr.flush() +""" + +_FAILING_WORKER = """ +import sys, json +sys.stderr.write("@@ERROR@@" + json.dumps("model download failed: connection reset") + "\\n") +sys.stderr.flush() +sys.exit(1) +""" + +# Exits 0 / prints @@DONE@@ but never writes the two output files -- a +# worker-side bug this should still catch rather than trust blindly. +_LIAR_WORKER = """ +import sys +sys.stderr.write("@@DONE@@\\n") +sys.stderr.flush() +""" + + +def _stub(code: str): + def fake_spawn(device: str, vocals_path: Path, out_dir: Path) -> list[str]: + return [sys.executable, "-c", code, device, str(vocals_path), str(out_dir)] + + return fake_spawn + + +@pytest.fixture() +def stems_dir(tmp_path: Path) -> Path: + d = tmp_path / "stems" + d.mkdir() + (d / "vocals.wav").write_bytes(b"RIFF") + return d + + +@pytest.fixture() +def job() -> Job: + return Job(id="abcdefabc630") + + +def test_split_duet_success(job, stems_dir, monkeypatch): + monkeypatch.setattr(ds_mod, "get_demucs_device", lambda: "cpu") + monkeypatch.setattr(ds_mod, "_spawn_cmd", _stub(_SUCCESS_WORKER)) + + result = ds_mod.split_duet(job, stems_dir) + + assert result == ["voice_1", "voice_2"] + assert (stems_dir / "voice_1.wav").is_file() + assert (stems_dir / "voice_2.wav").is_file() + + +def test_split_duet_missing_vocals(job, tmp_path, monkeypatch): + empty = tmp_path / "stems" + empty.mkdir() + with pytest.raises(SeparationError, match="vocals.wav not found"): + ds_mod.split_duet(job, empty) + + +def test_split_duet_worker_failure_surfaces_stderr(job, stems_dir, monkeypatch): + monkeypatch.setattr(ds_mod, "get_demucs_device", lambda: "cpu") + monkeypatch.setattr(ds_mod, "_spawn_cmd", _stub(_FAILING_WORKER)) + + with pytest.raises(SeparationError) as exc: + ds_mod.split_duet(job, stems_dir) + assert "connection reset" in str(exc.value) + + +def test_split_duet_rejects_worker_that_wrote_nothing(job, stems_dir, monkeypatch): + monkeypatch.setattr(ds_mod, "get_demucs_device", lambda: "cpu") + monkeypatch.setattr(ds_mod, "_spawn_cmd", _stub(_LIAR_WORKER)) + + with pytest.raises(SeparationError, match="did not produce"): + ds_mod.split_duet(job, stems_dir) + + +def test_split_duet_leaves_base_stems_untouched(job, stems_dir, monkeypatch): + monkeypatch.setattr(ds_mod, "get_demucs_device", lambda: "cpu") + monkeypatch.setattr(ds_mod, "_spawn_cmd", _stub(_SUCCESS_WORKER)) + + ds_mod.split_duet(job, stems_dir) + + assert (stems_dir / "vocals.wav").read_bytes() == b"RIFF" diff --git a/tests/test_stems_api.py b/tests/test_stems_api.py index d98e9b78..0ddd700e 100644 --- a/tests/test_stems_api.py +++ b/tests/test_stems_api.py @@ -183,10 +183,50 @@ def test_peaks_returns_json_for_done_job(client, tmp_path): r = client.get(f"/api/jobs/{job.id}/stems/peaks.json") assert r.status_code == 200 assert r.headers["content-type"] == "application/json" - assert "immutable" in r.headers.get("cache-control", "") assert r.json() == payload +def test_peaks_are_never_served_immutable(client, tmp_path): + """The browser must always ask whether this file changed. + + It used to be sent `immutable` for a year, which was true when a finished + job's stems never changed again. The on-demand splits broke that: both add + stems to a done job and rewrite peaks.json. `immutable` is taken literally, + so the browser would not revalidate even on a hard reload, and the user got + the new audio under the old waveform. + """ + job = Job(id="abcdefabcde1") + job.status = "done" + _jobs[job.id] = job + _make_peaks_file(tmp_path, job.id, {"vocals": [[-0.1, 0.2]]}) + + cc = client.get(f"/api/jobs/{job.id}/stems/peaks.json").headers.get("cache-control", "") + assert "immutable" not in cc, f"peaks.json must stay revalidatable, got {cc!r}" + assert "must-revalidate" in cc or "no-cache" in cc + + +def test_peaks_reflect_a_split_that_added_stems(client, tmp_path): + """A split rewrites peaks.json, and the next request must show the new stems.""" + job = Job(id="abcdefabcde2") + job.status = "done" + _jobs[job.id] = job + _make_peaks_file(tmp_path, job.id, {"vocals": [[-0.1, 0.2]]}) + + first = client.get(f"/api/jobs/{job.id}/stems/peaks.json").json() + assert "voice_1" not in first + + # what the duet split does to this file + _make_peaks_file( + tmp_path, + job.id, + {"vocals": [[-0.1, 0.2]], "voice_1": [[-0.3, 0.3]], "voice_2": [[-0.4, 0.4]]}, + ) + + second = client.get(f"/api/jobs/{job.id}/stems/peaks.json").json() + assert "voice_1" in second and "voice_2" in second + assert second["voice_1"] == [[-0.3, 0.3]] + + def test_peaks_404_when_file_missing(client, tmp_path): job = Job(id="abcdefabcdeb") job.status = "done" diff --git a/tests/test_subprocess_encoding.py b/tests/test_subprocess_encoding.py index f0806e4d..d036413c 100644 --- a/tests/test_subprocess_encoding.py +++ b/tests/test_subprocess_encoding.py @@ -22,7 +22,8 @@ SITES = [ ("app/pipeline/separate.py", "the Demucs worker"), - ("app/pipeline/vocal_split.py", "the vocal-split worker"), + # Both on-demand splits (lead/backing, duet) share one supervisor. + ("app/pipeline/split_runner.py", "the on-demand split workers"), ("app/api/jobs.py", "ffprobe on upload"), ] @@ -35,7 +36,7 @@ def test_decoding_is_pinned_not_left_to_the_locale(path: str, what: str): assert 'errors="replace"' in src, f"{what} would still raise on an undecodable byte" -@pytest.mark.parametrize("path", ["app/pipeline/separate.py", "app/pipeline/vocal_split.py"]) +@pytest.mark.parametrize("path", ["app/pipeline/separate.py", "app/pipeline/split_runner.py"]) def test_children_are_told_to_write_utf8(path: str): """The parent reading utf-8 is only half of it.""" src = Path(path).read_text(encoding="utf-8") diff --git a/uv.lock b/uv.lock index e600d1ba..391e7cb5 100644 --- a/uv.lock +++ b/uv.lock @@ -85,35 +85,52 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/da/42/e921fccf5015463e32a3cf6ee7f980a6ed0f395ceeaa45060b61d86486c2/anyio-4.13.0-py3-none-any.whl", hash = "sha256:08b310f9e24a9594186fd75b4f73f4a4152069e3853f1ed8bfbf58369f4ad708", size = 114353, upload-time = "2026-03-24T12:59:08.246Z" }, ] +[[package]] +name = "asteroid-filterbanks" +version = "0.4.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "torch", version = "2.2.2", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/90/fa/5c2be1f96dc179f83cdd3bb267edbd1f47d08f756785c016d5c2163901a7/asteroid-filterbanks-0.4.0.tar.gz", hash = "sha256:415f89d1dcf2b13b35f03f7a9370968ac4e6fa6800633c522dac992b283409b9", size = 24599, upload-time = "2021-04-09T20:03:07.456Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c5/7c/83ff6046176a675e6a1e8aeefed8892cd97fe7c46af93cc540d1b24b8323/asteroid_filterbanks-0.4.0-py3-none-any.whl", hash = "sha256:4932ac8b6acc6e08fb87cbe8ece84215b5a74eee284fe83acf3540a72a02eaf5", size = 29912, upload-time = "2021-04-09T20:03:05.817Z" }, +] + [[package]] name = "audio-separator" version = "0.44.5" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "audioop-lts", marker = "python_full_version >= '3.13'" }, - { name = "beartype" }, - { name = "diffq", marker = "sys_platform != 'win32'" }, + { name = "audioop-lts", marker = "(python_full_version >= '3.13' and platform_machine != 'x86_64') or (python_full_version >= '3.13' and sys_platform != 'darwin')" }, + { name = "beartype", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "diffq", marker = "(platform_machine != 'x86_64' and sys_platform == 'darwin') or (sys_platform != 'darwin' and sys_platform != 'win32')" }, { name = "diffq-fixed", marker = "sys_platform == 'win32'" }, - { name = "einops" }, - { name = "julius" }, - { name = "librosa" }, - { name = "ml-collections" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "onnx-weekly" }, - { name = "onnx2torch-py313" }, - { name = "pydub" }, - { name = "pyyaml" }, - { name = "requests" }, - { name = "resampy" }, - { name = "rotary-embedding-torch", version = "0.6.5", source = { registry = "https://pypi.org/simple" } }, - { name = "samplerate" }, - { name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "six" }, - { name = "soundfile" }, - { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" } }, - { name = "tqdm" }, + { name = "einops", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "julius", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "librosa", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "ml-collections", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "onnx-weekly", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "onnx2torch-py313", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "pydub", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "pyyaml", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "requests", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "resampy", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "rotary-embedding-torch", version = "0.6.5", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "samplerate", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "six", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "soundfile", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "tqdm", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/e6/a1/91884a9cbbace7d1eb480f6446e10fc2cdf0aa8d86da414f4593557b6340/audio_separator-0.44.5.tar.gz", hash = "sha256:58866bc61d0c692fff8a52cda67c284f7847a844048987135677a007bf0ae794", size = 344654, upload-time = "2026-07-20T22:02:44.933Z" } wheels = [ @@ -391,8 +408,8 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/66/54/eb9bfc647b19f2009dd5c7f5ec51c4e6ca831725f1aea7a993034f483147/contourpy-1.3.2.tar.gz", hash = "sha256:b6945942715a034c671b7fc54f9588126b0b8bf23db2696e3ca8328f3ff0ab54", size = 13466130, upload-time = "2025-04-15T17:47:53.79Z" } wheels = [ @@ -466,8 +483,8 @@ resolution-markers = [ "(python_full_version == '3.11.*' and platform_machine != 'x86_64') or (python_full_version == '3.11.*' and sys_platform != 'darwin')", ] dependencies = [ - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/58/01/1253e6698a07380cd31a736d248a3f2a50a7c88779a1813da27503cadc2a/contourpy-1.3.3.tar.gz", hash = "sha256:083e12155b210502d0bca491432bb04d56dc3432f95a979b429f2848c3dbe880", size = 13466174, upload-time = "2025-07-26T12:03:12.549Z" } wheels = [ @@ -621,10 +638,10 @@ name = "diffq" version = "0.2.4" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cython" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" } }, + { name = "cython", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/5a/fd/4c58807bf855c5929ffa6da55f26dd6b9ae462a4193f5e09cc49fbbfd451/diffq-0.2.4.tar.gz", hash = "sha256:049064861e974ebf00d0badab8b324c775037371419eda3150985b9d477b5bd2", size = 157139, upload-time = "2023-05-05T12:39:43.089Z" } wheels = [ @@ -640,10 +657,10 @@ name = "diffq-fixed" version = "0.2.4" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cython" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" } }, + { name = "cython", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/be/96/8ca5acf5ecfd4108aa6f345cef171f2fda0c081cf0e7430671712586f172/diffq_fixed-0.2.4.tar.gz", hash = "sha256:cbc906b76fa23d1cf3c0ae517fbab744d9624980a068fe7fbb00dede1d83208d", size = 189695, upload-time = "2024-11-07T21:33:26.204Z" } wheels = [ @@ -681,7 +698,7 @@ name = "exceptiongroup" version = "1.3.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "typing-extensions" }, + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" } wheels = [ @@ -1256,16 +1273,16 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "contourpy", version = "1.3.2", source = { registry = "https://pypi.org/simple" } }, - { name = "cycler" }, - { name = "fonttools" }, - { name = "kiwisolver" }, - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, - { name = "packaging" }, - { name = "pillow" }, - { name = "pyparsing" }, - { name = "python-dateutil" }, + { name = "contourpy", version = "1.3.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "cycler", marker = "python_full_version < '3.11'" }, + { name = "fonttools", marker = "python_full_version < '3.11'" }, + { name = "kiwisolver", marker = "python_full_version < '3.11'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "packaging", marker = "python_full_version < '3.11'" }, + { name = "pillow", marker = "python_full_version < '3.11'" }, + { name = "pyparsing", marker = "python_full_version < '3.11'" }, + { name = "python-dateutil", marker = "python_full_version < '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/63/1b/4be5be87d43d327a0cf4de1a56e86f7f84c89312452406cf122efe2839e6/matplotlib-3.10.9.tar.gz", hash = "sha256:fd66508e8c6877d98e586654b608a0456db8d7e8a546eb1e2600efd957302358", size = 34811233, upload-time = "2026-04-24T00:14:13.539Z" } wheels = [ @@ -1323,16 +1340,16 @@ resolution-markers = [ "(python_full_version == '3.11.*' and platform_machine != 'x86_64') or (python_full_version == '3.11.*' and sys_platform != 'darwin')", ] dependencies = [ - { name = "contourpy", version = "1.3.3", source = { registry = "https://pypi.org/simple" } }, - { name = "cycler" }, - { name = "fonttools" }, - { name = "kiwisolver" }, - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, - { name = "packaging" }, - { name = "pillow" }, - { name = "pyparsing" }, - { name = "python-dateutil" }, + { name = "contourpy", version = "1.3.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "cycler", marker = "python_full_version >= '3.11'" }, + { name = "fonttools", marker = "python_full_version >= '3.11'" }, + { name = "kiwisolver", marker = "python_full_version >= '3.11'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "packaging", marker = "python_full_version >= '3.11'" }, + { name = "pillow", marker = "python_full_version >= '3.11'" }, + { name = "pyparsing", marker = "python_full_version >= '3.11'" }, + { name = "python-dateutil", marker = "python_full_version >= '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/49/64/f9a391af28f518b11ad45a8a712353c94a0aefce09d3703200e5c54b610a/matplotlib-3.11.1.tar.gz", hash = "sha256:69647db5746941c793d6e445a4cd349323ffb87d9cc958c2ad84a659b4832d30", size = 32612045, upload-time = "2026-07-18T03:39:46.63Z" } wheels = [ @@ -1374,8 +1391,8 @@ name = "ml-collections" version = "1.1.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "absl-py" }, - { name = "pyyaml" }, + { name = "absl-py", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "pyyaml", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/b8/f8/1a9ae6696dbb6bc9c44ddf5c5e84710d77fe9a35a57e8a06722e1836a4a6/ml_collections-1.1.0.tar.gz", hash = "sha256:0ac1ac6511b9f1566863e0bb0afad0c64e906ea278ad3f4d2144a55322671f6f", size = 61356, upload-time = "2025-04-17T08:25:02.247Z" } wheels = [ @@ -1387,8 +1404,8 @@ name = "ml-dtypes" version = "0.6.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/12/72/307d7c4bd0600601c7133fba5cb78af7db968152951c1cd473abb1cda782/ml_dtypes-0.6.0.tar.gz", hash = "sha256:5e60251d32ced5598972e4d5e06a2f044341f9291402551a3f6f0ec44f9299b0", size = 3032327, upload-time = "2026-08-13T14:14:40.215Z" } wheels = [ @@ -1512,8 +1529,8 @@ resolution-markers = [ "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'", ] dependencies = [ - { name = "llvmlite", version = "0.44.0", source = { registry = "https://pypi.org/simple" } }, - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" } }, + { name = "llvmlite", version = "0.44.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/1c/a0/e21f57604304aa03ebb8e098429222722ad99176a4f979d34af1d1ee80da/numba-0.61.2.tar.gz", hash = "sha256:8750ee147940a6637b80ecf7f95062185ad8726c8c28a2295b8ec1160a196f7d", size = 2820615, upload-time = "2025-04-09T02:58:07.659Z" } wheels = [ @@ -1534,9 +1551,9 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "llvmlite", version = "0.47.0", source = { registry = "https://pypi.org/simple" } }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "llvmlite", version = "0.47.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/f6/c5/db2ac3685833d626c0dcae6bd2330cd68433e1fd248d15f70998160d3ad7/numba-0.65.1.tar.gz", hash = "sha256:19357146c32fe9ed25059ab915e8465fb13951cf6b0aace3826b76886373ab23", size = 2765600, upload-time = "2026-04-24T02:02:56.551Z" } wheels = [ @@ -1739,7 +1756,7 @@ name = "nvidia-cudnn-cu12" version = "9.1.0.70" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas-cu12" }, + { name = "nvidia-cublas-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/9f/fd/713452cd72343f682b1c7b9321e23829f00b842ceaedcda96e742ea0b0b3/nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl", hash = "sha256:165764f44ef8c61fcdfdfdbe769d687e06374059fbb388b6c89ecb0e28793a6f", size = 664752741, upload-time = "2024-04-22T15:24:15.253Z" }, @@ -1750,7 +1767,7 @@ name = "nvidia-cufft-cu12" version = "11.2.1.3" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink-cu12" }, + { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/27/94/3266821f65b92b3138631e9c8e7fe1fb513804ac934485a8d05776e1dd43/nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl", hash = "sha256:f083fc24912aa410be21fa16d157fed2055dab1cc4b6934a0e03cba69eb242b9", size = 211459117, upload-time = "2024-04-03T20:57:40.402Z" }, @@ -1769,9 +1786,9 @@ name = "nvidia-cusolver-cu12" version = "11.6.1.9" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-cublas-cu12" }, - { name = "nvidia-cusparse-cu12" }, - { name = "nvidia-nvjitlink-cu12" }, + { name = "nvidia-cublas-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "nvidia-cusparse-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/3a/e1/5b9089a4b2a4790dfdea8b3a006052cfecff58139d5a4e34cb1a51df8d6f/nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl", hash = "sha256:19e33fa442bcfd085b3086c4ebf7e8debc07cfe01e11513cc6d332fd918ac260", size = 127936057, upload-time = "2024-04-03T20:58:28.735Z" }, @@ -1782,7 +1799,7 @@ name = "nvidia-cusparse-cu12" version = "12.3.1.170" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "nvidia-nvjitlink-cu12" }, + { name = "nvidia-nvjitlink-cu12", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/db/f7/97a9ea26ed4bbbfc2d470994b8b4f338ef663be97b8f677519ac195e113d/nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl", hash = "sha256:ea4f11a2904e2a8dc4b1833cc1b5181cde564edd0d5cd33e3c168eff2d1863f1", size = 207454763, upload-time = "2024-04-03T20:58:59.995Z" }, @@ -1838,11 +1855,11 @@ name = "onnx-weekly" version = "1.23.0.dev20260817" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "ml-dtypes" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "protobuf" }, - { name = "typing-extensions" }, + { name = "ml-dtypes", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "protobuf", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "typing-extensions", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/9e/44/f24ce74b129bbcbeac50e07e30ee807f3abaa711f81873fe2afe110e484d/onnx_weekly-1.23.0.dev20260817.tar.gz", hash = "sha256:860d42175f474562fe05236f168ba2340c66b566119750792caa8cada18c74da", size = 5983184, upload-time = "2026-08-17T00:28:03.157Z" } wheels = [ @@ -1871,11 +1888,11 @@ name = "onnx2torch-py313" version = "1.6.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "onnx-weekly" }, - { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" } }, - { name = "torchvision" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "onnx-weekly", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "torchvision", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/1e/bb/6301c1ea4c8f8859aff60d2d58a038fd68f9cdb507d1c60776d4e3b93b1f/onnx2torch_py313-1.6.0.tar.gz", hash = "sha256:d27b54c7e170f12ad252fe186e6cab8f3cade8ee4e596682acc5a3afddadb753", size = 49594, upload-time = "2025-04-11T03:52:32.88Z" } wheels = [ @@ -1890,11 +1907,11 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "flatbuffers" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" } }, - { name = "packaging" }, - { name = "protobuf" }, - { name = "sympy" }, + { name = "flatbuffers", marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "packaging", marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "protobuf", marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "sympy", marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/15/41/3253db975a90c3ce1d475e2a230773a21cd7998537f0657947df6fb79861/onnxruntime-1.24.3-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:3e6456801c66b095c5cd68e690ca25db970ea5202bd0c5b84a2c3ef7731c5a3c", size = 17332766, upload-time = "2026-03-05T17:18:59.714Z" }, @@ -1926,10 +1943,10 @@ resolution-markers = [ "(python_full_version == '3.11.*' and platform_machine != 'x86_64') or (python_full_version == '3.11.*' and sys_platform != 'darwin')", ] dependencies = [ - { name = "flatbuffers" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" } }, - { name = "packaging" }, - { name = "protobuf" }, + { name = "flatbuffers", marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "packaging", marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "protobuf", marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/7c/a8/0520890321b8ff40b908cf165a93eb58fbc8f85c14db637277ea866c9544/onnxruntime-1.29.0-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:07c5907474dec4a2792fd7626b753dc66707808385a6d9eecf993db0066a9d0f", size = 21420890, upload-time = "2026-08-17T22:53:33.429Z" }, @@ -2359,9 +2376,9 @@ name = "resampy" version = "0.4.3" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "numba", version = "0.65.1", source = { registry = "https://pypi.org/simple" } }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "numba", version = "0.65.1", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/29/f1/34be702a69a5d272e844c98cee82351f880985cfbca0cc86378011078497/resampy-0.4.3.tar.gz", hash = "sha256:a0d1c28398f0e55994b739650afef4e3974115edbe96cd4bb81968425e916e47", size = 3080604, upload-time = "2024-03-05T20:36:08.119Z" } wheels = [ @@ -2388,8 +2405,8 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "einops" }, - { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" } }, + { name = "einops", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/99/7e/8dbb4a950a99ec2f459ec1eb73818c519b56a1fee8a9a98f7a57460a0d5a/rotary_embedding_torch-0.6.5.tar.gz", hash = "sha256:a3623274c559e0215922edc7cd14068d5a64dea1ca469e1ecd642b293ee8af78", size = 7114, upload-time = "2024-08-20T20:52:44.155Z" } wheels = [ @@ -2406,8 +2423,8 @@ resolution-markers = [ "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'", ] dependencies = [ - { name = "einops" }, - { name = "torch", version = "2.2.2", source = { registry = "https://pypi.org/simple" } }, + { name = "einops", marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "torch", version = "2.2.2", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/71/38/74783585b1f0282fddd3faf1abd6dd20977255c27e31737eced2d7ec05f1/rotary_embedding_torch-0.8.9.tar.gz", hash = "sha256:b213f153cad1d108064d930544fb3af678d56515893d3f869a7a146f87997e3f", size = 7497, upload-time = "2025-07-27T01:26:14.675Z" } wheels = [ @@ -2444,9 +2461,9 @@ name = "samplerate" version = "0.1.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "cffi" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "cffi", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/da/9c/6a13fbc59b1ceaef194a53d3aeafa983057133fb13660c423854184eafa7/samplerate-0.1.0.tar.gz", hash = "sha256:75ef725e6cd9c4545569caf4c47147beab7b53b2c36e5122e8c285d348f88847", size = 4044998, upload-time = "2017-02-24T00:01:56.505Z" } wheels = [ @@ -2462,11 +2479,11 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "joblib" }, - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, - { name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" } }, - { name = "threadpoolctl" }, + { name = "joblib", marker = "python_full_version < '3.11'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "scipy", version = "1.15.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "threadpoolctl", marker = "python_full_version < '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/98/c2/a7855e41c9d285dfe86dc50b250978105dce513d6e459ea66a6aeb0e1e0c/scikit_learn-1.7.2.tar.gz", hash = "sha256:20e9e49ecd130598f1ca38a1d85090e1a600147b9c02fa6f15d69cb53d968fda", size = 7193136, upload-time = "2025-09-09T08:21:29.075Z" } wheels = [ @@ -2509,11 +2526,11 @@ resolution-markers = [ "(python_full_version == '3.11.*' and platform_machine != 'x86_64') or (python_full_version == '3.11.*' and sys_platform != 'darwin')", ] dependencies = [ - { name = "joblib" }, - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, - { name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" } }, - { name = "threadpoolctl" }, + { name = "joblib", marker = "python_full_version >= '3.11'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "scipy", version = "1.17.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "threadpoolctl", marker = "python_full_version >= '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/0e/d4/40988bf3b8e34feec1d0e6a051446b1f66225f8529b9309becaeef62b6c4/scikit_learn-1.8.0.tar.gz", hash = "sha256:9bccbb3b40e3de10351f8f5068e105d0f4083b1a65fa07b6634fbc401a6287fd", size = 7335585, upload-time = "2025-12-10T07:08:53.618Z" } wheels = [ @@ -2552,8 +2569,8 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/0f/37/6964b830433e654ec7485e45a00fc9a27cf868d622838f6b6d9c5ec0d532/scipy-1.15.3.tar.gz", hash = "sha256:eae3cf522bc7df64b42cad3925c876e1b0b6c35c1337c93e12c0f366f55b0eaf", size = 59419214, upload-time = "2025-05-08T16:13:05.955Z" } wheels = [ @@ -2616,8 +2633,8 @@ resolution-markers = [ "(python_full_version == '3.11.*' and platform_machine != 'x86_64') or (python_full_version == '3.11.*' and sys_platform != 'darwin')", ] dependencies = [ - { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "numpy", version = "1.26.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, ] sdist = { url = "https://files.pythonhosted.org/packages/7a/97/5a3609c4f8d58b039179648e62dd220f89864f56f7357f5d4f45c29eb2cc/scipy-1.17.1.tar.gz", hash = "sha256:95d8e012d8cb8816c226aef832200b1d45109ed4464303e997c5b13122b297c0", size = 30573822, upload-time = "2026-02-23T00:26:24.851Z" } wheels = [ @@ -2744,8 +2761,8 @@ name = "standard-aifc" version = "3.13.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "audioop-lts" }, - { name = "standard-chunk" }, + { name = "audioop-lts", marker = "python_full_version >= '3.13'" }, + { name = "standard-chunk", marker = "python_full_version >= '3.13'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/c4/53/6050dc3dde1671eb3db592c13b55a8005e5040131f7509cef0215212cb84/standard_aifc-3.13.0.tar.gz", hash = "sha256:64e249c7cb4b3daf2fdba4e95721f811bde8bdfc43ad9f936589b7bb2fae2e43", size = 15240, upload-time = "2024-10-30T16:01:31.772Z" } wheels = [ @@ -2766,7 +2783,7 @@ name = "standard-sunau" version = "3.13.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "audioop-lts" }, + { name = "audioop-lts", marker = "python_full_version >= '3.13'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/66/e3/ce8d38cb2d70e05ffeddc28bb09bad77cfef979eb0a299c9117f7ed4e6a9/standard_sunau-3.13.0.tar.gz", hash = "sha256:b319a1ac95a09a2378a8442f403c66f4fd4b36616d6df6ae82b8e536ee790908", size = 9368, upload-time = "2024-10-30T16:01:41.626Z" } wheels = [ @@ -2791,6 +2808,7 @@ name = "stemdeck" source = { editable = "." } dependencies = [ { name = "all-in-one-infer" }, + { name = "asteroid-filterbanks" }, { name = "audio-separator", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, { name = "audioread", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, { name = "beat-this" }, @@ -2825,6 +2843,7 @@ dev = [ [package.metadata] requires-dist = [ { name = "all-in-one-infer", specifier = ">=3.1,<4" }, + { name = "asteroid-filterbanks", specifier = ">=0.4" }, { name = "audio-separator", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'", specifier = ">=0.24" }, { name = "audioread", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'", specifier = ">=3.0" }, { name = "beat-this", specifier = ">=1.1" }, @@ -2932,13 +2951,13 @@ resolution-markers = [ "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'", ] dependencies = [ - { name = "filelock" }, - { name = "fsspec" }, - { name = "jinja2" }, - { name = "networkx", version = "3.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "networkx", version = "3.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "sympy" }, - { name = "typing-extensions" }, + { name = "filelock", marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "fsspec", marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "jinja2", marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "networkx", version = "3.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "networkx", version = "3.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "sympy", marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, + { name = "typing-extensions", marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/3b/55/7192974ab13e5e5577f45d14ce70d42f5a9a686b4f57bbe8c9ab45c4a61a/torch-2.2.2-cp310-none-macosx_10_9_x86_64.whl", hash = "sha256:b2e2200b245bd9f263a0d41b6a2dab69c4aca635a01b30cca78064b0ef5b109e", size = 150788930, upload-time = "2024-03-27T21:08:09.98Z" }, @@ -2957,11 +2976,11 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "filelock" }, - { name = "fsspec" }, - { name = "jinja2" }, - { name = "networkx", version = "3.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "networkx", version = "3.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "filelock", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "fsspec", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "jinja2", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "networkx", version = "3.4.2", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "networkx", version = "3.6.1", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, { name = "nvidia-cublas-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, { name = "nvidia-cuda-cupti-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, { name = "nvidia-cuda-nvrtc-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, @@ -2975,10 +2994,10 @@ dependencies = [ { name = "nvidia-nccl-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, { name = "nvidia-nvjitlink-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, { name = "nvidia-nvtx-cu12", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, - { name = "setuptools", marker = "python_full_version >= '3.12'" }, - { name = "sympy" }, + { name = "setuptools", marker = "(python_full_version >= '3.12' and platform_machine != 'x86_64') or (python_full_version >= '3.12' and sys_platform != 'darwin')" }, + { name = "sympy", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, { name = "triton", marker = "platform_machine == 'x86_64' and sys_platform == 'linux'" }, - { name = "typing-extensions" }, + { name = "typing-extensions", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/37/81/aa9ab58ec10264c1abe62c8b73f5086c3c558885d6beecebf699f0dbeaeb/torch-2.6.0-cp310-cp310-manylinux1_x86_64.whl", hash = "sha256:6860df13d9911ac158f4c44031609700e1eba07916fff62e21e6ffa0a9e01961", size = 766685561, upload-time = "2025-01-29T16:19:12.12Z" }, @@ -3009,7 +3028,7 @@ resolution-markers = [ "python_full_version < '3.11' and platform_machine == 'x86_64' and sys_platform == 'darwin'", ] dependencies = [ - { name = "torch", version = "2.2.2", source = { registry = "https://pypi.org/simple" } }, + { name = "torch", version = "2.2.2", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine == 'x86_64' and sys_platform == 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/76/70/ca793994d37815070f6b53932b71822f66cfb3e197e6937426815998221e/torchaudio-2.2.2-cp310-cp310-macosx_10_13_x86_64.whl", hash = "sha256:b1d58201d108e85db3e35b84319f33884f61f327c38ead86913218c8c1acc3dd", size = 3398751, upload-time = "2024-03-27T21:12:31.998Z" }, @@ -3028,7 +3047,7 @@ resolution-markers = [ "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')", ] dependencies = [ - { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" } }, + { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/38/aa/f634960ac094e3fc6869f5c214ccfa6f74da2b1a89cefac024f6c650a717/torchaudio-2.6.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:0eda1cd876f44fc014dc04aa680db2fa355a83df5d834398db6dd5f5cd911f4c", size = 1808471, upload-time = "2025-01-29T16:29:43.783Z" }, @@ -3054,10 +3073,10 @@ name = "torchvision" version = "0.21.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, - { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "pillow" }, - { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" } }, + { name = "numpy", version = "2.2.6", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.11' and platform_machine != 'x86_64') or (python_full_version < '3.11' and sys_platform != 'darwin')" }, + { name = "numpy", version = "2.4.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version >= '3.11' and platform_machine != 'x86_64') or (python_full_version >= '3.11' and sys_platform != 'darwin')" }, + { name = "pillow", marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, + { name = "torch", version = "2.6.0", source = { registry = "https://pypi.org/simple" }, marker = "platform_machine != 'x86_64' or sys_platform != 'darwin'" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/a9/20/72eb0b5b08fa293f20fc41c374e37cf899f0033076f0144d2cdc48f9faee/torchvision-0.21.0-1-cp310-cp310-manylinux_2_28_aarch64.whl", hash = "sha256:5568c5a1ff1b2ec33127b629403adb530fab81378d9018ca4ed6508293f76e2b", size = 2327643, upload-time = "2025-03-18T17:25:51.165Z" },