Skip to content

Add FunASR recognizer (local multilingual ASR: SenseVoice/Paraformer)#903

Open
LauraGPT wants to merge 5 commits into
Uberi:masterfrom
LauraGPT:add-funasr-recognizer
Open

Add FunASR recognizer (local multilingual ASR: SenseVoice/Paraformer)#903
LauraGPT wants to merge 5 commits into
Uberi:masterfrom
LauraGPT:add-funasr-recognizer

Conversation

@LauraGPT

Copy link
Copy Markdown

Adds a FunASR recognizer: Recognizer.recognize_funasr(audio_data, model="iic/SenseVoiceSmall", ...).

FunASR is an open-source speech toolkit (SenseVoice, Paraformer, Fun-ASR-Nano) that runs fully locally, no API key, with strong multilingual accuracy — particularly Chinese, Cantonese, Japanese, Korean. SenseVoice (the default) is multilingual with auto language detection.

Implementation mirrors the existing local recognizers (e.g. vosk): a recognize() in speech_recognition/recognizers/funasr.py that converts AudioData to 16 kHz/16-bit PCM via get_raw_data(convert_rate=16000, convert_width=2), runs the model, and returns cleaned text. The model is cached across calls; funasr is an optional extra (pip install "SpeechRecognition[funasr]").

Verified: the AudioData.get_raw_data(16000, 2) -> int16 -> float32 conversion was tested on a 16 kHz sample, and the FunASR/SenseVoice transcription core (PCM -> text) was verified end-to-end (produces the expected transcript with tags stripped).

import speech_recognition as sr
r = sr.Recognizer()
with sr.AudioFile("chinese.wav") as src:
    audio = r.record(src)
print(r.recognize_funasr(audio))  # local, multilingual

@LauraGPT

Copy link
Copy Markdown
Author

Added lightweight FunASR recognizer tests in e38853a.

What the tests cover without downloading a real model:

  • missing optional funasr dependency raises SetupError;
  • AudioData.get_raw_data(convert_rate=16000, convert_width=2) is used before converting PCM to float32 samples;
  • AutoModel is cached by (model, device) across calls;
  • unknown languages fall back to FunASR/SenseVoice auto language detection;
  • Recognizer.recognize_funasr is wired to the FunASR recognizer.

Local verification:

python -m pytest tests/recognizers/test_funasr.py -q
# 4 passed, 2 warnings in 0.09s

python -m pytest tests/test_recognition.py tests/recognizers/test_funasr.py -q
# 5 passed, 6 skipped, 2 warnings in 0.07s

python -m py_compile speech_recognition/recognizers/funasr.py tests/recognizers/test_funasr.py speech_recognition/__init__.py
git diff --check

The warnings are the existing Python 3.12 aifc / audioop deprecation warnings from package import.

@LauraGPT

LauraGPT commented Jul 7, 2026

Copy link
Copy Markdown
Author

CI status note: the current unstable state is not a test failure from this diff. The four GitHub Actions runs on e38853a7 (Type check, Static analysis, Ensure RST is well-formed, and Unit tests) are all in action_required, which means the fork PR workflows are waiting for maintainer approval to run.

Local validation already covered the new recognizer path without downloading a real FunASR model, as noted above. Once the workflows are approved, I can follow up on any real failures if they appear.

@LauraGPT

LauraGPT commented Jul 7, 2026

Copy link
Copy Markdown
Author

Added a small docs/lint follow-up in 2dcc004.

Changes:

  • list FunASR in the README engine support list;
  • add the funasr optional dependency / SpeechRecognition[funasr] install note;
  • add recognize_funasr(...) to reference/library-reference.rst;
  • remove an unused patch import from tests/recognizers/test_funasr.py that flake8 would flag.

Verification run locally on the PR branch:

python3 -m pytest tests/recognizers/test_funasr.py -q
# 4 passed, 2 warnings

python3 -m pytest tests/test_recognition.py tests/recognizers/test_funasr.py -q
# 5 passed, 6 skipped, 2 warnings

python3 -m py_compile speech_recognition/recognizers/funasr.py tests/recognizers/test_funasr.py speech_recognition/__init__.py

rstcheck README.rst CONTRIBUTING.rst
# Success! No issues detected.
rstcheck --ignore-directives autofunction reference/*.rst
# Success! No issues detected.

flake8 --ignore=E501,E701,W503 --extend-exclude .venv,venv,build --doctests speech_recognition/recognizers/funasr.py tests/recognizers/test_funasr.py speech_recognition/__init__.py
# passed

Note: this PR's GitHub Actions are still waiting for maintainer approval to run on the fork, so I cannot inspect CI results yet. Once approved, I can follow up on any real failures.

@LauraGPT

LauraGPT commented Jul 7, 2026

Copy link
Copy Markdown
Author

Fresh FunASR-side status recheck (2026-07-07 UTC):

  • PR is open, non-draft, and mergeable at head 2dcc004a; mergeable_state=unstable is not from a failing check log.
  • GitHub reports 4 check suites on this head, all completed/action_required, with 0 check-runs/logs. This is the fork-workflow approval gate for Type check, Static analysis, Ensure RST is well-formed, and Unit tests, not a diff failure.
  • Thread-aware review audit shows 0 current unresolved review threads.
  • Fresh local validation on the PR branch:
    • git diff --check -- . ':(exclude)README.rst' passed. (README.rst is excluded here because upstream carries CRLF/trailing-space-sensitive lines; the project CI validates RST separately.)
    • python3 -m py_compile speech_recognition/recognizers/funasr.py tests/recognizers/test_funasr.py passed.
    • python3 -m pytest tests/recognizers/test_funasr.py -q -> 4 passed.
    • README, pyproject.toml optional extra, and library reference entries for FunASR / recognize_funasr are present.

Remaining action is maintainer approval of the fork workflows so the repo CI can run on GitHub.

@LauraGPT

Copy link
Copy Markdown
Author

@ftnext, could you approve the four fork workflow suites and review this local FunASR recognizer when convenient?

Fresh verification on exact head 2dcc004a against current master:

  • branch is clean, mergeable, and 0 commits behind;
  • isolated Python 3.12 .[dev] environment: full pytest 36 passed, 24 skipped;
  • focused FunASR tests: 4 passed;
  • mypy: no issues in 32 source files;
  • flake8, both RST checks, Python compilation, and the CRLF-aware diff check pass;
  • real CPU smoke with FunASR 1.3.14 and official iic/SenseVoiceSmall exercised AudioFile -> AudioData -> Recognizer.recognize_funasr, returned 开放时间早上9点至下午5点。, stripped rich tags, and populated exactly one model-cache entry.

The four GitHub check suites still have action_required with zero check runs, so there is no CI log or code failure for me to address until a maintainer approves them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants