Skip to content

Fix invalid escape sequences in text normalization and models - #3709

Merged
LauraGPT merged 1 commit into
modelscope:mainfrom
MBK-fr:fix/invalid-escape-sequence-syntaxwarning
Sep 28, 2026
Merged

LauraGPT merged 1 commit into
modelscope:mainfrom
MBK-fr:fix/invalid-escape-sequence-syntaxwarning

Conversation

@MBK-fr

@MBK-fr MBK-fr commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Problem

Three first-party modules use invalid escape sequences in non-raw string literals:

  • fun_text_processing/text_normalization/normalize.py - \., \w
  • funasr/models/fsmn_kws/encoder.py - \[
  • funasr/models/fun_asr_nano/tools/format5res.py - \.

On Python 3.12+ each raises SyntaxWarning: invalid escape sequence on import.

Why it matters for FunASR

FunASR declares requires-python >=3.7.0 and runs the 3.10-3.12 CI matrix, so the warnings surface on the 3.12 job.

Fix

Escape the backslashes; every string value (and the regexes built from them) is byte-for-byte identical (ruff W605-style). (An identical copy of format5res.py under examples/ was left untouched to keep this scoped to the package source.)

Verification

In-memory compile() under -W error is clean on all three files after (raised before); each module's AST is unchanged.


🤖 Generated with Claude Code

https://claude.ai/code/session_014HBQNzAf5C2E3MiJC48HQw

Three modules use invalid escapes in non-raw string literals:
- fun_text_processing/text_normalization/normalize.py - \., \w
- funasr/models/fsmn_kws/encoder.py - \[
- funasr/models/fun_asr_nano/tools/format5res.py - \.

On Python 3.12+ each raises SyntaxWarning: invalid escape sequence on
import, and FunASR runs the 3.10-3.12 CI matrix. Escaping the
backslashes leaves every string value identical (ruff W605-style).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014HBQNzAf5C2E3MiJC48HQw

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I independently checked the exact three changed files at 7cae84872c088166b544d789de29632f2a7911dc against base 486b4b7ceb27b72db24a84c6e1be7cf6c5be6609, using Python 3.12.3.

With warnings treated as errors, in-memory compilation reproduces the invalid-escape failure in all three base files, while all three candidate files compile cleanly. Comparing complete module ASTs with location attributes excluded gives identical trees for each pair, including the interpolated normalization pattern and string constants. No behavioral discrepancy was found within this scope.

This validates the submitted package-source cleanup, not all repository copies or all supported Python versions. As noted in the PR description, the separate example copy of format5res.py is unchanged. I did not import these modules, load model weights, or run inference/full runtime tests. This is a bounded verification update, not a merge or full-runtime approval.

@LauraGPT LauraGPT left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head 7cae84872c088166b544d789de29632f2a7911dc against current main 2d4566d4a4c84d73e1f828efa2dedfa88f33677f.

Python 3.12 warning-as-error compilation reproduces the invalid-escape failure in all three current-main files, while the PR head and current-main synthetic merge compile cleanly. Full ASTs excluding source locations are identical for base versus head and head versus merge in all three files. compileall and git diff --check pass. In an isolated dependency environment, the same focused text-normalization and KWS set passes 20/20 on current main and 20/20 on the synthetic merge.

The first system-environment pytest attempt is retained as non-green evidence: two KWS files failed collection because rapidfuzz was absent and NumPy 2.3.5 was incompatible with the installed SciPy/sklearn binaries. That baseline environment failure is not represented as code success.

No blocking code issue remains in this scope. Approved for an ordinary exact-head squash merge.

@LauraGPT
LauraGPT merged commit da12182 into modelscope:main Sep 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants