Skip to content

Load data files from embedded resources by default - #4

Merged
rabbitism merged 2 commits into
mainfrom
feature/embedded-resources
Sep 7, 2026
Merged

rabbitism merged 2 commits into
mainfrom
feature/embedded-resources

Conversation

@rabbitism

Copy link
Copy Markdown
Member

概述

按之前讨论的方案,将 9 个数据文件(dict.txt、HMM/POS 模型、idf.txt、stopwords.txt 等)改为默认从程序集内嵌资源加载,自定义目录机制保留为覆盖手段。桌面用户零感知,Avalonia mobile(Android/iOS)等无文件系统布局的场景开箱即用。

变更

  • 默认内嵌加载:ConfigManager 新增 OpenResource(fileName) / ReadResourceText(fileName),默认通过 manifest resource stream 读取(无反射、NativeAOT 安全);HasCustomConfigFileDir 指示当前模式。
  • 自定义目录保留:显式设置 ConfigManager.ConfigFileBaseDir 或环境变量 JIEBA_CONFIG_FILE_DIR 时,全部数据文件改为从磁盘读取(即原有的自定义词典覆盖场景);ConfigFileBaseDir getter 语义不变(始终返回解析后的绝对路径)。
  • 改造的加载点:WordDictionary.LoadDict、FinalSeg/Viterbi、PosSeg/Viterbi、PosSegmenter.LoadWordTagTab、KeywordExtractor(新增 SetDefaultStopWords)、IdfLoader(新增内嵌加载构造器)。用户显式传路径的 API(LoadUserDict、SetStopWords(path)、SetNewPath、SetIdfPath)保持不变。
  • 打包:Segmenter.csproj 增加 EmbeddedResource(保留复制到输出目录,供自定义目录模式开箱使用)。
  • 测试:测试套件现在以内嵌模式运行(SetupTests 不再设置自定义目录);新增 TestConfigManager 覆盖两种模式的资源解析。
  • 文档:README 安装和配置章节改为「默认零配置 + 自定义目录覆盖」。

验证

  • 解决方案构建 0 警告;全量测试 124 个(新增 2 个):122 通过 / 2 跳过。
  • NativeAOT 冒烟:发布产物删除 Resources 目录后运行,六项功能全部 PASS——内嵌加载完全自包含。

The 9 dictionary/model files are embedded in JiebaNet.Segmenter.dll and
loaded via manifest resource streams, so no file system layout is
required (Android/iOS/WASM friendly, AOT safe). When a custom config
directory is set (ConfigManager.ConfigFileBaseDir or the
JIEBA_CONFIG_FILE_DIR env var), files are read from that directory
instead, keeping the existing customization story. The test suite now
runs in embedded mode, plus dedicated ConfigManager mode tests.
The embedded resources are the only default source now, so the loose
copies are no longer needed; tests that relied on them either read the
embedded resources directly or use hermetic temp directories.
@rabbitism
rabbitism merged commit ead9d43 into main Sep 7, 2026
5 checks passed
@rabbitism
rabbitism deleted the feature/embedded-resources branch September 7, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant