fix(header-serde): change train() to return Result and skip subdirectories#929
Open
kushdab wants to merge 2 commits into
Open
fix(header-serde): change train() to return Result and skip subdirectories#929kushdab wants to merge 2 commits into
kushdab wants to merge 2 commits into
Conversation
- Filters out subdirectories from the training set (was the first TODO) - Changes the return type from Vec<u8> to Result<Vec<u8>> and replaces the two .unwrap() calls with .explain_err() / ? (second TODO) - Updates the existing test helper to use .expect() and adds a new test that asserts train() returns Ok on a known-good directory Closes cloudflare#925
Now that dict::train() returns Result<Vec<u8>>, change main() to return Result<(), Box<dyn std::error::Error>> so the error propagates to the OS exit code instead of being silently ignored.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #925.
The
dict::train()function had two related TODOs:Skip subdirectories
fs::read_diryields both files and directories. Passing a directory path tozstd::dict::from_filescauses an IO error at runtime. The fix adds anis_file()guard so only regular files are included in the training set.Return
Resultinstead of panicking the two.unwrap()calls (one onread_dir, one ondict::from_files) caused a silent process-kill on any bad path or training failure. The return type is nowResult<Vec<u8>>using the crate's existingpingora_errormachinery (OrErr::explain_err).Changes
pingora-header-serde/src/dict.rsVec<u8>→Result<Vec<u8>>fs::read_dir(...).unwrap()→.explain_err(InternalError, ...)?.is_file()filter in thefilter_mapclosure to skip subdirectoriesdict::from_files(...).unwrap()→.explain_err(InternalError, ...)train(path)totrain(path).expect(...)test_train_skips_subdirectoriesregression testpingora-header-serde/src/trainer.rsmain()now returnsResult<(), Box<dyn std::error::Error>>and propagates errors with?, so the binary exits with a non-zero code and a readable message instead of panicking silently