Skip to content

Revise ppa dataset prep script to chunk images across multiple tar files - #298

Merged
rlskoeser merged 7 commits into
developfrom
feature/ppa-dataset-imagechunking
Oct 2, 2026
Merged

rlskoeser merged 7 commits into
developfrom
feature/ppa-dataset-imagechunking

Conversation

@rlskoeser

@rlskoeser rlskoeser commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Associated Issue(s): resolves #297

Changes in this PR

  • Now creates multiple image tar files, chunked based on first letter of work id
  • Stores combination of tar file name and image path in the pages jsonl

@rlskoeser rlskoeser changed the title Revise ppa dataset prep script to hcunk images across multiple tar files Revise ppa dataset prep script to chunk images across multiple tar files Sep 29, 2026
@codecov

codecov Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.03960% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.71%. Comparing base (a826111) to head (8a53622).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #298      +/-   ##
===========================================
+ Coverage    85.55%   85.71%   +0.16%     
===========================================
  Files           25       25              
  Lines         2672     2730      +58     
===========================================
+ Hits          2286     2340      +54     
- Misses         386      390       +4     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rlskoeser
rlskoeser merged commit a0ee0ea into develop Oct 2, 2026
9 checks passed
@rlskoeser
rlskoeser deleted the feature/ppa-dataset-imagechunking branch October 2, 2026 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant