spool: handle startup backlog and resync after overflow - #5
Conversation
- seed existing spool files into stability tracking so recent pre-start files are processed - avoid blocking on large startup backlogs by enqueueing stably or tracking for later delivery - rescan the spool when fsnotify reports ErrEventOverflow to recover missed files - add tests for startup-recent files, backlog handling, and overflow resync Tests: GOCACHE=/tmp/go-build-cache go test ./internal/spool
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Check if we're at max capacity | ||
| if len(fileStability) >= w.maxPendingFiles { | ||
| log.Printf("Warning: max pending files reached (%d), dropping oldest", w.maxPendingFiles) | ||
| // Remove oldest entry |
There was a problem hiding this comment.
Because seedExistingFiles now funnels pre-existing spool files through trackFile, this branch will drop the oldest entries once the backlog exceeds maxPendingFiles. If Santa restarts with more files in spool/new than the channel buffer plus maxPendingFiles (e.g., >1100 with defaults) or a resync sees a large overflow backlog, those extra files are removed from tracking and never delivered, whereas the previous implementation would block but eventually process all of them. That silently loses collected events during large startup/overflow backlogs.
Useful? React with 👍 / 👎.
1 participant
Tests: GOCACHE=/tmp/go-build-cache go test ./internal/spool