Skip to content

perf(filesystem): optimize ListDirectory allocations and syscalls - #222

Draft
codspeed-hq[bot] wants to merge 1 commit into
mainfrom
codspeed/optim-batch-allocate-structs-and-skip-redundant-os-lstat-1781211269869
Draft

perf(filesystem): optimize ListDirectory allocations and syscalls#222
codspeed-hq[bot] wants to merge 1 commit into
mainfrom
codspeed/optim-batch-allocate-structs-and-skip-redundant-os-lstat-1781211269869

Conversation

@codspeed-hq

@codspeed-hq codspeed-hq Bot commented Jun 11, 2026

Copy link
Copy Markdown

Summary

Optimizes Filesystem.ListDirectory — invoked on every directory-listing HTTP request — by eliminating redundant syscalls and reducing heap allocations.

Flamegraph analysis of BenchmarkListDirectory revealed three independent sources of overhead beyond user/group lookups:

1. Redundant os.Lstat calls for directory entries

os.ReadDir returns DirEntry values whose IsDir() already knows the file type from the getdents syscall — no extra stat is needed. The original code called os.Lstat on every entry, including directories. We now check entry.IsDir() first and skip os.Lstat entirely for directories. os.Lstat is still used for files/symlinks (where full stat info is required), and a fallback handles the rare DT_UNKNOWN case.

2. Per-entry heap allocations causing GC pressure

Each File/Subdirectory was individually heap-allocated (&File{...}) and appended. For N entries this created N separate heap objects. We now pre-allocate backing slices ([]File, []Subdirectory) sized to len(entries) and take pointers into them, reducing N allocations to 2 and lowering GC scan work.

3. fmt.Sprintf("%o", ...) for permission formatting

Replaced with strconv.FormatUint(uint64(info.Mode()), 8), avoiding reflection and intermediate buffer allocation.

Benchmark Results

Validated locally through the CodSpeed CLI (walltime):

Benchmark Base Head Change
BenchmarkListDirectory 102.6 µs 69.1 µs +48.47%

No regressions on other benchmarks. All existing filesystem and API tests pass.

Notes

Owner/group resolution for files is preserved via new lookupUsername/lookupGroupname helpers, which resolve UID/GID from the already-obtained syscall.Stat_t and fall back to the numeric ID string on lookup failure — matching the prior behavior.


Note

Optimizes ListDirectory by pre-allocating backing slices for File/Subdirectory structs, skipping os.Lstat for directory entries (using DirEntry.IsDir() from getdents), and replacing fmt.Sprintf with strconv.FormatUint for permission formatting.

Written by Mendral for commit ce9ce31.

@mendral-app mendral-app Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

The optimization is sound. Backing slice pointers are safe because fileBacking and subdirBacking are each capped at numEntries which is the total count, so neither can exceed capacity and trigger reallocation. The DT_UNKNOWN fallback is correctly handled. Permission formatting produces identical output.

Tag @mendral-app with feedback or questions. View session

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants