Skip to content

Import-DbaParquet - Release the parquet file when the import is done - #10543

Merged
potatoqualitee merged 2 commits into
developmentfrom
fix-parquet-file-handle
Aug 10, 2026
Merged

Import-DbaParquet - Release the parquet file when the import is done#10543
potatoqualitee merged 2 commits into
developmentfrom
fix-parquet-file-handle

Conversation

@andreasjordan

Copy link
Copy Markdown
Collaborator

Fixes #10542.

Import-DbaParquet kept the imported file locked after the import had finished, until the garbage collector
got around to finalizing the stream. How long that took varied from run to run, so a file could not reliably
be deleted, moved or overwritten right after importing it.

Cause

Get-ParquetReader opens a file stream and passes it to Parquet.NET without saying what should happen to it:

$reader = [Parquet.ParquetReader]::CreateAsync($stream).GetAwaiter().GetResult()

The overload in play is

CreateAsync(Stream input, ParquetOptions parquetOptions = null, Boolean leaveStreamOpen = True, CancellationToken cancellationToken)

leaveStreamOpen defaults to true, so disposing the reader deliberately leaves the stream open. The
command does dispose the reader, but nothing disposes the stream, so the handle lived until finalization.

leaveStreamOpen is now passed as false. All four arguments are passed explicitly because the stream
overload can only be selected by its full signature - with fewer arguments the string filePath overload is a
candidate too.

Testing

The symptom that made this visible was a test that could not clean up after itself: on PowerShell 7 the Pester
run failed in Remove-TestDrive with The process cannot access the file 'staging.ecdc_parquet_test.parquet' because it is being used by another process, while all 17 tests passed. It failed about every other run,
which is why it read as test flakiness rather than a defect.

before after
Import-DbaParquet.Tests.ps1 on PowerShell 7.4, five consecutive runs failed ~1 in 2 5 of 5 clean
Import-DbaParquet.Tests.ps1 on Windows PowerShell 5.1 17 of 17 17 of 17

No test change was needed - the test was right and the command was wrong.

Note

This touches the same file as #10538 and #10541 but a different part of it. All three merge cleanly, verified
by merging them locally and running the tests above against the combination.

馃 Generated with Claude Code

andreasjordan and others added 2 commits August 8, 2026 19:27
Get-ParquetReader hands a file stream to ParquetReader.CreateAsync without passing
leaveStreamOpen, and that parameter defaults to true. Disposing the reader therefore left the
stream open and the imported file locked until the garbage collector got around to finalizing
it, which is an arbitrary amount of time after the import finished.

The visible symptom was a test that could not clean up after itself: on PowerShell 7 the Pester
run of Import-DbaParquet failed roughly every other time in Remove-TestDrive with "The process
cannot access the file staging.ecdc_parquet_test.parquet because it is being used by another
process", while all 17 tests passed. For users the effect is worse than a failing cleanup,
because a file cannot be deleted, moved or overwritten after importing it until a collection
happens to run.

leaveStreamOpen is now passed as false, so disposing the reader closes the file.

(do Import-DbaParquet)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@potatoqualitee
potatoqualitee merged commit de4339c into development Aug 10, 2026
31 of 32 checks passed
@potatoqualitee
potatoqualitee deleted the fix-parquet-file-handle branch August 10, 2026 04:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Import-DbaParquet keeps the imported file locked until the garbage collector runs

2 participants