Repository navigation
Conversation
Decode each base64 chunk straight into the remote file stream through a CryptoStream instead of allocating a per-chunk byte array with [Convert]::FromBase64String. Each array landed on the Large Object Heap and piled up between Gen2 collections, exhausting the remote host's memory (#55). Also force a garbage collection every 50 chunks so the remaining per-chunk LOH garbage (unique script text and ScriptBlocks) cannot accumulate over a long upload. Disposing the StreamWriter flushes the final base64 block and closes the whole stream chain.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #55.
Each upload chunk allocated two large objects on the remote host: the
base64 script text and the byte array from
[Convert]::FromBase64String.Both land on the Large Object Heap. The heap only drains on a Gen2
collection, which may not run during a long upload. A 200MB file can
therefore exhaust the remote PowerShell host's memory.
This change makes two fixes.
CryptoStream. No per-chunk byte array is allocated.[System.GC]::Collect()every 50 chunks. This bounds theremaining per-chunk garbage (script text and compiled script blocks).
The change keeps the existing chunked upload and the script block cache
clear. It drops no behavior.
This needs a real large-file upload test against a Windows host before
merge.