fix putObject never resolves when the input stream gets destroyed o… - #1480
Open
yucao2521 wants to merge 1 commit into
Open
fix putObject never resolves when the input stream gets destroyed o…#1480yucao2521 wants to merge 1 commit into
putObject never resolves when the input stream gets destroyed o…#1480yucao2521 wants to merge 1 commit into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR fixes a critical issue where ChangesStream Error Handling for putObject
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/internal/client.ts`:
- Around line 1802-1849: The async uploader awaits the pipeline and the consumer
separately which can deadlock if makeRequestAsyncOmit rejects inside the
consumer (o); change the try block to await both the pipeline and the consumer
together (e.g., await Promise.all([streamPromise.pipeline(body, chunkier), o]))
so the pipeline and the async iterator (o) are observed as one unit and ensure
abortMultipartUpload is still called on any rejection from either side; keep
references to chunkier, o, makeRequestAsyncOmit, streamPromise.pipeline and
abortMultipartUpload when making the change.
- Around line 1808-1840: The bug is that partNumber is incremented before
uploading the current chunk, causing chunk N to be uploaded as part N+1; fix by
keeping the current chunk on the current partNumber: do not increment partNumber
until after you have either skipped a matching oldPart (the existing branch
where you compare oldPart.etag === md5.toString('hex') should increment and
continue) or after a successful upload and pushing to eTags; specifically,
remove the premature partNumber++ that appears immediately before the upload
RequestOption, and instead increment partNumber only after eTags.push({ part:
partNumber, etag }) (and keep the existing increment where you skip matched
oldPart).
In `@tests/unit/test.js`:
- Around line 670-674: The test creates a Readable stream `s` and calls
`s.destroy(new Error('stream error'))` without an 'error' listener which can
cause an unhandled exception; before calling `s.destroy(...)` add an empty error
handler like `s.on('error', () => {})` (i.e., update the test around the
`it('should fail when stream is destroyed with an error', ...)` block so the
`Stream.Readable` instance `s` has `s.on('error', () => {})` attached prior to
calling `client.putObject('bucket', 'object', s)` and `s.destroy(...)`).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: cbae5fd8-c249-4ba8-910e-9df28e9a463a
📒 Files selected for processing (4)
src/internal/client.tssrc/internal/helper.tstests/functional/functional-tests.jstests/unit/test.js
yucao2521
force-pushed
the
fix/put-stream
branch
from
June 14, 2026 08:23
9f01f3e to
55172c4
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/internal/client.ts`:
- Around line 1688-1692: The code currently only checks for destroy errors via
getReadableStreamError(stream) but accepts streams that have already been
consumed without error. Since getReadableStreamError() returns undefined for
both "still open" and "finished cleanly" states, you need to add an additional
check after the error probe to reject exhausted streams. After checking if an
error exists from getReadableStreamError(), also verify that the stream is not
already finished or exhausted. If the stream is in a finished/exhausted state
(meaning it was already consumed and returned no error), throw an error to
prevent uploading empty data when the caller provided a positive size. This
ensures that only viable streams proceed past this validation point.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: e4f2da02-ff3a-4313-a8fe-d8fdc0d8fb5a
📒 Files selected for processing (4)
src/internal/client.tssrc/internal/helper.tstests/functional/functional-tests.jstests/unit/test.js
🚧 Files skipped from review as they are similar to previous changes (2)
- tests/unit/test.js
- tests/functional/functional-tests.js
Member
|
could you address the comments @yucao2521 ? |
yucao2521
force-pushed
the
fix/put-stream
branch
from
June 21, 2026 09:21
55172c4 to
43646fc
Compare
Author
|
Hi @prakashsvmx , I've resolved the comments from CodeRabbit. Please have a review. Thanks. BR, |
Member
|
Thank you, |
yucao2521
force-pushed
the
fix/put-stream
branch
2 times, most recently
from
June 22, 2026 18:49
56db184 to
874ce60
Compare
yucao2521
force-pushed
the
fix/put-stream
branch
from
June 22, 2026 19:04
874ce60 to
380526f
Compare
Author
|
Ah, sorry. It's fixed now, please check again. Thanks, @prakashsvmx . |
There was a problem hiding this comment.
Pull request overview
Fixes an issue where putObject() could remain unsettled when an input Readable stream is destroyed, by improving stream error detection and multipart upload stream wiring, and updates tests to assert the new rejection behavior.
Changes:
- Add readable stream error introspection (
getReadableStreamError) and use it inputObject()to fail fast on destroyed/non-readable streams. - Refactor multipart upload stream handling to use
pipeline()and abort multipart uploads on failure. - Update unit/functional tests to assert specific rejection messages for destroyed streams (including multipart scenarios).
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
src/internal/helper.ts |
Adds getReadableStreamError() and relaxes isReadableStream() detection to allow destroyed streams to be handled via finished(). |
src/internal/client.ts |
Uses getReadableStreamError() in putObject(); refactors multipart upload flow to use pipeline() and abort on error. |
tests/unit/test.js |
Updates/extends unit tests to expect “Premature close” or the explicit destroy error message. |
tests/functional/functional-tests.js |
Converts destroyed-stream functional steps to promise-based rejection assertions and expands coverage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
38
to
40
| const assert = chai.assert | ||
| const expect = chai.expect | ||
|
|
Comment on lines
+1811
to
+1818
| const oldPart = oldParts[partNumber] | ||
| if (oldPart) { | ||
| if (oldPart.etag === md5.toString('hex')) { | ||
| eTags.push({ part: partNumber, etag: oldPart.etag }) | ||
| partNumber++ | ||
| continue | ||
| } | ||
| } |
Comment on lines
+1845
to
+1850
| })().catch((err) => { | ||
| if (!chunkier.closed) { | ||
| chunkier.destroy(err) | ||
| } | ||
| throw err | ||
| }) |
Comment on lines
+1854
to
+1857
| } catch (err) { | ||
| await this.abortMultipartUpload(bucketName, objectName, uploadId) | ||
| throw err | ||
| } |
Member
|
could you please check and validate the review comments @yucao2521 ? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes: #1479
Summary by CodeRabbit
Bug Fixes
Tests
Refactor