[miniflare] Add backend resources for email capture and storage - #15064
[miniflare] Add backend resources for email capture and storage#15064tpmmorris wants to merge 13 commits into
Conversation
🦋 Changeset detectedLatest commit: e5d3aaf The changes in this PR will be included in the next version bump. This PR includes changesets to release 8 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
Codeowners approval required for this PR:
Show detailed file reviewers
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| // Decode MIME "encoded-word" headers (e.g. `=?utf-8?B?...?=`) in each reply's | ||
| // display text so the explorer shows readable subjects. | ||
| const decoded = { | ||
| ...email, | ||
| replies: email.replies.map((reply) => ({ | ||
| ...reply, | ||
| raw: decodeWords(reply.raw), | ||
| })), | ||
| }; |
There was a problem hiding this comment.
🟡 Reply message shown in the email inspector can be altered from what was actually sent
The full raw text of each reply is run through a header-decoding step (decodeWords(reply.raw) at packages/miniflare/src/workers/local-explorer/resources/email.ts:328) before being returned, so the reply a developer inspects can differ from the message the worker actually produced.
Impact: A reply whose body happens to contain encoded-word-looking text is displayed altered, and the accompanying base64 copy of the same reply no longer matches what is shown.
Why decoding the whole MIME blob is wrong here
decodeWords from postal-mime decodes RFC 2047 =?charset?enc?...?= sequences anywhere in the string it is given. Here it is applied to reply.raw, which is the complete MIME message (headers and body), not just a header value. Any literal encoded-word sequence appearing in the body — e.g. a quoted example, a forwarded header, or test fixture text — will be rewritten.
The API contract also becomes self-inconsistent: email_handler-reply.raw is documented as "Raw MIME content of the reply" and rawBase64 as the "Lossless base64 representation of the reply MIME" (see packages/miniflare/src/workers/local-explorer/openapi.local.json), but only raw is transformed here — rawBase64 is passed through untouched from the stored record, so the two fields can disagree for the same reply.
If the goal is readable subjects in the UI, the decoding should be applied to individual parsed header values for display, leaving raw byte-faithful.
Was this helpful? React with 👍 or 👎 to provide feedback.
| rawBase64: bytesToBase64(rawEmailBuffer), | ||
| }); | ||
|
|
||
| this.ctx.waitUntil( |
There was a problem hiding this comment.
🟡 Emails sent just before the dev session shuts down are never written to disk or logged
The on-disk copy of a sent email and its log line are queued to run in the background (this.ctx.waitUntil(...) at packages/miniflare/src/workers/email/send_email.worker.ts:370 and :451) instead of being finished before the send call returns, so a script that sends an email and then immediately shuts the local dev session down loses the saved message entirely.
Impact: Short-lived usages (for example sending through getPlatformProxy() and then disposing) no longer reliably produce the .eml/text/HTML/attachment files or the "send_email binding called..." log they used to.
Why the deferred work can be dropped
Before this change send() awaited every storeTempFile() call and logged before resolving, so by the time the caller's await env.SEND_EMAIL.send(...) returned the files existed. Now both branches resolve immediately after the in-workerd capture, deferring the loopback /core/store-temp-file writes and logging to ctx.waitUntil.
Miniflare#dispose() aborts, stops the loopback server and tears down workerd before drainEmailArtifactManager() runs (packages/miniflare/src/index.ts:3490-3496); drain() only awaits operations that already reached the Node side (packages/miniflare/src/plugins/email/artifacts.ts:86-89), so waitUntil work that has not yet issued its loopback request is simply discarded.
The test updates in this PR reflect the new asynchrony (the miniflare email specs now poll with vi.waitFor, and the get-platform-proxy e2e no longer asserts on the file), but callers that dispose right after sending have no way to wait.
Was this helpful? React with 👍 or 👎 to provide feedback.
@cloudflare/autoconfig
@cloudflare/build-output-utils
@cloudflare/config
create-cloudflare
@cloudflare/deploy-helpers
@cloudflare/kv-asset-handler
miniflare
@cloudflare/pages-functions
@cloudflare/pages-shared
@cloudflare/unenv-preset
@cloudflare/vite-plugin
@cloudflare/vitest-pool-workers
@cloudflare/workers-auth
@cloudflare/workers-editor-shared
@cloudflare/workers-utils
wrangler
commit: |
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
7fccbb7 to
494e214
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…torage and api exposure.
| throw new TypeError("Sent email record does not match its table"); | ||
| } | ||
| const sentEmail = email; | ||
| const artifacts: EmailArtifact[] = []; |
There was a problem hiding this comment.
🟡 Saved copies of large outgoing emails are never cleaned up and unrelated files are deleted instead
When an old sent email is dropped from the local history, the wrong on-disk copies are identified for deletion (sentEmail.raw !== undefined at packages/miniflare/src/workers/email/email-store.ts:241) for any message that was recorded in chunks, so its saved .eml copy is left behind forever.
Impact: Temporary email files for larger outgoing messages accumulate in the project/system temp folders for the whole dev session instead of being cleaned up.
Why chunked sent records never match the raw-file branch
reportSentEmail in packages/miniflare/src/workers/email/send_email.worker.ts:163-181 streams any message whose base64 body exceeds 64 KB (i.e. raw body > ~48 KB). That path destructures raw and rawBase64 out of the record before calling store.beginSent(metadata), and finishSent re-adds only rawBase64 (packages/miniflare/src/workers/email/email-store.ts:485-495). The persisted record therefore has raw === undefined.
On eviction, #insert re-parses the evicted row and calls getArtifacts("sent", ...). Because raw is missing, it takes the else branch and reports email-text/email-html/email-attachment artifacts (paths that were never written for an EmailMessage send) instead of the email/<id>.eml artifact that send() actually wrote via storeTempFile(..., "email", id, id) (packages/miniflare/src/workers/email/send_email.worker.ts:430-438). The unlink of the non-existent paths is ignored, and the real .eml is leaked.
| const artifacts: EmailArtifact[] = []; | |
| if (sentEmail.raw !== undefined || sentEmail.rawBase64 !== undefined) { |
Was this helpful? React with 👍 or 👎 to provide feedback.
| const headers: string[] = [`From: ${body.from}`, `To: ${body.to.join(", ")}`]; | ||
| if (body.cc?.length) { | ||
| headers.push(`Cc: ${body.cc.join(", ")}`); |
There was a problem hiding this comment.
🟡 Blind-copy recipients on a test email are silently dropped
Blind-copy addresses supplied when sending a test email are never written into the composed message (only Cc is added at packages/miniflare/src/workers/local-explorer/resources/email.ts:198-200), so they silently disappear even though the API documents that they will appear.
Impact: Developers testing an email handler that inspects blind-copy recipients see none, with no error or warning.
Contract mismatch between the documented endpoint and the composed MIME
The /email/routing/send OpenAPI description states: "Only the first to address is used as the envelope recipient; any other to/cc/bcc addresses appear only in the composed MIME headers" (packages/miniflare/src/workers/local-explorer/openapi.local.json, email-send-routing). buildMimeMessage emits From, To, Cc, Reply-To, Subject, Message-ID, Date and custom headers, but never a Bcc header. body.bcc is only read for control-character validation (packages/miniflare/src/workers/local-explorer/resources/email.ts:168).
| const headers: string[] = [`From: ${body.from}`, `To: ${body.to.join(", ")}`]; | |
| if (body.cc?.length) { | |
| headers.push(`Cc: ${body.cc.join(", ")}`); | |
| if (body.cc?.length) { | |
| headers.push(`Cc: ${body.cc.join(", ")}`); | |
| } | |
| if (body.bcc?.length) { | |
| headers.push(`Bcc: ${body.bcc.join(", ")}`); | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
2 participants
Fixes #13648
Add backend resources for Email interaction within Local Explorer
Adds storage and capture methods for emails sent from/received by a worker using durable objects for storage (in line with the new 'Observability' tab), and cdn-cgi endpoints to mimic the sending of an email to a worker. Also records actions taken by the
email()handler (received,forwarded,replied,rejected,unhandled), so that they can be mapped and displayed in local explorer in a similar manner as the dash. The Email result interface has been updated to also include a list of events.