Skip to content

feat(node): expose the raw response bytes alongside the decoded text - #1

Open
onamfc wants to merge 1 commit into
mainfrom
feat/binary-body
Open

feat(node): expose the raw response bytes alongside the decoded text#1
onamfc wants to merge 1 commit into
mainfrom
feat/binary-body

Conversation

@onamfc

@onamfc onamfc commented Aug 25, 2026

Copy link
Copy Markdown
Member

The fetcher read every response through a TextDecoder and returned only the string. That is lossy for anything that is not UTF-8, so an image fetched through it could never be recovered — a PNG's leading 0x89 decodes to U+FFFD and the original is gone.

Why it matters now

LinkForty's generated social cards (SIT-368/369) composite a user-supplied logo or hero image into the card, which means fetching an arbitrary URL server-side — exactly the SSRF position this fetcher exists to make safe.

Without access to bytes, that caller had two options: decode-and-hope (broken), or stand up its own fetcher and reimplement the per-hop address checks. The second is worse than it sounds — it would mean two copies of the control that matters most, and the copy would inevitably drift from this one.

The change

readCapped now collects Uint8Array chunks and decodes once at the end:

function toBody(bytes: Uint8Array): { body: string; bodyBytes: Uint8Array } {
  return { body: new TextDecoder().decode(bytes), bodyBytes: bytes };
}

One read, one code path, both shapes. body is byte-for-byte unchanged for existing callers — og-core's parser neither knows nor cares.

bodyBytes is optional on FetchResult so a custom adapter that only ever handles text remains valid.

Verification

  • 72 tests, up from 69. Three new cases: raw PNG bytes survive intact and are provably unrecoverable from body; text bodies still decode correctly including multibyte characters; the byte cap applies to both.
  • pnpm build, pnpm typecheck, pnpm lint clean.

The PNG test asserts the point directly:

expect(Array.from(result.bodyBytes)).toEqual(Array.from(png));
// The decoded string has already lost the data, which is the whole point.
expect(new TextEncoder().encode(result.body).length).not.toBe(png.length);

Versions

@linkforty/og-core and @linkforty/og-node both to 0.2.0 — minor, since FetchResult gains an optional field and no existing behaviour changes.

Note

This is a design flaw I introduced when specifying SIT-369: I wrote the ticket saying to fetch card images with createNodeFetcher, without checking that it returns text. Fixing it here rather than duplicating the fetcher in Cloud is the correct resolution — the guarded fetcher should be usable for every server-side fetch we do, not just HTML.

The fetcher read every response through a TextDecoder and returned only
the string. That is lossy for anything that is not UTF-8, so an image
fetched through it could never be recovered — a PNG's leading 0x89 byte
decodes to U+FFFD and the original is gone.

That mattered the moment a second caller appeared. LinkForty's generated
social cards composite a user-supplied logo or hero image into the card,
which means fetching an arbitrary URL server-side: exactly the SSRF
position this fetcher exists to make safe. Without bytes, that caller
would have had to stand up its own fetcher and reimplement the per-hop
address checks — two copies of the control that matters most.

readCapped now collects Uint8Array chunks and decodes once at the end,
so `body` is unchanged for existing callers and `bodyBytes` carries the
original. One read, one code path, both shapes.

`bodyBytes` is optional on FetchResult so a custom adapter that only ever
handles text stays valid.

72 tests, up from 69. Build, typecheck and lint clean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant