Skip to content

fix(deploy): a reader of a resource replaced in the deploy is sent, not skipped, when a secret rewrote its values (#4838) - #4851

Merged
go-to-k merged 5 commits into
mainfrom
fix/4838-rotated-secret-reader-generation
Oct 13, 2026
Merged

go-to-k merged 5 commits into
mainfrom
fix/4838-rotated-secret-reader-generation

Conversation

@go-to-k

@go-to-k go-to-k commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Closes #4838

Problem

Take an SQS queue named from a secret. The secret rotates, and a create-only change then REPLACES the queue under the new name (sdin-q-1 -> sdin-q-2). A reader of the queue that also spells the same {{resolve:...}} reference itself, for example in a tag beside Value: Fn::GetAtt Queue.Arn, was not updated:

  1. The reader's own resolution recorded sdin-q-2 -> {{resolve:...}}.
  2. So the new ARN was redacted to the same arn:...:{{resolve:...}} that the reader had stored for the old queue.
  3. The UPDATE arm's no-change re-check (recordMatchesDesired in src/deployment/deploy-engine/update.ts) compared equal, and the reader was skipped.
  4. AWS kept the deleted queue's ARN, with an SSM parameter or queue policy pointing at a queue that no longer exists.

Fix

recordMatchesDesired is now false, so the resource is SENT rather than skipped, when both of these hold:

  • the redaction rewrote any of its resolved values (redactionRewroteDesired): the redacted desired bag differs from the resolved one after the same NoEcho coordinate masking; and
  • it reads a resource whose physical id changed this run (readsProducerReplacedThisRun): any Ref / Fn::GetAtt / DependsOn target whose id differs from the one the deploy LOADED (healBaseline).

A reference does not say which value it stood for. After a replacement that changed the id, an equal reference form is therefore no evidence that AWS holds the value.

Still skipped:

  • a reader of the replaced resource whose values no secret rewrote;
  • a reader of a resource replaced under the SAME physical id (no rotation; --replace).

Cost: a resource that spells a secret and reads a resource replaced with a new id gets one update it may not need. Replacing with a new id is already a deploy that rewrites most readers.

The issue's other option (re-resolve the stored reference under the loaded physical id) would need the resolver at the compare site. The change here stays local to the skip decision. Both no-change skips (the plain one and the NoEcho-confirmed one) read the same predicate.

Create-only paths. A top-level property the redaction rewrote also counts as MOVED in the two places that decide whether a propagated replacement stands. Without that, such a property's reference forms compared equal, the ceiling was lowered, and a reader like an AWS::Lambda::EventSourceMapping (create-only EventSourceArn) was updated in place. Cloud Control fails such an update; an SDK provider would leave the mapping on the deleted queue. With the fix it is replaced, as CloudFormation would do.

Not covered (#4852): the check is scoped to this run, so these shapes still keep a reader on a deleted resource:

  • a reader left stale by an interrupted earlier deploy;
  • a nested child fed the ARN by a parameter;
  • a cross-stack consumer;
  • a producer updated in place whose attribute embeds the rotated value.

The general fix needs a persisted generation witness, which is a state-schema decision.

Tests

tests/unit/deployment/deploy-engine-reader-replaced-producer-4838.test.ts runs the real DeployEngine / DiffCalculator / resolver, using the #4802 harness. Cases:

  • Rotated secret plus FifoQueue replacement, with the reader spelling the secret in a tag: the reader is sent the new ARN. On main it is skipped (sent = []).
  • A reader of the replaced queue whose value no secret rewrote (GetAtt Queue.Kind) is still skipped.
  • A queue replaced under the same name (--replace) leaves the reader that spells the secret skipped.
  • An event source mapping whose create-only EventSourceArn reads the replaced queue is REPLACED, not updated in place.
  • Control: an unchanged queue makes no call.

Mutation probes (each restored from a byte-exact copy), all red:

  • dropping the guard;
  • dropping the rewrite test;
  • dropping the replaced test;
  • "replaced" meaning any loaded resource;
  • the create-only ceiling ignoring rewritten paths.

The same term in the NoEcho-parameter create-only arm (movedAt) has no dedicated case.

Integ

tests/integration/secret-derived-immutable-names, new step 5g. It builds on #4805's AssembledNameQueue / AssembledQueueReader, which, from step 5f on (SDIN_READER_EDIT=true), also spells the queue name's original token in a tag (AqName). Before 5f it does not, so the step-2 create still stores the plaintext ARN and step 4 still exercises #4805.

  1. Step 5g rotates the secret's aqueue field to sdin-aq2-<suffix>.
  2. It respells the queue's token by version stage (SDIN_AQ_RESPELL=true, ...:aqueue:AWSCURRENT}}). That is a create-only change, so the queue is REPLACED under the rotated name.
  3. Premises: the log has AssembledNameQueue ... replaced, the new queue's ARN ends with the rotated name, and the old queue is gone.
  4. LOAD-BEARING: the reader's SSM value equals the new queue's ARN. Pre-fix it goes red at this expect_eq: the reader keeps the deleted queue's ARN.
  5. LOAD-BEARING (create-only reader): AqSubscription, a new SNS subscription, subscribes the queue to the new AqTopic. Its create-only Endpoint reads the queue's ARN and its filter policy spells the same token. The log must show AqSubscription ... replaced, and the topic's only subscription endpoint must be the new queue's ARN. Pre-fix the subscription is skipped and stays subscribed to the deleted queue.

The leak gates cover both queue names, in 5g's deploy log and in the destroy log. Cleanup and the destroy gone-checks cover the replacement queue too. I synthesized both arms in a scratch copy. This lane has not run it against AWS.

Review round 1

Review round 2

Both re-reviews are CLEAN. Taken from their notes:

  • A real-AWS arm for the create-only path (security nit 2): AqTopic / AqSubscription in step 5g. Cleanup deletes the topic by its per-run name, the destroy checks it is gone, and the fixture's covers: list gains the two SNS types.
  • Residual, recorded here, not changed: rewrittenPaths is decided per resource, not per producer. A create-only path that spells a secret and reads a producer replaced under the SAME id, in a resource that also reads another producer replaced under a NEW id, is replaced although its value is equal. Narrowing it means intersecting with the paths that read a new-id producer. No real CDK app is known to produce the shape, and a stateful reader still meets the stateful guard and approval.

…ot skipped, when a secret rewrote its values (#4838)
@github-actions github-actions Bot added effort:medium One re-review round, or a run of an existing integ fixture severity:medium Undone: a capability is missing but a workaround exists, or it only shows under one condition labels Oct 12, 2026
@go-to-k
go-to-k merged commit 3680ba6 into main Oct 13, 2026
14 checks passed
@go-to-k
go-to-k deleted the fix/4838-rotated-secret-reader-generation branch October 13, 2026 02:31
@go-to-k

go-to-k commented Oct 13, 2026

Copy link
Copy Markdown
Owner Author

Merged — thanks. Step 5g made this one easy to trust: the pre-fix arm went red exactly where predicted (the reader kept the deleted queue's ARN), and the broad set passed on 0c5911c.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort:medium One re-review round, or a run of an existing integ fixture severity:medium Undone: a capability is missing but a workaround exists, or it only shows under one condition

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(deploy): a reader that spells a rotated secret is skipped when its secret-named producer is replaced, and keeps the deleted resource's ARN

1 participant