Repository navigation
Read per-message keys and signature versions 8–11 - #32
Closed
santiagomed wants to merge 20 commits into
Closed
santiagomed wants to merge 20 commits into
santiagomed wants to merge 20 commits into
Conversation
|
|
santiagomed
changed the title
Resync thrift/event.thrift and thrift/product.thrift with the schema the native clients use and regenerate the Rust types. The fields the SDK needs from the resync are MessageCreateEvent.contents_encryption_version, GroupMemberAddChange.current_description, GroupCreate.group_type, GroupMemberRemoveChange.conversation_key_version, ReplyingToPreview.raw_event_message_key / raw_event_edit_message_key, MediaAttachment.plaintext_sha256 and MessageEdit.message_id. Two declarations move (SideEffect after MessageCreateEvent, GrokAttachments before MessageAttachment) so the generator boxes list-of-union elements; the definitions are otherwise verbatim. No behavior change: union variants and attachment kinds the SDK does not model decode to the same outputs as when they were unknown (unknown group and settings changes, attachments left out), and reply-preview attachment claims ignore those kinds and the new plaintext hash, as before.
Rebuild each received signature's payload in the layout of the version it
carries instead of always using the v7 layout, so events signed by updated
clients verify:
- v8: the message payload signs the conversation key version as a
canonical integer ("null" when missing or not an integer);
GroupMemberAdd appends current_description.
- v9: every payload ends with ",signature_version=<n>"; GroupCreate
appends its group_type number.
- v10: the message payload appends contents_encryption_version ("null"
when unset); GroupCreate appends ttl_msec.
- v11: GroupMemberRemove appends conversation_key_version.
Signatures labeled above 11 are rejected. Versions 2 through 7 verify
exactly as before, including v7 messages whose key version is not an
integer. The SDK still signs at v7.
Payload known answers, a native-client signature over a v10 payload, and
the action-signature cases come from the native clients' tests.
A message whose MessageCreateEvent.contents_encryption_version is 1 or more is encrypted under a message key, HKDF-SHA256(conversation key, "xchat-message-key", "<sender id>:<message id>"), and its media under per-attachment keys, HKDF-SHA256(message key, "xchat-attachment-key", attachment id). decrypt_event and decrypt_events now open both forms with the conversation key callers already pass; an unset, zero or negative version still means the conversation key. Media keys stay inside the SDK. When a ChatCore decrypts a message-key message it records the message's attachment keys under that conversation key, and decrypt_stream / stream_decryptor try the conversation key first and then the recorded keys, keeping whichever authenticates the first frame. An attachment's id is its attachment_id when set (empty included), else the message id for the first attachment and "<message id>#<index>" after; a URL card's images share the card's key. The records live in memory, are cleared by lock, and keep the 4,096 most recent keys. Every binding's stream decryption now goes through the chat instance so it sees those keys: Python's decrypt_stream calls ChatCore, and Go, .NET and the JVM create their incremental decryptor through a new internal native entry point, chat_xdk_stream_decryptor_new_with_chat (the existing chat_xdk_stream_decryptor_new stays). No public API changes. Reply previews quoting a message-key original open it with the 32-byte key the reply carries (raw_event_message_key / raw_event_edit_message_key), or derive it from the original's conversation key. An edit embedded next to a message-key original must name it by MessageEdit.message_id. A valid preview records the quoted attachments' keys under the reply's conversation key and, when known, the original's. Known answers come from the native clients' key derivation and tests.
encrypt_reply with reply_to_event now carries the quoted original's message key in raw_event_message_key, and its edit's in raw_event_edit_message_key, when they use one, so recipients without the original's conversation key can still open and validate the preview. A conversation key is never carried, so replies quoting conversation-key messages are unchanged on the wire. The key is resolved even when overrides make the original's contents unnecessary; failing to resolve it then only means no key is carried. encrypt_edit with target_event now sets MessageEdit.message_id to the edited message's id, which receivers need to tie an edit to a message-key original. With only the explicit target fields it stays unset. Replies still embed raw_event_ckces: SDK 0.5.0 and earlier need them to validate a preview whose original uses another key version, and clients that stopped reading the field ignore it.
Extend tests/fixtures/sdk_vectors.json with events whose contents and media are encrypted directly under known message and attachment keys taken from the native clients: a v10 message with two media without ids, the key change that delivers its conversation key, a v11 message from a short sender id, a reply that carries the quoted message's key, and a native-client signature over a v10 payload that signs contents_encryption_version 1. Every binding (Rust, Python, JS/WASM, Go, .NET, JVM) now checks that these decrypt and verify through decrypt_event / decrypt_events with only the conversation key, that the media open through decrypt_stream and the incremental stream decryptor once their message is decrypted, that the reply's preview validates, and that the known signature verifies only with the version it signed. The existing vectors are unchanged. internals gains frame_message_key_event, used by the fixture generator.
… to 11 CRYPTO.md gains the message-key and attachment-key derivations, how the stream decrypt methods find an attachment's key from the conversation key, the per-version payload changes verification follows (v8 to v11), the HKDF info input, and the carried reply keys and edit message id rules. API.md notes what callers see: replies carry per-message keys, edits name the edited message, and media of newer messages opens with the conversation key after its message is decrypted by the same Chat. ARCHITECTURE.md lists the two new internal modules.
The other constants in the signature-version table are public; this one is crate-private.
…e schema Remove the server-side annotations (type names and field encodings) from the definitions this branch adds to thrift/event.thrift and thrift/product.thrift, and reword comments that named backend services and internal rules. Annotations and comments do not reach the generated Rust code, which is unchanged.
santiagomed
force-pushed
the
cursor/message-keys-sig-versions-21f1
branch
from
October 14, 2026 02:18
8268e0b to
cd8d629
Compare
…rries it decrypt_stream and stream_decryptor take the message's event, the conversation key map and the media hash key instead of a raw key. The SDK finds the attachment the hash key names (media, a link card's banner or favicon, or media quoted in a reply preview), picks the key for the message's version, and derives the attachment's key. Nothing is kept between calls. Errors say what to pass when the key version is missing, the event is not an encrypted message with media, the hash key is not in it, or the bytes are wrong or truncated.
…n the media calls decryptStream and streamDecryptor in Python, JS/WASM, Go, .NET and JVM now take the message's event, the conversation key map and the media hash key, named and typed like decryptEvent's parameters in each language, in place of a raw key. A key-map entry that is not 32 bytes is an error. The C ABI follows: chat_xdk_decrypt_stream takes the event, the key-map JSON and the media hash key, and chat_xdk_stream_decryptor_new returns a result with the error text and writes the handle to an out-parameter.
The fixture gains media for one file (the short-sender message now carries one), a link card next to a photo, a conversation-key message, an unencrypted message, and the media hash keys of every file. The existing entries are unchanged. The Rust vector tests decrypt each file from its message event, whole and incrementally, including quoted media through the reply, and check each error.
Python, JS, Go, .NET and JVM decrypt one file, several files, a link card's banner and favicon, quoted media through a reply, and a conversation-key message's media, whole and incrementally, and check that each error says what to pass: a missing key version, a media hash key not in the message, an event that is not a message or has no media, wrong or truncated bytes, an unencrypted message, and a malformed key map. The stream round-trip tests decrypt through the conversation-key message.
Each example's decryptMedia takes the downloaded bytes, the raw event of the message that carries the media, the conversation key map and the media hash key. The live end-to-end flows pass the media message's raw event, and the offline tests decrypt through the fixture's conversation-key media message.
API.md documents the new decrypt_stream / stream_decryptor parameters in every binding, how the key is chosen, the errors and what each asks for, and updates the quick starts. CRYPTO.md describes media keys and how they are derived from the event, and ARCHITECTURE.md lists the media module.
The entry describes the new decrypt_stream / stream_decryptor parameters and the matching C ABI change, with a before/after snippet for Rust, Python, JS/TypeScript, Go, .NET and the JVM.
The wire types hold attachments as boxed lists, so the helpers that build them take the same type.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds support for per-message keys and signature versions 8–11, and makes media decryption take the message that carries the media.
Breaking:
decrypt_streamandstream_decryptortake new parameters in every binding, and so dochat_xdk_decrypt_streamandchat_xdk_stream_decryptor_newin the C ABI.CHANGELOG.mdhas a before/after for each language. Encryption is unchanged.Media decryption
Both calls take the base64 event of the message that carries the media and the conversation key map — the same two arguments
decrypt_eventtakes — plus the media's hash key from the decrypted message:decrypt_stream(encrypted, event_b64, conversation_keys, media_hash_key)stream_decryptor(event_b64, conversation_keys, media_hash_key)decrypt_stream(encrypted, event_b64, conversation_keys, media_hash_key)stream_decryptor(event_b64, conversation_keys, media_hash_key)decryptStream(encrypted, eventB64, conversationKeys, mediaHashKey)streamDecryptor(eventB64, conversationKeys, mediaHashKey)DecryptStream(encrypted, eventB64, conversationKeys, mediaHashKey)StreamDecryptor(eventB64, conversationKeys, mediaHashKey)DecryptStream(encrypted, eventB64, conversationKeys, mediaHashKey)StreamDecryptor(eventB64, conversationKeys, mediaHashKey)decryptStream(encrypted, eventB64, conversationKeys, mediaHashKey)streamDecryptor(eventB64, conversationKeys, mediaHashKey)The key map has the type each binding's
decrypt_eventuses (HashMap<String, XChatConversationKey>,dict[str, bytes],ConversationKeyMap,map[string][]byte,Dictionary<string, byte[]>,Map<String, byte[]>).The SDK parses the event, takes the key for the message's version from the map, decrypts the message, and finds the attachment the hash key names:
It then derives that file's key (an attachment key derived from the message key and the attachment id for messages that use a message key, the conversation key otherwise) and decrypts with that key alone. Nothing is kept between calls, so any instance can decrypt any media from these arguments.
Errors say what to pass instead when:
In the C ABI,
chat_xdk_stream_decryptor_newreturns a result carrying the error text and writes the handle to an out-parameter, so Go, .NET and the JVM report the same errors as the other bindings.Per-message keys and signature versions 8–11
signature_versionsuffix and group type, v10contents_encryption_versionand group TTL, v11 member-remove key version). Versions above 11 are rejected.contents_encryption_version≥ 1 is encrypted under a key derived from the conversation key, the sender id and the message id.decrypt_event/decrypt_eventsopen both forms with the conversation key map callers already pass.MessageEdit.message_id.encrypt_editwithtarget_eventsetsMessageEdit.message_id; older readers skip both optional fields.thrift/event.thriftandthrift/product.thriftare resynced and regenerated; new union variants and attachment kinds produce the same outputs as when they were unknown.Tests, docs and examples
tests/fixtures/sdk_vectors.jsongains media vectors: one file, several files, a link card, a conversation-key message, an unencrypted message, and the media hash keys. Existing entries are unchanged. Every binding decrypts each file whole and incrementally, including quoted media through a reply, and checks each error.docs/API.mddocuments both calls in every binding, with the errors and updated quick starts.docs/CRYPTO.mddescribes media keys and how they are derived from the event. The examples in all six languages pass the media message's event todecryptMedia.Testing
make cipasses: fmt, clippy with-D warnings,cargo deny check licenses, andcargo test --workspace(619 core unit tests, 17 vector tests).The committed Go prebuilts (
go/chatxdk/libs/) match the new C ABI once the prebuilt sync workflows commit rebuilt archives to this branch.