Skip to content

Loading meta-data from JSON silently drops proto2 extensions #4478

Description

@hatyo

The yaml-tests framework can load a schema template from a JSON meta-data file (load schema template ... from <file>.json, implemented in CommandUtil.loadRecordMetaDataFromJson). That parse goes through JsonFormat, which implements the proto3 JSON mapping and treats proto2 extensions as if they did not exist. Extensions written into the file are therefore dropped on the way back in, without any warning.

This matters because descriptor options that the meta-data depends on are carried by extensions. A vector field is a bytes field whose precision and dimensions live in an extension of google.protobuf.FieldOptions:

  "embedding": { "type": "TYPE_BYTES",
    "options": { "com.apple.foundationdb.record.field": {
        "vectorOptions": { "precision": 64, "dimensions": 512 }}}}

After loading, that field is a plain bytes column instead of a vector

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions