fix: Use LONGBLOB for SQL registry proto columns on MySQL

.claude/rules/feast-components.md

@@ -24,6 +24,7 @@ For testing patterns and debugging, also read `skills/feast-testing/SKILL.md`.
 
 - **Unit tests**: add or update tests in `sdk/python/tests/unit/infra/<subsystem>/`
 - **Integration tests**: run `make test-python-integration-local`; add a universal test case in `sdk/python/tests/integration/` if the change affects retrieval or materialization behavior
+- **SQL registry binary columns**: in `infra/registry/sql.py`, a new column that stores a serialized proto or blob metadata must use `ProtoBytes`, not `LargeBinary` directly — `LargeBinary` maps to MySQL `BLOB` (64 KB cap) and silently truncates large protos
 - **Protos**: if you add a field to a proto message, recompile with `make protos` and update serialization helpers in `proto_registry_utils.py`
 - **Both SDKs**: if the change affects online serving, check whether the Go server (`go/`) also needs updating
 - **Skills/Rules**: if the change introduces new patterns, interfaces, or conventions that agents should follow, update the relevant section in `skills/feast-architecture/SKILL.md` (and `skills/feast-testing/SKILL.md` if testing patterns changed)

.cursor/rules/feast-components.mdc

@@ -20,6 +20,7 @@ For testing patterns and debugging, also read `skills/feast-testing/SKILL.md`.
 
 - **Unit tests**: add or update tests in `sdk/python/tests/unit/infra/<subsystem>/`
 - **Integration tests**: run `make test-python-integration-local`; add a universal test case in `sdk/python/tests/integration/` if the change affects retrieval or materialization behavior
+- **SQL registry binary columns**: in `infra/registry/sql.py`, a new column that stores a serialized proto or blob metadata must use `ProtoBytes`, not `LargeBinary` directly — `LargeBinary` maps to MySQL `BLOB` (64 KB cap) and silently truncates large protos
 - **Protos**: if you add a field to a proto message, recompile with `make protos` and update serialization helpers in `proto_registry_utils.py`
 - **Both SDKs**: if the change affects online serving, check whether the Go server (`go/`) also needs updating
 - **Skills/Rules**: if the change introduces new patterns, interfaces, or conventions that agents should follow, update the relevant section in `skills/feast-architecture/SKILL.md` (and `skills/feast-testing/SKILL.md` if testing patterns changed)

docs/reference/registries/sql.md

@@ -83,7 +83,84 @@ If you are running Feast in Kubernetes, set the `image.repository` and
 There are some things to note about how the SQL registry works:
 - Once instantiated, the Registry ensures the tables needed to store data exist, and creates them if they do not.
 - Upon tearing down the feast project, the registry ensures that the tables are dropped from the database.
-- The schema for how data is laid out in tables can be found . It is intentionally simple, storing the serialized protobuf versions of each Feast object keyed by its name.
+- The schema for how data is laid out in tables can be found in the table definitions in [`sdk/python/feast/infra/registry/sql.py`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/infra/registry/sql.py). It is intentionally simple, storing the serialized protobuf versions of each Feast object keyed by its name.
+
+## MySQL: serialized-proto columns use `LONGBLOB`
+
+The registry stores each Feast object as a serialized protobuf in a binary
+column. On MySQL these columns are created as `LONGBLOB` (up to 4 GB). Earlier
+versions created them as `BLOB`, which caps at 64 KB — a single `FeatureView`
+proto routinely exceeds that, so MySQL would silently truncate the write and the
+registry would later fail to load with a protobuf `DecodeError` (for example,
+`feast serve` failing to start). Other dialects (PostgreSQL, SQLite) were never
+affected.
+
+New deployments get the correct schema automatically — the registry creates its
+tables as `LONGBLOB` on first use. When an existing MySQL/MariaDB registry still
+has `BLOB` columns, the registry logs an error at startup listing the affected
+columns (it does not refuse to start — a registry whose protos all fit in 64 KB
+is unaffected). **Existing deployments are not migrated automatically**: the
+registry only creates tables that do not already exist, and it has no
+schema-migration step, so previously created `BLOB` columns remain `BLOB`. To
+upgrade an existing MySQL registry, alter each serialized-proto column to
+`LONGBLOB`, for example:
+
+> ⚠️ **Run the migration carefully on a live registry.** A `BLOB`→`LONGBLOB`
+> change is a column *data-type* change, which MySQL InnoDB performs with
+> `ALGORITHM=COPY` — a full table rebuild under a metadata lock that blocks
+> readers and writers for the duration (potentially minutes on a large table
+> such as `feature_view_version_history`). `ALGORITHM=INPLACE` is **not**
+> generally supported for this change and is rejected with
+> `ER_ALTER_OPERATION_NOT_SUPPORTED_REASON` on most builds — do not rely on it.
+>
+> **Before running any `ALTER TABLE`:**
+>
+> 1. **Stop all `feast apply` and materialization jobs.** This is required, not
+>    optional — a write of a `>64 KB` proto to a not-yet-widened `BLOB` column
+>    truncates silently with no error, and concurrent writes also extend the
+>    `ALTER`'s lock duration.
+> 2. Confirm there are no active writers (e.g. `SHOW PROCESSLIST`).
+> 3. Verify you have a backup of the registry database.
+>
+> Then, to minimize the lock window:
+>
+> - On large tables, or on managed MySQL (AWS RDS, Aurora) without shell access,
+>   use an online schema-change tool —
+>   [`pt-online-schema-change`](https://docs.percona.com/percona-toolkit/pt-online-schema-change.html)
+>   (Percona Toolkit) or [`gh-ost`](https://github.com/github/gh-ost) — which
+>   rebuild the table without a long-held lock. For small tables a plain
+>   `ALTER TABLE` in the maintenance window is fine.
+> - Apply one table at a time so a failure is easy to isolate and re-run.
+> - Resume jobs only after all `ALTER TABLE` statements complete successfully.
+> - Rollback is safe (revert `MODIFY ... BLOB`) **only** while no stored proto
+>   exceeds 64 KB; otherwise a revert re-introduces truncation.
+
+```sql
+ALTER TABLE projects               MODIFY project_proto              LONGBLOB NOT NULL;
+ALTER TABLE entities               MODIFY entity_proto               LONGBLOB NOT NULL;
+ALTER TABLE data_sources           MODIFY data_source_proto          LONGBLOB NOT NULL;
+ALTER TABLE feature_views          MODIFY materialized_intervals     LONGBLOB,
+                                   MODIFY feature_view_proto         LONGBLOB NOT NULL,
+                                   MODIFY user_metadata              LONGBLOB;
+ALTER TABLE stream_feature_views   MODIFY feature_view_proto         LONGBLOB NOT NULL,
+                                   MODIFY user_metadata              LONGBLOB;
+ALTER TABLE on_demand_feature_views MODIFY feature_view_proto        LONGBLOB NOT NULL,
+                                   MODIFY user_metadata              LONGBLOB;
+ALTER TABLE label_views            MODIFY feature_view_proto         LONGBLOB NOT NULL,
+                                   MODIFY user_metadata              LONGBLOB;
+ALTER TABLE feature_services       MODIFY feature_service_proto      LONGBLOB NOT NULL;
+ALTER TABLE saved_datasets         MODIFY saved_dataset_proto        LONGBLOB NOT NULL;
+ALTER TABLE validation_references  MODIFY validation_reference_proto LONGBLOB NOT NULL;
+ALTER TABLE managed_infra          MODIFY infra_proto                LONGBLOB NOT NULL;
+ALTER TABLE permissions            MODIFY permission_proto           LONGBLOB NOT NULL;
+-- LARGE TABLE: one row per versioned apply — likely the slowest ALTER. Use
+-- pt-online-schema-change or gh-ost if this registry has significant history.
+ALTER TABLE feature_view_version_history MODIFY feature_view_proto   LONGBLOB NOT NULL;
+```
+
+Any object whose proto already exceeded 64 KB before the upgrade may have been
+stored truncated; re-run `feast apply` for those objects after altering the
+columns so the full proto is rewritten.
 
 ## Example Usage: Concurrent materialization
 The SQL Registry should be used when materializing feature views concurrently to ensure correctness of data in the registry. This can be achieved by simply running feast materialize or feature_store.materialize multiple times using a correctly configured feature_store.yaml. This will make each materialization process talk to the registry database concurrently, and ensure the metadata updates are serialized.