Skip to content

AVRO-4332: [java] Add support for enum default values in IDL serialization - #3822

Open
prygunov wants to merge 2 commits into
apache:mainfrom
prygunov:java-enum-default
Open

AVRO-4332: [java] Add support for enum default values in IDL serialization#3822
prygunov wants to merge 2 commits into
apache:mainfrom
prygunov:java-enum-default

Conversation

@prygunov

Copy link
Copy Markdown

What is the purpose of the change

This pull request fixes a bug in IdlUtils.writeSchema where enum schemas with a default value would silently lose that default when serialized to IDL format. As a result, round-tripping a schema through IDL would drop the enum default.

Verifying this change

Added enumDefaultIsWrittenToIdl test in IdlUtilsTest that:

  • Verifies an enum schema with a default value serializes correctly (output contains } = ACTIVE;)
  • Verifies an enum schema without a default value is unaffected (output contains no =)

Documentation

  • Does this pull request introduce a new feature? no
  • If yes, how is the feature documented? not applicable

Copilot AI review requested due to automatic review settings June 14, 2026 18:44
@github-actions github-actions Bot added the Java Pull Requests for Java binding label Jun 14, 2026

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds support for emitting enum default values when serializing schemas to Avro IDL, with accompanying tests to validate the output.

Changes:

  • Update IDL serialization to append = <default>; after enum definitions when an enum default is present.
  • Add a unit test to verify enum defaults are (and are not) written to IDL output.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
lang/java/idl/src/main/java/org/apache/avro/idl/IdlUtils.java Writes enum defaults into the generated IDL output when present on the schema.
lang/java/idl/src/test/java/org/apache/avro/idl/IdlUtilsTest.java Adds coverage to assert enum default serialization behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lang/java/idl/src/main/java/org/apache/avro/idl/IdlUtils.java
Comment thread lang/java/idl/src/test/java/org/apache/avro/idl/IdlUtilsTest.java Outdated
@prygunov prygunov changed the title Add support for enum default values in IDL serialization [Java] Add support for enum default values in IDL serialization Jun 14, 2026
@prygunov

prygunov commented Aug 6, 2026

Copy link
Copy Markdown
Author

@RyanSkraba can you take this to the next RC?)

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Suppressed comments (1)

lang/java/idl/src/test/java/org/apache/avro/idl/IdlUtilsTest.java:117

  • The PR description says the test also verifies that an enum schema without a default value is unaffected (no "=") but the current test only covers the "with default" case. To avoid regressions (e.g., always emitting "= ..." or a stray "="), add a negative assertion (or a second test) for an enum created without a default, or update the PR description to match what is actually tested.
  @Test
  public void enumDefaultIsWrittenToIdl() throws IOException {
    Schema withDefault = Schema.createEnum("Status", null, "naming", asList("ACTIVE", "INACTIVE"), "ACTIVE");

    StringWriter withDefaultWriter = new StringWriter();
    IdlUtils.writeIdlProtocol(withDefaultWriter, withDefault);

    assertTrue(withDefaultWriter.toString().contains("} = ACTIVE;"),
        "Enum with default should serialize default value");
  }

@iemejia
iemejia self-requested a review August 11, 2026 22:02
@iemejia

iemejia commented Aug 11, 2026

Copy link
Copy Markdown
Member

Hi @prygunov, sorry for taking so long to review this — thanks for the fix and for your patience.

The change is correct and a clear improvement. The writer was asymmetric with IdlReader (which already parses enum defaults via exitEnumDefault), so any enum with a default was silently dropped on round-trip. The fix matches both the IDL grammar (RBrace defaultSymbol=enumDefault?} = SYMBOL;) and the existing fixtures (status_schema.avdl, simple.avdl). I applied it locally and confirmed a full write → parse round-trip now preserves getEnumDefault(), with no change for enums that have no default.

A couple of things before we can merge:

  1. JIRA ticket + title. Per our contribution guidelines, PRs need an associated JIRA issue and the title should follow AVRO-XXXX: [java] .... Could you please create a ticket at https://issues.apache.org/jira/projects/AVRO/issues/ and update the PR title accordingly (e.g. AVRO-XXXX: [java] Add support for enum default values in IDL serialization)?

  2. Strengthen the test. The current test only covers the positive case, though the PR description also mentions the "no default" case. Could you add:

    • a negative assertion that an enum without a default emits no =, and
    • a real round-trip assertion that proves the reported bug is fixed — write to IDL, parse it back, and check the default survives:
    IdlReader reader = new IdlReader();
    Schema roundTripped;
    try (InputStream in = new ByteArrayInputStream(withDefaultWriter.toString().getBytes(StandardCharsets.UTF_8))) {
      roundTripped = reader.parse(in).getNamedSchemas().get("naming.Status");
    }
    assertEquals("ACTIVE", roundTripped.getEnumDefault());

Thanks again!

@prygunov prygunov changed the title [Java] Add support for enum default values in IDL serialization [Java] AVRO-4332 Add support for enum default values in IDL serialization Aug 13, 2026
@prygunov prygunov changed the title [Java] AVRO-4332 Add support for enum default values in IDL serialization AVRO-4332: [java] Add support for enum default values in IDL serialization Aug 13, 2026
prygunov and others added 2 commits August 17, 2026 11:30
…ation

IdlUtils.writeSchema did not emit the enum default when serializing a
schema to IDL, even though IdlReader parses it. As a result, an enum
default was silently dropped when round-tripping a schema through IDL.

Append "= <default>;" after the enum body when the schema has a default,
matching the IDL grammar (RBrace defaultSymbol=enumDefault?), and add
tests covering the with-default, without-default and write-then-parse
round-trip cases.
The IdlUtils test class was never executed: Surefire only includes
classes matching **/Test** (name starting with "Test"), but the class
was named IdlUtilsTest, so its tests silently rotted since AVRO-3677.

Rename IdlUtilsTest to TestIdlUtils so the suite runs, and fix the
problems this uncovers:

- byte[] values serialized to an empty string because the byte[]
  serializer discarded MAPPER.writeValueAsString(...) instead of
  writing to the generator; write the value to the generator.
- The callToJson test helper had the same discard bug, so every
  *ToJson assertion previously compared against an empty string.
- The happy-flow fixtures lived under org/apache/avro/util and were
  unreachable from the org.apache.avro.idl package; move them beside
  the test and regenerate them from the current writer output.
- getMainSchema() now returns the record directly, so drop the
  obsolete union unwrapping in validateHappyFlowForSingleSchema.
- Map/collection JSON expectations now include the ", " separator the
  MAPPER emits.
@iemejia
iemejia force-pushed the java-enum-default branch from 5d496c2 to a052a75 Compare August 17, 2026 14:40
@iemejia

iemejia commented Aug 17, 2026

Copy link
Copy Markdown
Member

Thanks @prygunov — the enum-default fix is correct. I pushed some changes directly to finish it up (force-pushed, so reset your local copy):

1. Squashed your 3 commits into one under your authorship with a proper AVRO-4332 message (tree unchanged).

2. Added a maintainer commit fixing pre-existing breakage your PR surfaced. The IdlUtilsTest class never actually ran in CI — Surefire only includes **/Test** (names starting with Test), so it's been dormant since it was added. Your new tests wouldn't have run either (not your fault — you followed the file's convention). Renaming it to TestIdlUtils uncovered latent bugs I fixed:

  • Real bug in IdlUtils: the byte[] serializer discarded its output instead of writing to the generator, so bytes serialized to "".
  • The callToJson test helper had the same discard bug.
  • The validateHappyFlow* fixtures were in the wrong package (unreachable → NPE); moved and regenerated them.
  • Dropped obsolete union-unwrapping and fixed the map/collection separator expectations.

The idl module now runs 39 green tests (your 4 enum tests + 18 previously hidden).

@iemejia iemejia left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Java Pull Requests for Java binding

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants