added generation of type specific 'MarshalJSON' & 'UnmarshalJSON' implementations by sailingKieler · Pull Request #98 · TypeFox/fastbelt

sailingKieler · 2026-06-08T07:02:49Z

Contributes support for exporting and importing ASTs in Json.
Includes:

a generator component producing json_gen.go files
- they contribute methods turning AstNodes into impls of json.Marshaler & json.Unmarshaler
generic MarshalJSON & UnmarshalJSON implementations for Reference
util/json/json.go demonstrates the procedure of importing an AST in json into a Fastbelt container
a simple round trip AST export and import test including cross references to other documents (circular) by means of the Arithmetics test language,
- focus is on successfully resolving the (cross document) cross-references
a roundtrip AST export and import test (no external cross-refs by now) by means of the Grammar language,
- focus is on completeness of feature processing and cardinalities:
  boolean, string, AstNode, Reference x 1, ?, *, +

Note:

Minor supplements in fixture.go
Fix of a potential nil pointer de-reference in symbol.go

There's one known potential issue:
During the import of an AST in json the Token instances are always initialized with token type Token_ID.
The leads to a compiler issue if there is no ID token type.

We could improve this by an analysis of the Parser rules, but that will never be complete.
Imagine the grammar defines interfaces of types that are never instantiated by parser rules, but the data are always imported vom Json snapshots.
To mitigate this we might offer an opportunity in the grammar to determine a token type for fields of type string or composite. To be discussed.

spoenemann · 2026-06-10T12:32:14Z

+	}
+	// We expect at this point that all references have been attempted to resolve.
+	if !r.resolved.Load() {
+		return nil, errors.New("reference not resolved")


Proposal: generate a partial reference like in Langium:

https://github.com/eclipse-langium/langium/blob/7ad7c76e85134185146e1a99bd352e7527ed1fec/packages/langium/src/serializer/json-serializer.ts#L174-L177

This is just preliminary, I don't know which context instance to rely on for calling Ref(...), yet
(since this function is called by json.Mashall(...), not by ourselves), and whether we should attempt to resolve here, at all.

AFAICS, r.resolved is true if resolution has been attempted, regardless of success or failure.
Hence, this guards stops the serialization, e.g., if the document hasn't been build until DocStateLinked at least.

spoenemann · 2026-06-10T12:33:20Z

+		RefText string `json:"refText"`
+		URI     string `json:"uri,omitempty"`
+		Err     string `json:"error,omitempty"`
+	}{


This struct def should be exported. And I propose aligning this with Langium for cross-framework compatibility:

https://github.com/eclipse-langium/langium/blob/7ad7c76e85134185146e1a99bd352e7527ed1fec/packages/langium/src/serializer/json-serializer.ts#L96-L105

This struct def should be exported.

Why?
It's purely technical and kind of a vehicle.
It might make sense to rely in the two (Un)Marshal methods on the same definition, though.

And I propose aligning this with Langium for cross-framework compatibility:

That's my plain, too.

spoenemann · 2026-06-10T12:43:10Z

+		Err     string `json:"error,omitempty"`
+	}{
+		RefText: r.unit.String(),
+		URI:     uri,


In Langium, we omit the URI if the reference target is in the same document. This can reduce the size of generated JSON by a lot.

Thought: can we include something like a URI converter here as well? Exporting JSONs with absolute file URIs would be pretty useless in many cases.

https://github.com/eclipse-langium/langium/blob/7ad7c76e85134185146e1a99bd352e7527ed1fec/packages/langium/src/serializer/json-serializer.ts#L161-L167

In Langium, we omit the URI if the reference target is in the same document. This can reduce the size of generated JSON by a lot.

That's achieved by the omitempty setting. Of course, the prop value shouldn't be set upfront.

Thought: can we include something like a URI converter here as well? Exporting JSONs with absolute file URIs would be pretty useless in many cases.

Yes, that's my plan.

sailingKieler

Thanks for the hints Miro.
This state is a very early one, it doesn't represent the one I have in mind already.

I also seek for a solution that is compatible with Langium, s.t. we could use it as an interface for data exchange

More changes are coming soon :-)

sailingKieler · 2026-06-10T13:17:38Z

+		Err     string `json:"error,omitempty"`
+	}{
+		RefText: r.unit.String(),
+		URI:     uri,


In Langium, we omit the URI if the reference target is in the same document. This can reduce the size of generated JSON by a lot.

That's achieved by the omitempty setting. Of course, the prop value shouldn't be set upfront.

Thought: can we include something like a URI converter here as well? Exporting JSONs with absolute file URIs would be pretty useless in many cases.

Yes, that's my plan.

sailingKieler · 2026-06-10T13:21:09Z

+		RefText string `json:"refText"`
+		URI     string `json:"uri,omitempty"`
+		Err     string `json:"error,omitempty"`
+	}{


This struct def should be exported.

Why?
It's purely technical and kind of a vehicle.
It might make sense to rely in the two (Un)Marshal methods on the same definition, though.

And I propose aligning this with Langium for cross-framework compatibility:

That's my plain, too.

sailingKieler · 2026-06-10T13:28:15Z

+	}
+	// We expect at this point that all references have been attempted to resolve.
+	if !r.resolved.Load() {
+		return nil, errors.New("reference not resolved")


This is just preliminary, I don't know which context instance to rely on for calling Ref(...), yet
(since this function is called by json.Mashall(...), not by ourselves), and whether we should attempt to resolve here, at all.

AFAICS, r.resolved is true if resolution has been attempted, regardless of success or failure.
Hence, this guards stops the serialization, e.g., if the document hasn't been build until DocStateLinked at least.

…lementations * also added generic 'MarshalJSON' & 'UnmarshalJSON' implementation for 'Reference'

spoenemann reviewed Jun 10, 2026

View reviewed changes

sailingKieler commented Jun 10, 2026

View reviewed changes

sailingKieler force-pushed the cs/json-marshalling branch 2 times, most recently from 23336ec to 10ec511 Compare June 17, 2026 18:23

msujew mentioned this pull request Jun 23, 2026

Fix completion after grammar action #116

Merged

sailingKieler force-pushed the cs/json-marshalling branch 3 times, most recently from 66a1bfa to f72abbc Compare July 3, 2026 13:29

sailingKieler added 7 commits July 8, 2026 17:28

added generation of type specific 'MarshalJSON' & 'UnmarshalJSON' imp…

b7d6024

…lementations * also added generic 'MarshalJSON' & 'UnmarshalJSON' implementation for 'Reference'

progress on 'json_generator.go'

2b4e9b0

unmarshaling

adae413

improvement in 'util/json.go'

f7503c8

hot fix in reference.go

6af4e24

progress, linking seems to work

9c18c80

refinements

9ca89d3

sailingKieler force-pushed the cs/json-marshalling branch from f72abbc to cc2a141 Compare July 9, 2026 13:08

sailingKieler marked this pull request as ready for review July 9, 2026 13:30

sailingKieler added 5 commits July 9, 2026 15:44

more refinements

9da1fda

first json test

b50ed3d

fix in symbols handling in 'symbols.go'

715d573

some supplements to 'fixture.go'

0006602

json roundtrip test with/for grammar language

8ca6a6c

sailingKieler force-pushed the cs/json-marshalling branch from cc2a141 to 8ca6a6c Compare July 9, 2026 13:45

sailingKieler added 2 commits July 9, 2026 17:01

refined benchmark to capture relevant measures only

3b72d78

optimization of slice allocation during unmarshaling

631fd58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

added generation of type specific 'MarshalJSON' & 'UnmarshalJSON' implementations#98

added generation of type specific 'MarshalJSON' & 'UnmarshalJSON' implementations#98
sailingKieler wants to merge 14 commits into
mainfrom
cs/json-marshalling

sailingKieler commented Jun 8, 2026 •

edited

Loading

Uh oh!

spoenemann Jun 10, 2026

Uh oh!

sailingKieler Jun 10, 2026

Uh oh!

spoenemann Jun 10, 2026

Uh oh!

sailingKieler Jun 10, 2026

Uh oh!

spoenemann Jun 10, 2026

Uh oh!

sailingKieler Jun 10, 2026

Uh oh!

sailingKieler left a comment

Uh oh!

sailingKieler Jun 10, 2026

Uh oh!

sailingKieler Jun 10, 2026

Uh oh!

sailingKieler Jun 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

sailingKieler commented Jun 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sailingKieler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sailingKieler commented Jun 8, 2026 •

edited

Loading