Skip to content

Add support for JSON Index creation syntax in SQL Server 2025 (TSql170) #147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Leila Lali (llali) merged 21 commits into from
Jul 30, 2025
Merged

Add support for JSON Index creation syntax in SQL Server 2025 (TSql170) #147

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
269592c
Initial plan
Copilot Jul 22, 2025
90ad82c
Initial planning and exploration for JSON Index support
Copilot Jul 22, 2025
3e23f8b
Add JSON Index support - AST definitions, grammar rules, and script g…
Copilot Jul 22, 2025
9d7e0db
Complete JSON Index implementation - fix parser recognition and add c…
Copilot Jul 22, 2025
753a74b
Final fix - correct error count expectations for older SQL parser ver…
Copilot Jul 22, 2025
73a1645
Remove antlr-2.7.5.jar file from repository
Copilot Jul 22, 2025
9de9b7f
Remove tmp folder from repository and add to .gitignore
Copilot Jul 22, 2025
4865dd4
Revert global.json SDK version back to 8.0.410
Copilot Jul 22, 2025
373dc4d
Remove NUL file - ANTLR build artifact
Copilot Jul 22, 2025
ff38699
Use CreateXmlIndexStatement pattern - inherit from IndexStatement ins…
Copilot Jul 22, 2025
53a2aba
Address reviewer feedback: rename JsonPaths to ForJsonPaths and add c…
Copilot Jul 22, 2025
28c5564
Fix error test position calculations for accurate error reporting
Copilot Jul 22, 2025
667ca5c
merge from main
llali Jul 22, 2025
9656c7e
Change JSON Index column from ColumnReferenceExpression to Identifier…
Copilot Jul 22, 2025
89c0efc
Remove ErrorTest90/ErrorTest100 calls and fix missing comma in Only17…
Copilot Jul 22, 2025
50a1bff
Fix parser error test offsets - correct positions for JSON keyword er…
Copilot Jul 22, 2025
d0e2993
Remove ErrorTest80 call from ParserErrorsTests as requested
Copilot Jul 22, 2025
b6073da
Remove ErrorTest160 calls from ParserErrorsTests for JSON Index - onl…
Copilot Jul 22, 2025
ab3a74f
fixing the test
llali Jul 23, 2025
eee2f48
Remove filegroup support from JSON Index syntax
Copilot Jul 28, 2025
04e7f67
Fix JSON Index error count in Only170SyntaxTests - correct to 8 error…
Copilot Jul 28, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions SqlScriptDom/Parser/TSql/Ast.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4603,8 +4603,8 @@
<Class Name="CreateJsonIndexStatement" Base="IndexStatement" Summary="Represents the create JSON index statement.">
<InheritedMember Name="Name" ContainerClass="IndexStatement" />
<InheritedMember Name="OnName" ContainerClass="IndexStatement" />
<Member Name="JsonColumn" Type="ColumnReferenceExpression" Summary="The JSON column for the index."/>
<Member Name="JsonPaths" Type="StringLiteral" Collection="true" Summary="The JSON paths specified in the FOR clause. Optional may have zero elements."/>
<Member Name="JsonColumn" Type="Identifier" Summary="The JSON column for the index."/>
Comment thread
llali marked this conversation as resolved.
<Member Name="ForJsonPaths" Type="StringLiteral" Collection="true" Summary="The JSON paths specified in the FOR clause. Optional may have zero elements."/>
<InheritedMember Name="IndexOptions" ContainerClass="IndexStatement" />
<Member Name="OnFileGroupOrPartitionScheme" Type="FileGroupOrPartitionScheme" Summary="The filegroup or partition scheme. Might be null."/>

@llali Leila Lali (llali) Jul 28, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like file group is not supported for json index. remove file group from ast.xml and from TSQL170.g and apply the same change in tests

Copilot AI Jul 28, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed filegroup support from JSON Index syntax as requested. The changes include:

  • Removed OnFileGroupOrPartitionScheme member from AST
  • Removed filegroup handling from TSql170.g grammar rule
  • Updated script generator to exclude filegroup generation
  • Removed filegroup test cases from test scripts and baselines

JSON indexes now support only the core syntax without ON filegroup clause. (commit eee2f48)

</Class>
Expand Down
8 changes: 4 additions & 4 deletions SqlScriptDom/Parser/TSql/TSql170.g
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16988,7 +16988,7 @@ createJsonIndexStatement [IToken tUnique, bool? isClustered] returns [CreateJson
{
Identifier vIdentifier;
SchemaObjectName vSchemaObjectName;
ColumnReferenceExpression vJsonColumn;
Identifier vJsonColumn;
FileGroupOrPartitionScheme vFileGroupOrPartitionScheme;
StringLiteral vPath;

Expand All @@ -17010,7 +17010,7 @@ createJsonIndexStatement [IToken tUnique, bool? isClustered] returns [CreateJson
{
vResult.OnName = vSchemaObjectName;
}
LeftParenthesis vJsonColumn=column tRParen:RightParenthesis
LeftParenthesis vJsonColumn=identifier tRParen:RightParenthesis
{

@zijchen Z (zijchen) Jul 22, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this for?

Copilot AI Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "options {greedy = true; }" directive is used to resolve parser conflicts with common table expressions and XML namespaces. This is a standard ANTLR pattern used throughout the grammar file (appears in 30+ locations) to ensure the WITH clause is properly recognized in CREATE JSON INDEX statements. The comment explains this is needed due to conflict with withCommonTableExpressionsAndXmlNamespaces.

vResult.JsonColumn = vJsonColumn;
UpdateTokenInfo(vResult, tRParen);
Expand All @@ -17019,12 +17019,12 @@ createJsonIndexStatement [IToken tUnique, bool? isClustered] returns [CreateJson
tFor:For LeftParenthesis
vPath=stringLiteral
{
AddAndUpdateTokenInfo(vResult, vResult.JsonPaths, vPath);
AddAndUpdateTokenInfo(vResult, vResult.ForJsonPaths, vPath);
}
(
Comma vPath=stringLiteral
{
AddAndUpdateTokenInfo(vResult, vResult.JsonPaths, vPath);
AddAndUpdateTokenInfo(vResult, vResult.ForJsonPaths, vPath);
}
)*
RightParenthesis
Expand Down
4 changes: 2 additions & 2 deletions ...ScriptDom/SqlServer/ScriptGenerator/SqlScriptGeneratorVisitor.CreateJsonIndexStatement.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,12 @@ public override void ExplicitVisit(CreateJsonIndexStatement node)
}

// FOR clause with JSON paths
if (node.JsonPaths != null && node.JsonPaths.Count > 0)
if (node.ForJsonPaths != null && node.ForJsonPaths.Count > 0)
{
NewLineAndIndent();
GenerateKeyword(TSqlTokenType.For);
GenerateSpace();
GenerateParenthesisedCommaSeparatedList(node.JsonPaths);
GenerateParenthesisedCommaSeparatedList(node.ForJsonPaths);
}

GenerateIndexOptions(node.IndexOptions);
Expand Down
4 changes: 3 additions & 1 deletion Test/SqlDom/Only170SyntaxTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ public partial class SqlDomTests
private static readonly ParserTest[] Only170TestInfos =
{
new ParserTest170("RegexpTVFTests170.sql", nErrors80: 1, nErrors90: 1, nErrors100: 0, nErrors110: 0, nErrors120: 0, nErrors130: 0, nErrors140: 0, nErrors150: 0, nErrors160: 0),
new ParserTest170("JsonIndexTests170.sql", nErrors80: 2, nErrors90: 10, nErrors100: 10, nErrors110: 10, nErrors120: 10, nErrors130: 10, nErrors140: 10, nErrors150: 10, nErrors160: 10)
new ParserTest170("JsonIndexTests170.sql", nErrors80: 2, nErrors90: 10, nErrors100: 10, nErrors110: 10, nErrors120: 10, nErrors130: 10, nErrors140: 10, nErrors150: 10, nErrors160: 10),

@llali Leila Lali (llali) Jul 28, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix the number of errors after your last commit. it should be 8 errors now not 10

Copilot AI Jul 28, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the error count from 10 to 8 errors for JSON Index tests. After removing filegroup support, there are now 8 CREATE JSON INDEX statements in the test file instead of 10, so the expected error count for older parser versions was updated accordingly. (commit 04e7f67)

new ParserTest170("AlterDatabaseManualCutoverTests170.sql", nErrors80: 4, nErrors90: 4, nErrors100: 4, nErrors110: 4, nErrors120: 4, nErrors130: 4, nErrors140: 4, nErrors150: 4, nErrors160: 4),
new ParserTest170("CreateColumnStoreIndexTests170.sql", nErrors80: 3, nErrors90: 3, nErrors100: 3, nErrors110: 3, nErrors120: 3, nErrors130: 0, nErrors140: 0, nErrors150: 0, nErrors160: 0)
};

private static readonly ParserTest[] SqlAzure170_TestInfos =
Expand Down
58 changes: 58 additions & 0 deletions Test/SqlDom/ParserErrorsTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4384,6 +4384,64 @@ public void CreateIndexStatementErrorTest()
new ParserErrorInfo(47, "SQL46010", "col1"));
}

/// <summary>
/// JSON Index error tests - ensure JSON Index syntax is rejected in older versions and malformed syntax produces appropriate errors
/// </summary>
[TestMethod]
[Priority(0)]
[SqlStudioTestCategory(Category.UnitTest)]
public void CreateJsonIndexStatementErrorTest()
{
// JSON Index syntax should not be supported in SQL Server versions prior to 2025 (TSql170)
// Test basic JSON Index syntax in older versions
ParserTestUtils.ErrorTest160("CREATE JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(7, "SQL46010", "JSON"));
ParserTestUtils.ErrorTest150("CREATE JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(7, "SQL46010", "JSON"));
ParserTestUtils.ErrorTest140("CREATE JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(7, "SQL46010", "JSON"));
ParserTestUtils.ErrorTest130("CREATE JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(7, "SQL46010", "JSON"));
ParserTestUtils.ErrorTest120("CREATE JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(7, "SQL46010", "JSON"));
ParserTestUtils.ErrorTest110("CREATE JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(7, "SQL46010", "JSON"));

// JSON Index with FOR clause in older versions
ParserTestUtils.ErrorTest160("CREATE JSON INDEX idx1 ON table1 (jsonColumn) FOR ('$.name')",
new ParserErrorInfo(7, "SQL46010", "JSON"));

// JSON Index with WITH clause in older versions
ParserTestUtils.ErrorTest160("CREATE JSON INDEX idx1 ON table1 (jsonColumn) WITH (FILLFACTOR = 90)",
new ParserErrorInfo(7, "SQL46010", "JSON"));

@zijchen Z (zijchen) Jul 22, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove these, all tests against older parser versions are unnecessary here, they should be covered in Only170SyntaxTests already

Copilot AI Jul 22, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the ErrorTest160 calls from ParserErrorsTests.cs as they are unnecessary - older version rejection is already covered in Only170SyntaxTests. (commit b6073da)


// Test that UNIQUE and CLUSTERED/NONCLUSTERED are not allowed with JSON indexes in TSql170
TSql170Parser parser170 = new TSql170Parser(true);
ParserTestUtils.ErrorTest(parser170, "CREATE UNIQUE JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(14, "SQL46010", "JSON"));
ParserTestUtils.ErrorTest(parser170, "CREATE CLUSTERED JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(17, "SQL46010", "JSON"));
ParserTestUtils.ErrorTest(parser170, "CREATE NONCLUSTERED JSON INDEX idx1 ON table1 (jsonColumn)",
new ParserErrorInfo(20, "SQL46010", "JSON"));

// Test malformed JSON Index syntax in TSql170
// Missing column specification
ParserTestUtils.ErrorTest(parser170, "CREATE JSON INDEX idx1 ON table1",
new ParserErrorInfo(32, "SQL46029"));

// Empty FOR clause
ParserTestUtils.ErrorTest(parser170, "CREATE JSON INDEX idx1 ON table1 (jsonColumn) FOR ()",
new ParserErrorInfo(51, "SQL46029"));

// Invalid JSON path (missing quotes)
ParserTestUtils.ErrorTest(parser170, "CREATE JSON INDEX idx1 ON table1 (jsonColumn) FOR ($.name)",
new ParserErrorInfo(51, "SQL46010", "$"));

// Missing table name
ParserTestUtils.ErrorTest(parser170, "CREATE JSON INDEX idx1 ON (jsonColumn)",
new ParserErrorInfo(26, "SQL46010", "("));
}

/// <summary>
/// Check that the value of MAXDOP index option is within range
/// </summary>
Expand Down