feat(isthmus)!: inject ConverterProvider into the SQL statement parsers by nielspardon · Pull Request #1035 · substrait-io/substrait-java

nielspardon · 2026-07-24T06:27:22Z

Summary

Thread a single ConverterProvider through both the CREATE-statement parsing that builds the
Calcite catalog and the query parsing, so one parser configuration (the provider's
getSqlParserConfig()) is applied consistently end-to-end. Previously the CREATE-statement path and
the query path could apply different parser settings, so the identifier casing used to build a
NamedScan name could disagree with the casing used to resolve it in a query.

Also adds a shared ConverterProvider.DEFAULT instance, used as the default for the no-arg parsing
entry points.

Breaking changes

The SqlParser.Config-based parsing entry points (public since v0.94.0) are removed in favour
of passing a ConverterProvider:

SubstraitSqlStatementParser.parseStatements(String, SqlParser.Config)
SubstraitSqlToCalcite.convertQueries(String, CatalogReader, SqlParser.Config) and the
5-arg SqlParser.Config overload

SqlToSubstrait.convert(String, CatalogReader, SqlDialect) is deprecated: the SqlDialect
argument no longer influences parsing and it delegates to convert(String, CatalogReader). Until the
builder arrives (next PR in the stack), parser configuration is customised by subclassing
ConverterProvider and overriding getSqlParserConfig().

Notable changes

Class	Change
`SubstraitSqlStatementParser`	`parseStatements(String, ConverterProvider)` owns the `SqlParser` instantiation; `SqlParser.Config` overload removed
`SubstraitSqlToCalcite`	New `convertQueries(sql, catalog, ConverterProvider[, operatorTable])`; internal overloads route through `ConverterProvider`; `SqlParser.Config` overloads removed
`SubstraitCreateStatementParser`	New `processCreateStatements(ConverterProvider, …)` / `processCreateStatementsToCatalog(ConverterProvider, …)` overloads — same parser config as query parsing
`SqlToSubstrait`	`convert(sql, catalog)` uses the `ConverterProvider` path; `convert(sql, catalog, SqlDialect)` `@Deprecated` (arg ignored)
`SqlExpressionToSubstrait`	Routes CREATE parsing through the provider
`ConverterProvider`	Adds the shared `DEFAULT` instance
`IsthmusEntryPoint` (CLI)	Threads its provider through catalog and query parsing consistently

Split of #983

Per @vbarua's request, #983 is split into three stacked PRs (review/merge bottom-up):

feat(isthmus)!: inject ConverterProvider into the SQL statement parsers #1035 — inject ConverterProvider into the SQL statement parsers (this PR)
feat(isthmus): introduce a ConverterProvider builder #1036 — introduce the ConverterProvider builder
feat(isthmus): configurable unquoted identifier casing via the ConverterProvider builder #1037 — configurable unquoted identifier casing via the builder

Replaces #983.

🤖 Generated with AI

Thread a single ConverterProvider through both the CREATE-statement parsing that builds the Calcite catalog and the query parsing, so one parser configuration (the provider's getSqlParserConfig()) is applied consistently end-to-end. Add a shared ConverterProvider.DEFAULT instance for the no-arg parsing entry points. The SqlParser.Config-based parsing entry points (public since v0.94.0) are removed in favour of passing a ConverterProvider: - SubstraitSqlStatementParser.parseStatements(String, SqlParser.Config) - SubstraitSqlToCalcite.convertQueries(..., SqlParser.Config) overloads SqlToSubstrait.convert(String, CatalogReader, SqlDialect) is deprecated: the SqlDialect argument no longer influences parsing and it delegates to convert(String, CatalogReader). Parser configuration is customised by subclassing ConverterProvider and overriding getSqlParserConfig().

vbarua · 2026-07-24T14:40:47Z

+      ConverterProvider converterProvider,
+      SqlOperatorTable operatorTable)
+      throws SqlParseException {
+    SqlValidator validator = new SubstraitSqlValidator(catalogReader, operatorTable);


Now that we're injecting the ConverterProvider, should we use the operatorTable available in ConverterProvider#getSqlOperator here?

good catch. pushed a commit that refactors this part

vbarua · 2026-07-24T14:54:30Z

+   * @return list of {@link SubstraitTable}s generated from the CREATE statements
   * @throws SqlParseException if parsing fails or statements are invalid
   */
+  public static List<SubstraitTable> processCreateStatements(


minor: would it make sense to move this under the original processCreateStatement from which it was extracted? Would also have made the diff easier to review.

pushed a commit that moves the method as suggested

vbarua · 2026-07-24T15:00:10Z

+        rootSchema,
+        defaultSchema,
+        SubstraitTypeSystem.TYPE_FACTORY,
+        SqlConverterBase.CONNECTION_CONFIG);


(not for this PR) I wonder if it would make sense to get the RelTypeFactory and SqlConverterBase from the ConverterProvider as well. The former is already configured in the ConverterProvider.

can probably include that in the builder PR

Revisiting this now that the builder PR (#1036) has merged — it landed about half of this. processCreateStatementsToCatalog(converterProvider, ...) now builds the CalciteCatalogReader with converterProvider.getTypeFactory() and converterProvider.getCalciteConnectionConfig().

What it did not cover is the type derivation underneath, on that same injected path: createSubstraitTable(...) still builds the row type with SubstraitTypeSystem.TYPE_FACTORY.createStructType(...), and the column types come from col.dataType.deriveType(VALIDATOR) where VALIDATOR/EMPTY_CATALOG are public static final and built on the global factory plus SqlConverterBase.CONNECTION_CONFIG. So a builder-configured typeFactory yields a catalog reader that uses it while the tables inside are typed by the global one — harmless today (the provider getter returns the same CASE_SENSITIVE=false config), but a custom type factory carrying a different type system would be silently bypassed.

Finishing it off in a focused follow-up PR rather than widening #1037, since making those two constants provider-derived is a public-API-shaped change worth its own review.

Opened #1050 for this.

…ibling Relocate processCreateStatements(ConverterProvider, String) to sit directly beneath the single-argument processCreateStatements(String) that delegates to it, grouping the two overloads together. Pure code move; no behavior change.

The SQL statement converters now derive the operator table from the injected ConverterProvider rather than accepting it as a separate parameter, so one provider configures both parsing and the valid operators end-to-end: - convertQuery/convertQueries build the validator with converterProvider.getSqlOperatorTable(), dropping the redundant convertQueries(ConverterProvider, SqlOperatorTable) overload and the SqlToSubstrait.operatorTable field that fed it. - Add convertQuery(String, CatalogReader, ConverterProvider) mirroring the plural overload, and migrate callers off passing the operator table by hand. - SqlExpressionToSubstrait's extended-expression validator now honors the provider's operator table too, matching the query path. - Memoize DynamicConverterProvider.getSqlOperatorTable() so reusing a converter across conversions no longer rebuilds the dynamic operator table each call. - Deprecate the raw-SqlOperatorTable convertQuery/convertQueries overloads in favour of the ConverterProvider ones.

vbarua

LGTM

I'll leave the merging to you. The PR message is a bit of mouthful for a commit message, especially because we'll try and generate the release notes from it as well.

## Summary Add `ConverterProvider.builder()` and a nested `Builder` — a readable, forward-compatible way to configure a `ConverterProvider`, most importantly to supply a full Calcite `SqlParser.Config` now that the parsers consume the provider's config (#1035). `ConverterProvider.DEFAULT_SQL_PARSER_CONFIG` (`SqlParser.Config.DEFAULT` + `TO_UPPER` + `SqlDdlParserImpl.FACTORY` + `LENIENT`) is exposed as the base for customized configs, e.g. `DEFAULT_SQL_PARSER_CONFIG.withUnquotedCasing(Casing.UNCHANGED)`. ## Sole construction path + subclassing seam All public `ConverterProvider` constructors are `@Deprecated` in favour of `builder()`, and `DEFAULT` is `builder().build()`. Subclassing routes through a new `protected ConverterProvider(Builder)` seam — the single place instances are created — so adding a component later touches only the builder and this constructor, never a subclass `super(...)` call. `DynamicConverterProvider` and `AutomaticDynamicFunctionMappingConverterProvider` construct via `super(builder)`. Now that the provider is fully configurable, call sites that still used global defaults are routed through it: the builder's `typeConverter` reaches the scalar/aggregate/window converters, and the CREATE-statement catalog reader uses the provider's type factory and connection config. ## Compatibility Source-compatible — no public API removed, deprecated constructors still delegate to the builder, and behavior is unchanged for the default provider. ## Stack (#983 split, merge bottom-up) 1. #1035 — inject `ConverterProvider` into the SQL statement parsers (merged) 2. **#1036 — this PR** 3. #1037 — configurable unquoted identifier casing via the builder

This was referenced Jul 24, 2026

feat(isthmus): introduce a ConverterProvider builder #1036

Merged

feat(isthmus): configurable unquoted identifier casing via the ConverterProvider builder #1037

Open

feat(isthmus)!: configurable SQL parser config via ConverterProvider builder #983

Closed

nielspardon requested a review from vbarua July 24, 2026 06:28

vbarua reviewed Jul 24, 2026

View reviewed changes

Comment thread isthmus/src/main/java/io/substrait/isthmus/SqlToSubstrait.java

vbarua reviewed Jul 24, 2026

View reviewed changes

nielspardon added 2 commits July 24, 2026 20:09

nielspardon requested a review from vbarua July 24, 2026 18:18

vbarua approved these changes Jul 24, 2026

View reviewed changes

nielspardon merged commit e018342 into main Jul 25, 2026
12 checks passed

nielspardon deleted the feat/inject-converterprovider-parsers branch July 25, 2026 06:25

nielspardon mentioned this pull request Jul 30, 2026

feat(isthmus)!: preserve aggregate output types and semantics through Calcite #1017

Open

nielspardon mentioned this pull request Jul 31, 2026

refactor(isthmus): derive CREATE-statement table types from the provider #1050

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(isthmus)!: inject ConverterProvider into the SQL statement parsers - #1035

feat(isthmus)!: inject ConverterProvider into the SQL statement parsers#1035
nielspardon merged 3 commits into
mainfrom
feat/inject-converterprovider-parsers

nielspardon commented Jul 24, 2026 •

edited

Loading

Uh oh!

vbarua Jul 24, 2026

Uh oh!

nielspardon Jul 24, 2026

Uh oh!

Uh oh!

vbarua Jul 24, 2026

Uh oh!

nielspardon Jul 24, 2026

Uh oh!

vbarua Jul 24, 2026

Uh oh!

nielspardon Jul 24, 2026

Uh oh!

nielspardon Jul 31, 2026

Uh oh!

nielspardon Jul 31, 2026

Uh oh!

vbarua left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

nielspardon commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Breaking changes

Notable changes

Split of #983

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vbarua left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nielspardon commented Jul 24, 2026 •

edited

Loading