Skip to content

refactor(isthmus): derive CREATE-statement table types from the provider - #1050

Open
nielspardon wants to merge 1 commit into
mainfrom
feat/provider-sourced-catalog-config
Open

refactor(isthmus): derive CREATE-statement table types from the provider#1050
nielspardon wants to merge 1 commit into
mainfrom
feat/provider-sourced-catalog-config

Conversation

@nielspardon

@nielspardon nielspardon commented Jul 31, 2026

Copy link
Copy Markdown
Member

Summary

Finishes the deferred half of @vbarua's comment on #1035
"I wonder if it would make sense to get the RelTypeFactory and SqlConverterBase from the
ConverterProvider as well"
. #1036 threaded the provider into the CalciteCatalogReader
construction; this covers the type derivation underneath it.

The gap

On the provider-injected path (processCreateStatementsToCatalog(provider, …)
processCreateStatementsToSchema(provider, …)createSubstraitTable(…)):

  • createSubstraitTable built the row type with SubstraitTypeSystem.TYPE_FACTORY, not the
    provider's type factory.
  • Column types came from col.dataType.deriveType(VALIDATOR), where VALIDATOR and
    EMPTY_CATALOG are static final, built on the global type factory plus
    SqlConverterBase.CONNECTION_CONFIG.

So a builder-configured typeFactory produced a catalog reader that used it while the tables
inside were typed by the global one. That is harmless today — getCalciteConnectionConfig()
returns the same case-insensitive config as SqlConverterBase.CONNECTION_CONFIG — but a custom
type factory carrying a different type system (different decimal precision limits, say) was
silently bypassed for table column types.

Changes

  • New private createEmptyCatalog(ConverterProvider) and createValidator(ConverterProvider)
    helpers. Because SubstraitSqlValidator takes its type factory from the catalog reader, fixing
    the empty catalog fixes column derivation as well.
  • The validator is built once per call and passed, with the provider's type factory, into
    createSubstraitTable.
  • EMPTY_CATALOG and VALIDATOR are kept and now defined via the same helpers against
    ConverterProvider.DEFAULT, so their values are unchanged and nothing public was removed.

Note on scope

Two things worth calling out explicitly, both easy to drop if you would rather they were separate:

  1. The validator is now built with converterProvider.getSqlOperatorTable() instead of a
    hardcoded SubstraitOperatorTable.INSTANCE, matching how feat(isthmus)!: inject ConverterProvider into the SQL statement parsers #1035 made the query path source its
    operator table. Identical for the default provider; it only matters for a provider that adds
    operators.
  2. EMPTY_CATALOG and VALIDATOR now have no callers inside the class either (each entry point
    derives its own from the provider). They are still public, so they are retained — but if you
    want them deprecated in favour of the provider-derived path, that is a one-line follow-up.

Independent of #1037 — no overlapping files, so the two can merge in either order.

🤖 Generated with AI

The builder work threaded the ConverterProvider into the CalciteCatalogReader
construction, but the type derivation underneath still used global defaults on
that same injected path:

- createSubstraitTable built the row type with SubstraitTypeSystem.TYPE_FACTORY
  instead of the provider's type factory.
- Column types came from col.dataType.deriveType(VALIDATOR), where VALIDATOR and
  EMPTY_CATALOG are static finals built on the global type factory and
  SqlConverterBase.CONNECTION_CONFIG.

So a builder-configured typeFactory produced a catalog reader that used it while
the tables inside were typed by the global one. Harmless today, since
getCalciteConnectionConfig() returns the same case-insensitive config, but a
custom type factory carrying a different type system was silently bypassed.

Add createEmptyCatalog(ConverterProvider) and createValidator(ConverterProvider)
helpers, build the validator once per call, and pass the provider's type factory
and that validator into createSubstraitTable. EMPTY_CATALOG and VALIDATOR are
retained and now defined via the same helpers against ConverterProvider.DEFAULT,
so their values are unchanged.

The validator is also built with the provider's SqlOperatorTable rather than a
hardcoded SubstraitOperatorTable.INSTANCE, matching how the query path already
sources its operator table. Identical for the default provider.

CreateStatementParserProviderConfigTest covers both entry points; 3 of its 4
tests fail without this change.
assertSame(
fieldType,
typeFactory.canonizeType(fieldType),
"column types should be interned in the provider's type factory");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice targeted coverage around both entry points. Non-blocking test thought: Calcite uses a static interner here, so canonizing a field type cannot distinguish the provider factory from the global factory. Recording createSqlType calls on RecordingTypeFactory, or asserting a custom type-system result, would make this check protect the validator path too.

@bvolpato bvolpato left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Clean provider-config propagation through CREATE parsing, with focused coverage for both entry points and defaults. Left one non-blocking test-strengthening note inline.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants