feat(isthmus)!: preserve aggregate output types and semantics through Calcite

[rkondakov]

Fixes #1016.

What

A Substrait aggregate used to lose part of itself on the way to Calcite, and could not be
recovered on the way back. This PR gives an aggregate a resolved binding that travels with it, so
the plan's declared output type, its function options, its aggregation phase and the exact
extension declaration all survive a Substrait → Calcite → Substrait round trip.

Why — what was lost

	Before	After
Declared output type	Passed to AggregateCall.create, but RelBuilder re-infers a call's type from its operator, so Calcite's inference won. sum(DECIMAL(10,2)), which sum:dec declares as DECIMAL(38,2), came back with the argument's own width.	The plan's type is preserved.
Two measures, same shape, different types	AggregateCall.equals ignores the stored type and RelBuilder deduplicates, so they collapsed into one column — the output arity changed.	Both survive; deduplication is switched off for exactly those aggregates.
Function options	No place for them in a Calcite aggregate call, so count(a) with overflow: ERROR came back without the option, and two counts differing only by an option were indistinguishable.	Carried on the binding and restored.
Aggregation phase	Converting back always produced INITIAL_TO_RESULT. A distributed plan with partial aggregates was silently turned into a plan of full aggregations.	The phase is carried and restored.
Which declaration	Re-matched against the extension catalog; among equally-shaped declarations it could only guess. A phase consuming an intermediate state could not be matched at all, because the call's operands are then accumulator state rather than the declaration's arguments.	The exact declaration the plan used is taken from the binding.
Global aggregate with no groupings	Built with zero grouping sets, so measures were re-inferred as if there were no empty group.	Built as one empty grouping set — the correct global aggregation.

How

Each converted measure resolves a ResolvedAggregateBinding, which captures the semantic identity
of the invocation: anchor, kind-aware arguments (value / type / enum, including the selected enum
option), options, phase and invocation semantics. Where Calcite cannot hold what the binding
carries — the chosen output type differs from its inference, or the invocation has options or a
phase other than INITIAL_TO_RESULT — the aggregate operator is wrapped so that both the binding
and the type travel with it. AggregateFunctionConverter then rebuilds the invocation from that
binding instead of re-matching the operator.

The binding records the plan as it was converted, so it is dropped again when a planner rule has
since changed the call's arguments, and DISTINCT is always read off the Calcite call, because a
rule may legitimately have removed a redundant one.

Supporting this needed TypeExpressionEvaluator to actually work: it used to throw
UnsupportedOperationException("NYI") for anything but an already concrete return type. It now
binds numbered wildcards (the any1 of min(any1) -> any1) and integer type parameters (the P
and S of DECIMAL<P,S>) from the actual argument types and substitutes them, honouring variadic
parameter consistency and the MIRROR nullability policy. Unsupported derivations — arithmetic
derivations, if/then, return programs — stay fail-closed and never fall back to a caller-supplied
type, which is what makes a derived type trustworthy enough to validate a plan against.

Commits

1. feat(core) — the resolved-binding model (ResolvedArgument, ResolvedFunctionBinding,
   ResolvedAggregateBinding), FunctionBindingResolver (resolve vs. opt-in validate), and a
   working TypeExpressionEvaluator. Self-contained; :core:build and :isthmus:build are green
   at this commit alone.
2. feat(isthmus)! — conversion changes: AggregateConversion configuration, the transport
   wrapper in AggregateFunctions, and the two conversion fixes (deduplication, global aggregate).

New configuration

AggregateConversion has two independent settings:

• OutputTypeSource — PLAN_OUTPUT (new default) preserves the plan's declared type;
  CALCITE_INFERENCE reproduces the previous behavior.
• FunctionBindingValidation — NONE (default) does not check the plan against the extension
  declaration; EXTENSION_DECLARATION requires the declared output type to match the derived one
  and rejects the plan otherwise.

The default is PLAN_OUTPUT + NONE: conversion never silently changes a type, but it also does
not assert that the plan is spec-compliant. Strict validation is deliberately opt-in, because
type derivation is fail-closed and would reject any function whose return expression it does not
yet support.

Breaking changes / migration

• A converted aggregate now carries the plan's declared output type instead of Calcite's inferred
  one. Pass AggregateConversion with OutputTypeSource.CALCITE_INFERENCE to SubstraitToCalcite
  to restore the previous behavior.
• The resulting AggregateCall may carry a wrapper operator rather than the plain SqlAggFunction.
  Consumers comparing the operator by identity should call AggregateFunctions.unwrapBound first;
  AggregateFunctions.boundBinding exposes the binding.
• Rollup and splitting are disabled for a wrapped call: both re-infer the transformed call's type
  from the underlying function and would discard the preserved one. Rules that match on SqlKind
  instead (e.g. AGGREGATE_REDUCE_FUNCTIONS) have no such hook and may still rewrite the call and
  drop the type.
• A ConverterProvider subclass overriding getSubstraitRelNodeConverter(RelBuilder) should
  override the new getSubstraitRelNodeConverter(RelBuilder, AggregateConversion) as well;
  otherwise its customization is skipped for a non-default configuration.
• Converting a plan whose aggregate has no groupings back to Substrait now yields a single empty
  grouping instead of none — the same global aggregation, spelled the way Calcite spells it.

Open question for maintainers

Should preserving the plan's declared output type be the default, or opt-in?

This PR makes PLAN_OUTPUT the default, so existing callers get the new behavior without changing
a line. The reasoning: a converter silently changing a plan's declared type is a correctness
problem — DECIMAL(38,2) becoming DECIMAL(10,2) changes results, not just metadata — and a
producer that has to know about a configuration flag to get its own types back is a sharp edge.

The cost is real, though: the new default is a behavior change for every existing consumer, and
where the declared type diverges from Calcite's inference the operator is wrapped, which disables
rollup and aggregate splitting for that call and makes call.getAggregation() == SUM-style
comparisons fail.

The alternative is to default to CALCITE_INFERENCE — byte-for-byte the previous behavior — and
let callers opt into PLAN_OUTPUT. That keeps every existing consumer untouched and makes the
wrapper strictly opt-in, at the price of leaving the type-loss bug on by default.

Happy to flip the default either way; the configuration supports both.

Testing

• :core — FunctionBindingResolverTest, ResolvedAggregateBindingTest,
  TypeExpressionEvaluatorTest, plus a small binding_extensions.yaml for wildcard cases.
• :isthmus — new cases in SubstraitRelNodeConverterTest.Aggregate covering declared decimal
  width, global aggregation, operator identity, enum arguments, options, phases (including a
  parameterized intermediate state), duplicate measures, rollup and split rules, both validation
  modes, unwrapping, and converter-factory dispatch.
• ./gradlew :core:build :isthmus:build is green at each of the two commits.

🤖 Generated with Claude Code

[rkondakov]

@nielspardon This PR lets aggregate output types come either from the plan or from Calcite’s inference.

I defaulted to the plan’s type because re-inference can change results, e.g. DECIMAL(38,2) to DECIMAL(10,2). The downside is that mismatches require a wrapper operator, which can break operator identity checks and disable rollup or aggregate splitting.

Keeping CALCITE_INFERENCE preserves current behavior but leaves type loss enabled by default.

Which default do you prefer?

[nielspardon]

Really nice piece of work — the problem statement is unusually clear and the design premise holds up.
I checked the two Calcite mechanics it rests on, because if either were wrong the wrapper would be
unnecessary complexity:

• RelBuilder.AggCallImpl2.aggregateCall(...) re-creates every pre-built AggregateCall with
  type = null (RelBuilder.java:4766-4773), so an explicit type genuinely cannot survive
  RelBuilder.aggregate.
• More fundamentally, Aggregate's primary constructor asserts
  typeMatchesInferred(aggCall, Litmus.THROW) (Aggregate.java:177), which Gradle test JVMs enable.
  So a plain SqlAggFunction carrying a divergent stored type fails an assertion — the type must
  travel on the operator's inference rule, and building a LogicalAggregate directly wouldn't help
  either. Worth putting that in BoundSqlAggFunction's Javadoc; it's a stronger justification than
  the RelBuilder one currently given.

I also validated the behaviour against the Substrait spec (local v0.98.0 checkout + upstream issue
history). Most of it is conformant, and two of the things that look like bugs here are actually
spec bugs — I've filed those upstream so they don't block you:

• substrait-io/substrait#1151 — the spec contradicts itself on arguments/output_type for
  non-INITIAL_TO_RESULT phases. algebra.proto:1870-1872 says "exactly the number of arguments
  specified in the function definition" while :1855-1859 says the inputs of an
  INTERMEDIATE_TO_* phase are the intermediate values. Both were added by the same commit
  (d4cfbe0, #231), and the author had described the intended [X] -> Y / [Y] -> Y / [Y] -> Z
  model on #257 eight hours earlier. Your core validateIntermediateSignature is the reading
  upstream intended; it's alignArguments that should move (see inline).
• substrait-io/substrait#1150 — quantile's return: LIST?<any> uses a plain any, which carries
  no identity, so its output type is underivable. It's the only standard-catalog aggregate your
  deriveOutputType fails on — worth listing as a known limitation and linking that issue.

Spec-conformant and verified, so no change needed: plain-any-binds-independently /
any1-identity; equalsIgnoringNullability (scalar_functions.md:128 licenses stripping only the
outermost nullability); the MIRROR override in both directions (scalar_functions.md:105 +
simple_extensions_schema.yaml:187-189 — "they will be ignored"; and tests/cases/boolean/or.test:5
asserts or(bool, bool) = bool independently); default MIRROR; DISCRETE exact matching; option
name/value case-insensitivity (algebra.proto:1876-1881 mandates it — pre-existing EnumArg.of is
the non-conformant one); unknown-option rejection; empty-preference rejection; variadic min
semantics; the INCONSISTENT branch; zero-groupings = one empty grouping set; and rejecting a
partial phase on a non-decomposable declaration.

On your open question — yes, keep PLAN_OUTPUT as the default. Silently narrowing
DECIMAL(38,2) to DECIMAL(10,2) changes results, and asking producers to know about a flag to get
their own types back is the sharper edge. Two things also make it less risky than the description
suggests: the "unrepresentable declared type throws" behaviour is unchanged from before (the old code
converted the declared type unconditionally too), and the wrapper is only added when the type
actually diverges or the invocation carries options/a phase.

About the length of this review

Up front: there are a lot of comments below, and I don't want that to read as a verdict on the
change. It isn't. This PR touches the function-binding, type-derivation and aggregate-conversion
surfaces all at once, and adds ~1000 non-test lines of new :core API — the comment count tracks
that surface area, not the quality of the work. Most items are a few lines; five come with a
ready-to-apply suggestion block and two more carry the replacement code inline.

Here's what I'd actually gate on, so the rest can be triaged or deferred without guessing:

Blocks merge — silent wrong answers (4)

#	What
2	alignArguments drops the binding for any decomposable declaration whose arity isn't 1, so a partial count(*) silently comes back as a full count(i64).
3	UNSPECIFIED phase is treated as a full aggregation, contradicting algebra.proto:1834.
5	The splitter opt-out misses SqlSingletonAggFunction, so AggregateRemoveRule silently drops the binding's phase and options.
8	typeMatches's return true fall-through means EXTENSION_DECLARATION accepts plans the spec forbids. At minimum make the fallback throw.

Worth doing in this PR (6) — 1 (wildcard predicates, spec conformance on new API), 4 (terminal
else), 6 (unwrapBound vs Calcite's assertion), 7 (unconditional inference), 10 (derive vs
resolveType, plus retargeting the five evaluator tests at the production path), 11 (the two
coverage Javadocs, which currently mis-describe what fails).

Fine as follow-ups (3 + the batched lists) — 9 (the intra-PR duplication), 12 (dead
required() guard), 13 (Immutables nit), the small-things list, and the ConverterProvider
architectural point, which is really a conversation about how it lands alongside #1035/#1036 rather
than a change to make here.

Of the missing tests, only the zero-arg count round trip (comment 2) is load-bearing — it's the
case that proves the fix. The rest can follow.

The recurring theme across the blocking four is the same: a few predicates and matchers are
permissive in ways that silently accept a plan rather than fail loudly. That matters more here than
it normally would, because EXTENSION_DECLARATION is sold as a validation mode — a validator that
returns true on a shape it doesn't understand is worse than one that refuses.

Happy to split this into two reviews (correctness now, polish later) if that's easier to work
through, or to pair on any of them.

---

---

Small things

• alignArguments: hoist the loop-invariant if (!declaration.variadic().isPresent()) out of the
  while.
• validateOptions lowercases declared option names into a map per call; two declared names
  differing only in case collide silently.
• Strict validation derives the intermediate type twice per measure
  (validateIntermediateSignature plus outputType()).
• ParameterBindings.isInteger and ReturnTypeEvaluator.resolveInteger both do
  try { Integer.parseInt } catch on the same token — one helper returning OptionalInt would do.
  The .trim() is dead: SubstraitLexer.g4:10 sends whitespace to channel(HIDDEN).
• bindInteger silently skips a numeric token rather than checking it against the actual value, so
  a declared DECIMAL<P,0> (factorial, functions_arithmetic_decimal.yaml:166-169) never
  verifies that the actual scale is 0.
• Tests: assertEquals(false, ...) / assertEquals(true, ...) in
  appliesMirrorNullabilityForScalars -> assertFalse/assertTrue; the three "must not throw"
  tests read better as assertDoesNotThrow; helper methods are interleaved with @Test methods in
  both new test classes.
• AggregateFunctions imports org.checkerframework...Nullable, which nothing else in
  isthmus/src/main uses (core uses jspecify, and this PR uses jspecify in ResolvedArgument).
  Overriding equals(Object)/unwrap/getRollup doesn't need it.
• Architectural, worth a conversation rather than a change here: AggregateConversion is threaded as
  a SubstraitToCalcite constructor parameter plus a second ConverterProvider factory overload
  plus an isDefault() dispatch branch. ConverterProvider is already the config carrier and is
  gaining a Builder in the stacked #1035/#1036 series — putting it there removes the extra
  overload, the isDefault() branch, the "override both" warning, and the
  new SubstraitToCalcite(provider, null, conversion) shape the tests are forced into.

---

Missing tests (batched comment)

• zero-arg count @ INTERMEDIATE_TO_RESULT round trip (comment 2) — the interesting phase case.
• a declared list<any1> against a non-list actual, and h(list<i32>, list<i32?>) (comment 8).
• UNSPECIFIED phase (comment 3).
• CALCITE_INFERENCE + EXTENSION_DECLARATION together.
• a grouping-sets plan containing an explicit empty grouping — exercises the hasEmptyGroup mirror
  on the non-global path, currently only covered via globalAggregation.
• the documented "AGGREGATE_REDUCE_FUNCTIONS may still drop the type" caveat isn't pinned, so a
  future Calcite bump won't reveal if it degrades.
• boundary test for the wildcard predicates (comment 1).

[nielspardon]

The spec bounds a numbered wildcard to exactly one digit, and both predicates disagree with it.

site/docs/extensions/index.md:117 writes the family as any[\d]? and :144 enumerates it as
"any1, any2, ..., any9". The shipped lexer agrees — grammar/SubstraitLexer.g4:63-64:

Any:    'ANY';
AnyVar: Any [0-9];

I ran that lexer over the variants. Because Identifier wins on maximal munch, any10 and
any123 are ordinary named type parameters, not wildcards:

any      -> Any('any')          any10    -> Identifier('any10')
any1     -> AnyVar('any1')      any123   -> Identifier('any123')
ANY2     -> AnyVar('ANY2')      anything -> Identifier('anything')

So isNumberedWildcard() over-accepts (any10, any123), and the pre-existing isWildcard()
over-accepts more broadly (anything, anywhere, any1x, any10). Two extra wrinkles:

• Character.isDigit is Unicode-aware, so any٢ (Arabic-Indic two) currently returns true.
• Worth fixing isWildcard() in the same change, because its real damage isn't matching — it's
  identity. ToTypeString.java:226 has the same startsWith("any") test and returns the literal
  "any", so a third-party f(anything) and f(any) compute the same signature key. Per
  index.md:117 only any[\d]? maps to any.

Suggested change

	return value().toLowerCase(Locale.ROOT).startsWith("any");
	}
	@Override
	public boolean isNumberedWildcard() {
	String literal = value().toLowerCase(Locale.ROOT);
	if (!literal.startsWith("any") || literal.length() == "any".length()) {
	return false;
	}
	for (int i = "any".length(); i < literal.length(); i++) {
	if (!Character.isDigit(literal.charAt(i))) {
	return false;
	}
	}
	return true;
	}
	return WILDCARD.test(value());
	}
	@Override
	public boolean isNumberedWildcard() {
	return NUMBERED_WILDCARD.test(value());
	}

The two constants go at the top of StringLiteral (PMD FieldDeclarationsShouldBeAtStartOfClass):

    // Transcribed from the Substrait type grammar's lexer, which is caseInsensitive:
    //     Any:    'ANY';
    //     AnyVar: Any [0-9];
    // Exactly one digit: "any10" lexes as an Identifier, i.e. an ordinary named type parameter
    // rather than a wildcard (spec v0.98.0, site/docs/extensions/index.md).
    // \A..\z rather than ^..$ because asPredicate() is find()-based, and asMatchPredicate() is
    // Java 11 while this module compiles with options.release = 8.
    private static final Predicate<String> WILDCARD =
        Pattern.compile("\\Aany[0-9]?\\z", Pattern.CASE_INSENSITIVE).asPredicate();
    private static final Predicate<String> NUMBERED_WILDCARD =
        Pattern.compile("\\Aany[0-9]\\z", Pattern.CASE_INSENSITIVE).asPredicate();

Add java.util.function.Predicate and java.util.regex.Pattern; java.util.Locale (line 4)
becomes unused and spotless will strip it. There's precedent for the idiom three files over —
SimpleExtension.java:53-54 uses Pattern.compile(...).asPredicate() in a
private static final Predicate<String>.

Two things to flag rather than silently absorb:

1. Tightening isWildcard() is not a no-op. FunctionConverter.java:829-833 returns true
   unconditionally for a wildcard target, and IgnoreNullableAndParameters.java:524-526 returns
   false for a StringLiteral, so a declaration named anything flips from "matches everything"
   to "matches nothing". No shipped extension is affected (the catalog only uses any, any1,
   any2), but it belongs in the PR body.
2. If you'd rather keep isWildcard() untouched in this PR, splitting it out is fine — but then
   please still fix isNumberedWildcard(), since that one is new API and cheap to get right now.

(Optional aside: prose enumerates any1…any9 while the grammar admits any0. \Aany[0-9]?\z
accepts any0, siding with the grammar, which I think is the right call.)

[nielspardon]

alignArguments contradicts validateIntermediateSignature, and loses a legal plan because of it.

This walks declaration.args() and demands one operand per non-enum declared argument. But your own
core contract (FunctionBindingResolver.validateIntermediateSignature) says an INTERMEDIATE_TO_*
invocation carries exactly one operand — the accumulator state — regardless of the declaration's
arity. The two disagree, and the spec (per substrait-io/substrait#1151 and #257) is on core's side.

Reachable with the standard catalog: functions_aggregate_generic.yaml:19-26 is the
record-counting count overload — zero declared args, decomposable: MANY, intermediate: i64.
In INTERMEDIATE_TO_RESULT, fromMeasure emits COUNT(state) with one operand; here the loop runs
zero iterations, the trailing while finds variadic() absent and returns empty, the binding is
discarded, and convert falls through to getFunctionFinder. That re-matches against the other
count impl (args: [x: any], return: i64), which accepts an i64 operand — so a partial
count(*) silently comes back as a full count(i64) @ INITIAL_TO_RESULT. No exception.

Both existing tests use arity-1 declarations (count:any, avg:dec) where the alignment
coincidentally works, which is why this isn't caught.

Suggested change

	List<ResolvedArgument> resolved = binding.function().arguments();
	List<FunctionArg> arguments = new ArrayList<>();
	int operandIndex = 0;
	List<ResolvedArgument> resolved = binding.function().arguments();
	// A phase that consumes the intermediate state takes exactly that state as its single value
	// argument, whatever arity the declaration itself has. FunctionBindingResolver's
	// validateIntermediateSignature already enforces that contract; aligning against
	// declaration.args() here contradicts it, and silently drops the binding for a decomposable
	// declaration whose arity is not 1 (e.g. the record-counting count() overload: zero declared
	// arguments, intermediate i64).
	if (binding.consumesIntermediateState()) {
	return operands.size() == 1
	? Optional.of(new ArrayList<FunctionArg>(operands))
	: Optional.empty();
	}
	List<FunctionArg> arguments = new ArrayList<>();
	int operandIndex = 0;

Please also add a test for the zero-arg count @ INTERMEDIATE_TO_RESULT round trip — it's the case
that proves the fix.

Separately: spark/src/main/scala/.../ToAggregateFunction.scala + ToLogicalPlan.scala already
implement a third position (declared arguments in every phase, phase as orthogonal metadata), and
ToLogicalPlan.scala:89-92 has a // HACK - count() needs to be rewritten as count(1) for this
exact declaration. Whichever model wins, spark will need to follow — worth a sentence in the PR body
so a reviewer who knows that module isn't surprised.

[nielspardon]

AGGREGATION_PHASE_UNSPECIFIED implies INTERMEDIATE_TO_RESULT per the proto.

algebra.proto:1833-1835:

enum AggregationPhase {
  // Implies `INTERMEDIATE_TO_RESULT`.
  AGGREGATION_PHASE_UNSPECIFIED = 0;

git blame shows that comment was added deliberately by d4cfbe0 (#231, "fix: specify how function
arguments are to be bound") — it's not a leftover. Right now UNSPECIFIED falls through both helpers
as if it were a full aggregation, and there's already an in-repo precedent for the correct handling:
spark/src/main/scala/io/substrait/spark/expression/ToAggregateFunction.scala:77-78 has
case SExpression.AggregationPhase.UNSPECIFIED => Final // UNSPECIFIED implies INTERMEDIATE_TO_RESULT.

Suggested change

	public boolean producesIntermediateState() {
	return phase() == Expression.AggregationPhase.INITIAL_TO_INTERMEDIATE
	|| phase() == Expression.AggregationPhase.INTERMEDIATE_TO_INTERMEDIATE;
	}
	/**
	* Returns whether this phase consumes the declaration's intermediate state instead of its
	* declared arguments. The arguments of such an invocation are intermediate values, so they are
	* not expected to match the declaration's argument list.
	*
	* @return {@code true} for the intermediate-to-intermediate and intermediate-to-result phases
	*/
	public boolean consumesIntermediateState() {
	return phase() == Expression.AggregationPhase.INTERMEDIATE_TO_INTERMEDIATE
	|| phase() == Expression.AggregationPhase.INTERMEDIATE_TO_RESULT;
	}
	public boolean producesIntermediateState() {
	return phase() == Expression.AggregationPhase.INITIAL_TO_INTERMEDIATE
	|| phase() == Expression.AggregationPhase.INTERMEDIATE_TO_INTERMEDIATE;
	}
	/**
	* Returns whether this phase consumes the declaration's intermediate state instead of its
	* declared arguments. The arguments of such an invocation are intermediate values, so they are
	* not expected to match the declaration's argument list.
	*
	* <p>An unspecified phase implies the intermediate-to-result phase, per the {@code
	* AggregationPhase} proto enum, so it consumes an intermediate state too.
	*
	* @return {@code true} for the intermediate-to-intermediate and intermediate-to-result phases,
	* and for an unspecified phase
	*/
	public boolean consumesIntermediateState() {
	return phase() == Expression.AggregationPhase.INTERMEDIATE_TO_INTERMEDIATE
	|| phase() == Expression.AggregationPhase.INTERMEDIATE_TO_RESULT
	|| phase() == Expression.AggregationPhase.UNSPECIFIED;
	}

carriesOpaqueSemantics in SubstraitRelNodeConverter (line 417) needs the matching treatment —
see comment 7.

[nielspardon]

A dropped argument silently misaligns three positional consumers.

The instanceof chain has no terminal else, so an unrecognised FunctionArg shortens the list.
That list is consumed strictly positionally in three places — validateSignature
(declared.get(Math.min(i, ...)) vs arguments.get(i)), valueAndTypeArgumentTypes feeding
bindParameters, and isthmus alignArguments — and the third degrades to Optional.empty()
rather than failing. The chain covers all three subtypes today, so this is a latent extension
hazard rather than a live bug.

The repo's convention for this dispatch is FunctionArg.accept(fnDef, argIdx, FuncArgVisitor, ctx)
(AGENTS.md), and FuncArgVisitor has no default methods, so a new kind becomes a compile error.
The call site has everything it needs — AggregateFunctionVariant extends Function, plus
EmptyVisitationContext.INSTANCE — exactly as AggregateFunctionProtoConverter does over the same
object. Core does use instanceof for FunctionArg elsewhere, so the criticism isn't "used
instanceof" but "used instanceof without the fail-loud else" that the other sites have
(ExpressionRexConverter throws IllegalStateException, FunctionArg throws
UnsupportedOperationException).

Minimum fix:

Suggested change

	}
	} else {
	throw new IllegalArgumentException("Unsupported function argument kind: " + arg);
	}

[nielspardon]

The splitter opt-out is keyed on exact class identity, so one rule slips through.

AggregateMergeRule and AggregateJoinTransposeRule ask for SqlSplittableAggFunction.class
exactly, so the opt-out works for them. But AggregateRemoveRule asks for the supertype
SqlSingletonAggFunction.class (AggregateRemoveRule.java:97 and :142), falls through to
delegate.unwrap(...), and SqlSumAggFunction.unwrap returns SumSplitter.INSTANCE for it
(clazz.isInstance(...) is true for both interfaces, since
SqlSplittableAggFunction extends SqlSingletonAggFunction).

That rule flattens the Aggregate into a Project. It casts to aggCall.type, so the type survives —
but the binding is gone, so the phase and options are silently dropped, which is exactly what
this opt-out exists to prevent.

Suggested change

	if (clazz == SqlSplittableAggFunction.class) {
	// Splitting (e.g. pushing an aggregate through a join) re-infers the split calls' types
	// from the underlying function, discarding the canonical type. Opt out until it is
	// binding-aware.
	return null;
	}
	if (SqlSingletonAggFunction.class.isAssignableFrom(clazz)) {
	// Splitting (e.g. pushing an aggregate through a join) re-infers the split calls' types
	// from the underlying function, discarding the canonical type. Opt out until it is
	// binding-aware. This covers the SqlSingletonAggFunction supertype as well, because
	// AggregateRemoveRule unwraps that rather than SqlSplittableAggFunction, and flattening
	// the aggregate would drop the binding's phase and options.
	return null;
	}

Needs import org.apache.calcite.sql.SqlSingletonAggFunction; (present in Calcite 1.42).

While here: if (clazz.isInstance(binding)) sits before super.unwrap, so unwrap(Object.class)
returns the binding rather than the operator. Tightening to
clazz == ResolvedAggregateBinding.class would be safer.

[nielspardon]

derive and resolveType are the same call with different answers.

After this PR's own one-line edit, SimpleExtension.Function.resolveType (SimpleExtension.java:657-659)
is evaluateExpression(returnType(), args(), variadic(), types) — byte-for-byte what derive
calls inline. The only difference is that derive applies the MIRROR nullability policy, and MIRROR
is the default (SimpleExtension.java:525-526, and scalar_functions.md:12 confirms the spec
default is MIRROR).

So resolveType is non-conformant for every MIRROR declaration, and the spec's own worked example
says so: scalar_functions.md gives add(i32, i32) -> i32 with MIRROR, and add(i32?, i32) must
return i32? — resolveType returns i32.

Two consequences worth acting on:

• Teach resolveType the nullability policy and have deriveOutputType delegate to it (that
  direction, since derive also wraps UnsupportedOperationException into
  InvalidFunctionBindingException, which two of your tests assert).
• Retarget TypeExpressionEvaluatorTest: all five tests call resolveType, i.e. the path
  production won't use, and every fixture they touch (sum:i32, sum:dec, nullif:any_any — all
  DECLARED_OUTPUT; coalesce:any with required args) is a case where the two paths coincide.
  resolveType has zero main-source callers in the repo.

Minor: derive drops the cause when re-wrapping (new InvalidFunctionBindingException(... e.getMessage())). Adding a (String, Throwable) constructor would keep the stack.

[nielspardon]

Both Javadocs describe the unsupported set as "arithmetic derivations, if/then, return
programs", and that's not what actually fails.

I ran the repo's own parser over every return:/intermediate: string in the catalogued YAMLs:

• 502 returns: 395 concrete, 22 derivable, 85 throw — dominated by varchar<L1> (24),
  precision_timestamp<P> (11), precision_timestamp_tz<P> (10), fixedchar<L1> (7),
  list<...> (7), interval_day<P> (7), precision_time<P> (4), plus 15 |- programs.
• 60 intermediates: 51 derivable, 9 throw. deriveIntermediateType is not dead — all the
  avg STRUCT<i64,i64> intermediates work, because an all-concrete struct parses to a concrete
  Type.Struct and hits the instanceof Type shortcut.
• There is no if/then derivation anywhere in the standard extensions.

So the practical blockers are the parameterized type classes, not exotic derivations. concat,
concat_ws, assume_timezone and strptime_* all fail, which a reader of the current wording
wouldn't expect. The EXTENSION_DECLARATION Javadoc is the worse of the two because its list has no
trailing ellipsis, so it reads as exhaustive.

Please replace both lists with the shapes that actually fail (varchar<L1>, fixedchar<L1>,
precision_time/timestamp/timestamp_tz<P>, interval_day<P>, list<anyN>, parameterized structs,
and multi-line return programs), and mention that quantile (LIST?<any>) is the one standard
aggregate that cannot be derived — cross-referencing substrait-io/substrait#1150, since that one is
a spec bug rather than a gap here.

[nielspardon]

The declared.required() guard is unreachable.

All three Argument implementors hard-code return true (SimpleExtension.java:237-240,
:271-274, :308-311), the generated ImmutableSimpleExtension.EnumArgument.Builder has no
required setter, and simple_extensions_schema.yaml's enumeration_arg declares
additionalProperties: false with only name/description/options — so there's no key to make
it settable. The throw is live and tested; it's the permissive fall-through that's dead.

This is a genuine spec layering mess rather than your bug: algebra.proto:1882-1885 still says
"Optional enum arguments must be bound using FunctionArgument.enum followed by either
Enum.specified or Enum.unspecified", so the proto describes optional enum arguments that the YAML
schema can no longer express. So I wouldn't delete unspecifiedEnumOption() — the proto path can
still produce one.

What I would change: either drop the if (declared.required()) branch (and the two Javadoc
sentences in this class and on ResolvedArgument.unspecifiedEnumOption() that describe a state the
model can't represent), or make EnumArgument.required() reflect reality. As written, the docs
promise a behaviour that can't occur.

[nielspardon]

Hand-written value class where its two same-PR siblings (ResolvedFunctionBinding,
ResolvedAggregateBinding) are @Value.Immutable with @Value.Check. 28 of ~69 lines are
equals/hashCode/toString/factories that Immutables would generate; FileOrFiles is the
precedent for the nested-discriminator-plus-Optional shape. Not blocking.

To be clear, this is not duplicating FunctionArg: Type extends FunctionArg, so a Type inside
a List<FunctionArg> already means "type argument" and you couldn't express "value argument
identified only by its type" there. That erasure is load-bearing for the staleness check, which
compares rebuilt ResolvedArguments rather than whole Expressions.

[nielspardon]

Same issue as my comment on TypeExpressionEvaluator's class Javadoc: this list ("arithmetic derivations, if/then, return programs") doesn't describe what actually fails, and here it has no trailing ellipsis so it reads as exhaustive. There is no if/then derivation anywhere in the standard extensions; the real blockers are the parameterized type classes (varchar<L1>, fixedchar<L1>, precision_time/timestamp/timestamp_tz<P>, interval_day<P>, list<anyN>, parameterized structs) plus the multi-line return programs. Since this enum is what a user reads to decide whether strict validation is adoptable for their catalog, it's worth being precise: concat, concat_ws, assume_timezone and strptime_* are all rejected today.