Render non-String scalars in jsonPathString without JSON quotes - #19029
Merged
Jackie-Jiang merged 1 commit intoJul 21, 2026
Merged
Conversation
jsonPathString and its fast variants stringified any non-String value via JsonUtils.objectToString, which wraps a java.util.UUID (and other non-JSON-native scalars a record extractor can materialize) in JSON string quotes. Extracting an Avro uuid field, for example, produced a double-quoted "657ae8f8-..." instead of the bare UUID. Route the resolved value through a shared jsonValueToString helper: UUID / LocalDate / LocalTime render via their natural (canonical / ISO-8601) toString, and everything else - numbers, Boolean, Timestamp (epoch millis), byte[], and Map / Collection / array containers - continues through JsonUtils.objectToString, matching the json-path-to-string behavior of jsonExtractScalar.
Jackie-Jiang
added
bug
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #19029 +/- ##
============================================
+ Coverage 57.10% 65.40% +8.29%
- Complexity 7 1405 +1398
============================================
Files 2629 3423 +794
Lines 156315 215967 +59652
Branches 25533 34184 +8651
============================================
+ Hits 89271 141245 +51974
- Misses 59423 63339 +3916
- Partials 7621 11383 +3762
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
xiangfu0
approved these changes
Jul 21, 2026
Contributor
|
Follow-up docs PR opened in pinot-contrib/pinot-docs: pinot-contrib/pinot-docs#932 |
xiangfu0
added a commit
to pinot-contrib/pinot-docs
that referenced
this pull request
Jul 21, 2026
## Summary - document the `JSONPATHSTRING` compatibility change from apache/pinot#19029 - clarify that unquoted `UUID`, `LocalDate`, and `LocalTime` rendering only applies when ingestion transforms read already-materialized object trees - note that `JSONPATHSTRINGFAST` and `JSONPATHSTRINGFIRSTMATCH` keep identical results ## Validation - cross-checked the behavior against `/Users/xiangfu/claude-workspace/pinot-doc-expert/pinot` and merged upstream PR apache/pinot#19029 - ran `git diff --check`
28 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
jsonPathString(and its fast variantsjsonPathStringFast/jsonPathStringFirstMatch) stringified any non-Stringresolved value viaJsonUtils.objectToString. Jackson serializes several non-JSON-native Java types — notablyjava.util.UUID— as a quoted JSON string, so extracting such a value produced a double-quoted result. For example, an Avrouuidlogical-type field extracted withJSONPATHSTRING(after, '$.customer_id')yielded"657ae8f8-..."(with literal quotes) instead of the bare UUID.These types never arise from parsing a JSON string (Jackson only yields
Boolean/Integer/Long/Double/String/null/List/Map); they only appear when the function operates on an already-materialized record tree — e.g. an ingestion transform reading an extractor-produced value.Fix
All entry points now route the resolved value through a shared
jsonValueToStringhelper:String→ returned verbatim (unquoted).UUID/LocalDate/LocalTime— the non-JSON-native scalars a record extractor can materialize, whichobjectToStringwould wrap in quotes — render via their natural canonical / ISO-8601toString.Number,Boolean,Timestamp(portable epoch millis),byte[], andMap/Collection/ array containers — continues throughJsonUtils.objectToString.This keeps
jsonPathStringconsistent with the query-timejsonExtractScalaron every value the query path can produce; the only deliberate divergences are the three quote-stripping overrides for typesjsonExtractScalarnever encounters.Compatibility note
This changes the observable output of the released
jsonPathString*scalar functions forUUID/LocalDate/LocalTimeleaves (old: JSON-quoted / Jackson-default form; new: unquoted canonical form). The window is narrow — these types only occur on already-materialized record trees, not JSON-string input — but during a rolling upgrade a mixed-version cluster can briefly return both forms, and previously-persisted derived-column values will differ from newly-ingested ones. The new form is the intended correctness fix, so no version gate is added.