fix expanded-corpus generation: media types, validation patterns, paths, allOf aliases - #47
Merged
Merged
Conversation
…hs, allOf aliases
Probe-driven fixes from running 241 previously-failing APIs.guru specs
(AWS, Adyen, apisetu.gov.in, api.video, ...) through client and server
generation. 210/240 now generate cleanly (was ~2%):
- allOf [$ref, {description}] self-references are type aliases again;
recursive expansion overflowed the stack on AWS-style specs.
- application/pdf responses classify as Binary; application/xml and +xml
suffixed types classify as Text on both response and request sides.
- best_content() falls back to character-data media (text/xml etc.)
instead of leaving XML-only request bodies unsupported.
- Validation pattern normalization: Java POSIX classes (\p{Print},
\p{Alpha}, ...) translate to ASCII ranges, ECMA \uXXXX escapes to
Rust \u{XXXX}, legacy octal escapes to \xNN. Patterns that still
cannot compile offline (look-around, backreferences) degrade to
"no pattern check" instead of failing generation.
- AWS x-pattern extension promotes to the standard pattern keyword.
- Validation bundle component keys are prefixed so a schema named like
a JSON Schema keyword (`id`) survives meta-schema validation.
- AWS query markers (/tags/{arn}#tagKeys) strip the #fragment before
route generation; webhook synthetic paths gain their leading `/`.
- Server code qualifies non-canonical schema names (not-found) with
their canonical Rust identifiers via rust_type_name().
Remaining known gaps (fail loudly, not silently wrong): form-style
object-array query items (AWS query protocol wire shape is undefined
by OpenAPI), multipart server extraction, YAML tab-indentation specs,
and one PathItem-as-string spec.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
AWS query-protocol specs (sqs, docdb, ec2, elasticache, neptune, rds,
redshift) declare form-style array parameters whose items reference a
flat structure — every property scalar — such as Tag { Key, Value }.
These now project to `Vec<T>` and serialize symmetrically on the wire
as `param.N.Prop=value` (e.g. `tags.1.Key=env&tags.1.Value=prod`):
- New ArrayItemType::FlatStructRef carries the resolved property names
so client and server emit identical keys without re-resolving.
- Item refs wrapped in annotation-only allOf (AWS's
`items: {allOf: [$ref, {xml}]}` idiom) are seen through, mirroring
the type-alias rule.
- The generated server groups `param.N.Prop` pairs by N and decodes
each group as one JSON object so serde fills the struct.
- Nested structures (arrays/objects below one level) stay rejected:
their wire shape below depth one is service-specific, and the
generator fails loudly rather than guessing.
Also fixes a client parse failure on EC2: wire property names like
`Type` were emitted as bare field idents (`item.type`); they now route
through the model generator's keyword escaping (`item.r#type`).
Regression coverage: server_query_roundtrip_test gains a Tag-array
parameter exercised end-to-end — the generated client emits
`tags.1.Key=env&tags.1.Value=prod` and the generated Axum server
observes typed `Vec<Tag>` values.
Real-world specs (Adyen CheckoutService/PaymentService, Amadeus trip-parser) carry literal tab characters inside block-scalar prose, which serde_yaml rejects outright per YAML 1.2's tab-indentation rule. parse_spec now retries once with tabs sanitized line-wise: tabs on whitespace-only lines drop away (blank lines carry no indent semantics), content tabs become a single space, keeping block-scalar indent auto-detection consistent with sibling prose lines. Adds a debug escape hatch used while hunting a client parse failure: OATR_DUMP_TOKENS_ON_PARSE_ERROR=<path> writes the generated token stream when syn rejects it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Probe-driven bug fixes from running 241 previously-failing APIs.guru specs (AWS, Adyen, apisetu.gov.in, api.video, …) through client and server generation. 210/240 now generate cleanly (was ~2% at the start of the probe).
Fixes
allOf: [$ref, {description}]self-references are type aliases again — recursive expansion overflowed the stack on AWS-style specs (3+ specs crashed)application/pdf→ Binary;application/xml++xmlsuffixes → Text (responses and requests);best_content()falls back to character-data media so XML-only request bodies are no longer unsupported\p{Print},\p{Alpha}, …) → ASCII ranges; ECMA\uXXXX→ Rust\u{XXXX}; legacy octal\000→\x00. Patterns that still can't compile offline (look-around, backreferences) degrade to "no pattern check" instead of failing the whole specx-patternpromotes to the standardpatternkeywordid(real AWS DataPipeline spec) survives draft-4 meta-schema validation/tags/{arn}#tagKeys) strip the#fragment; webhook synthetic paths gain their leading/not-found) are qualified with their canonical Rust identifiers viarust_type_name()Each fix ships with regression tests; full suite is green (75 test targets).
Remaining known gaps (fail loudly, not silently wrong)
param.N.Prop) is undefined by OpenAPI; generator rejects explicitly rather than guessingnot implemented)Test plan
cargo test— all green