docs: compare Progenitor workflows

Author: lightsofapollo

.beads/issues.jsonl

@@ -53,6 +53,8 @@
 {"id":"openapi-generator-cv4","title":"Support exploded query parameters (GH #27)","description":"GH issue 27: query params with style=form, explode=true (OAS defaults) and object schemas currently map to Option\u003cimpl AsRef\u003cstr\u003e\u003e and are sent as one opaque string. Per OAS/RFC6570 form-explode, each object property must become its own query pair (?color=red). Fix: analyzer synthesizes/resolves a typed struct for object query params with form+explode semantics; client generator emits req.query(\u0026struct) so reqwest/serde_urlencoded serializes properties as individual pairs.","notes":"Implemented on branch issue-27-exploded-query-params, draft PR https://github.com/gpu-cli/openapi-to-rust/pull/28. Close when PR merges. Follow-ups: openapi-generator-anu (deepObject/explode=false/arrays), openapi-generator-0jz (server side).","status":"closed","priority":1,"issue_type":"feature","assignee":"James Lal","owner":"james@littlebearlabs.io","created_at":"2026-07-12T23:22:24Z","created_by":"James Lal","updated_at":"2026-07-13T04:21:19Z","started_at":"2026-07-12T23:22:49Z","closed_at":"2026-07-13T04:21:19Z","close_reason":"Released in v0.6.0 (PR #28)","dependency_count":0,"dependent_count":0,"comment_count":0}
 {"id":"openapi-generator-5q8","title":"DateStrategy::Time emits broken serde codec for time::Date/time::Time (GH #25)","description":"GitHub issue gpu-cli/openapi-to-rust#25: fields with format: date/time under DateStrategy::Time get #[serde(with = \"time::serde::iso8601\")], but that module only supports OffsetDateTime — generated code fails to compile. Fix: emit time::serde::format_description! helper codec modules (time_date_format / time_time_format) into generated code, and correct the time dep requirement features (serde alone doesn't even enable rfc3339).","notes":"Fixed in PR https://github.com/gpu-cli/openapi-to-rust/pull/26 (branch worktree-issue-25-time-date-serde). Close when PR merges.","status":"closed","priority":1,"issue_type":"bug","assignee":"James Lal","owner":"james@littlebearlabs.io","created_at":"2026-07-11T18:36:28Z","created_by":"James Lal","updated_at":"2026-07-11T18:57:02Z","started_at":"2026-07-11T18:36:45Z","closed_at":"2026-07-11T18:57:02Z","close_reason":"Fixed in PR #26, merged to main, released in v0.5.3","dependency_count":0,"dependent_count":0,"comment_count":0}
 {"id":"openapi-generator-dpd","title":"Hybrid string-or-object discriminated unions deserialize-fail","description":"When an anyOf/oneOf union contains a discriminator AND a non-object branch (e.g. string-enum like ToolChoiceOptions), the generator emits a tagged enum that cannot deserialize the string form. Real-world hit: OpenAI ToolChoiceParam returns 'auto' in Response.tool_choice, but generated type is #[serde(tag=\"type\")] enum with no untagged String variant. Need to fall back to #[serde(untagged)] when the union mixes string/scalar branches with tagged-object branches, OR add a String fallback variant before the tagged variants.","notes":"Live repro: `ToolChoiceParam` from openai.yaml line 52518. anyOf has 8 branches; the first (`ToolChoiceOptions`) is a string-enum (\"none\"|\"auto\"|\"required\"), the rest are objects with discriminator propertyName=type. Generator emits `#[serde(tag=\"type\")] enum ToolChoiceParam { ToolChoiceOptions(ToolChoiceOptions), ... }` which cannot deserialize the string \"auto\" because serde tries to read a \"type\" field from a JSON string. Fix: when an anyOf/oneOf branch is a non-object schema (string/number/etc), the generator must emit `#[serde(untagged)]` with the scalar branch first OR add a String variant before the tagged variants. Hit on real OpenAI Responses API `Response.tool_choice` field.","status":"closed","priority":1,"issue_type":"bug","assignee":"James Lal","owner":"james@littlebearlabs.io","created_at":"2026-05-10T23:47:35Z","created_by":"James Lal","updated_at":"2026-05-11T00:12:46Z","started_at":"2026-05-10T23:54:10Z","closed_at":"2026-05-11T00:12:46Z","close_reason":"Fixed in src/analysis.rs: (dpd) analyze_oneof_union now downgrades to untagged when any branch is non-object — verified live against OpenAI Response.tool_choice='auto' which now deserializes as ToolChoiceParam::ToolChoiceOptions(Auto). (bgo) merge_schema_into_properties now ORs in is_nullable_pattern() for allOf-merged props — verified live against OpenAI Response.incomplete_details which is now Option\u003cResponseIncompleteDetails\u003e and deserializes null cleanly. All 4 smoke tests (OpenAI+Anthropic, sync+stream) pass.","dependency_count":0,"dependent_count":0,"comment_count":0}
+{"id":"openapi-generator-he1","title":"Publish a source-linked Progenitor comparison","description":"Add a respectful, dated website and README comparison between openapi-to-rust and Progenitor. Frame the projects around their different target use cases, cite Progenitor's primary documentation, include benchmark results with methodology caveats, represent Progenitor strengths, and avoid treating undocumented features as unsupported.","acceptance_criteria":"Website comparison uses version-pinned, source-linked claims; README contains a concise non-adversarial summary; benchmark claims link to reproducible evidence; navigation exposes the comparison; website quality gates pass.","notes":"Added /compare hub and /compare/progenitor source-linked decision page, benchmark table with limitations, README summary, navigation/footer/docs/LLM links, and corrected stale openapi-to-rust version/corpus count on the existing comparison. Astro build, html-validate, and SEO audit pass. In-app browser was unavailable for screenshot QA.","status":"in_progress","priority":2,"issue_type":"task","assignee":"James Lal","owner":"james@littlebearlabs.io","created_at":"2026-07-29T01:50:41Z","created_by":"James Lal","updated_at":"2026-07-29T01:54:36Z","started_at":"2026-07-29T01:50:49Z","dependency_count":0,"dependent_count":0,"comment_count":0}
+{"id":"openapi-generator-dwn","title":"Benchmark openapi-to-rust against Progenitor build workflows","description":"Create a reproducible, version-pinned benchmark comparing ahead-of-time openapi-to-rust generation with Progenitor's procedural-macro workflow on the same OpenAPI fixture. Measure clean Cargo check, no-op check, Rust source rebuild, and spec-change regeneration; document methodology and avoid unsupported marketing claims.","acceptance_criteria":"Benchmark records hardware/toolchain, pinned versions, fixture, commands, repeated samples, and reports median results with methodological caveats.","status":"in_progress","priority":2,"issue_type":"task","assignee":"James Lal","owner":"james@littlebearlabs.io","created_at":"2026-07-29T01:34:49Z","created_by":"James Lal","updated_at":"2026-07-29T01:34:54Z","started_at":"2026-07-29T01:34:54Z","dependency_count":0,"dependent_count":0,"comment_count":0}
 {"id":"openapi-generator-36a","title":"Run playground WASM lock bump only for release tags","description":"The playground WASM workflow currently runs on matching pushes to main and commits website/playground-wasm.lock. Restrict automatic runs to v* release tag pushes, while ensuring the generated lock bump is committed back to main rather than attempting to update the tag ref.","acceptance_criteria":"Pushes to main do not trigger the workflow; v* tag pushes publish the WASM asset; the lock bump is committed to main; manual dispatch remains safe.","status":"closed","priority":2,"issue_type":"bug","assignee":"James Lal","owner":"james@littlebearlabs.io","created_at":"2026-07-28T21:28:57Z","created_by":"James Lal","updated_at":"2026-07-28T21:30:21Z","started_at":"2026-07-28T21:29:00Z","closed_at":"2026-07-28T21:30:21Z","close_reason":"Restricted automatic playground WASM publishing to v* tag pushes and made tag-triggered lock commits target main; actionlint and diff checks pass.","dependency_count":0,"dependent_count":0,"comment_count":0}
 {"id":"openapi-generator-zeg","title":"Add Storyden OpenAPI document to the real-world corpus","description":"Vendor Southclaws/storyden api/openapi.yaml into specs, register its canonical download source, preserve its MPL-2.0 per-file licensing notice, update documented corpus counts, and run Storyden generation/compile coverage after the three discovered fixes.","acceptance_criteria":"specs/storyden.yaml is reproducibly sourced and carries clear MPL-2.0 attribution; corpus documentation counts are updated; parse-only and compile generation for Storyden pass with the fixed generator.","notes":"Approach: copy the canonical main-branch spec to specs/storyden.yaml, add an MPL-2.0/source notice at the file boundary, register the raw GitHub URL in specs/download.sh, update README/CONTRIBUTING counts from 55/54 to 56/55, then run scripts/spec-compile.sh storyden and the Storyden all-operation Axum compile smoke. Files: specs/storyden.yaml, specs/download.sh, README.md, CONTRIBUTING.md. License: upstream repository LICENSE is MPL-2.0; keep the vendored file under MPL-2.0 with explicit source/license notice.","status":"closed","priority":2,"issue_type":"task","assignee":"James Lal","owner":"james@littlebearlabs.io","created_at":"2026-07-28T21:26:54Z","created_by":"James Lal","updated_at":"2026-07-28T22:26:29Z","started_at":"2026-07-28T21:27:01Z","closed_at":"2026-07-28T22:26:29Z","close_reason":"Implemented and verified against the vendored Storyden spec: generated clients and Axum servers compile and round-trip; response/request media handling is representation-safe and all buffered client responses are bounded.","dependency_count":0,"dependent_count":0,"comment_count":0}
 {"id":"openapi-generator-2yk","title":"Exercise Storyden OpenAPI document through generators","description":"Fetch Southclaws/storyden api/openapi.yaml, run validation and representative generation workflows against this checkout, investigate reproducible failures or malformed output, and record concrete generator bugs.","acceptance_criteria":"The specification is exercised through relevant generator paths; failures are reduced to actionable evidence; confirmed generator bugs are tracked or fixed and verified.","notes":"Fetched Southclaws/storyden main/api/openapi.yaml (13,105 lines, 457,079 bytes). Direct client generation parsed 615 schemas and 237 operations, emitted types/client/mod/dependency manifest, and passed cargo check. Semantic inspection found scalar-alias array query fallback and discarded non-JSON success bodies. Per-operation Axum sweep found 13 front-end failures: 5 scalar-alias query arrays and 8 non-JSON response media operations. After excluding those, combined server output exposed 8 response enum/schema naming collisions (E0659/E0072). After also excluding collision operations, remaining 216 operations generated together with validation enabled and passed cargo check. Follow-up bugs: openapi-generator-6z1, openapi-generator-9en, openapi-generator-8c6.","status":"closed","priority":2,"issue_type":"task","assignee":"James Lal","owner":"james@littlebearlabs.io","created_at":"2026-07-28T21:05:25Z","created_by":"James Lal","updated_at":"2026-07-28T21:12:16Z","started_at":"2026-07-28T21:05:30Z","closed_at":"2026-07-28T21:12:16Z","close_reason":"Investigation complete; three concrete generator bug classes reproduced, reduced, tracked, and remaining Storyden server surface compile-verified.","dependency_count":0,"dependent_count":0,"comment_count":0}

README.md

@@ -55,6 +55,26 @@ Release history and breaking changes live in the [changelog](CHANGELOG.md).
 - **Snapshot testing** — `insta` snapshots for generated output.
 - **Optional `specta::Type` derives** for cross-language type sharing.
 
+## Compared with Progenitor
+
+[Progenitor](https://github.com/oxidecomputer/progenitor) is an established,
+opinionated OpenAPI 3.0.x client generator developed by Oxide, with particular
+alignment to Dropshot-generated APIs. It supports procedural-macro,
+build-script, and static-crate workflows, plus pagination streams, generated
+CLI support, and typed mocking helpers.
+
+openapi-to-rust grew around consuming large third-party specifications. Its
+focus includes OpenAPI 3.1 schema forms, ahead-of-time generated source,
+selected Reqwest client operations, SSE, and Axum server scaffolding with shared
+models. In one reproducible 120-operation benchmark, its normal ahead-of-time
+workflow had 26.5% lower median clean `cargo check` wall time; no-op builds were
+effectively identical. This is a version- and machine-specific observation, not
+a universal performance guarantee.
+
+See the [dated, source-linked Progenitor comparison](https://openapi-to-rust.dev/compare/progenitor)
+and the [benchmark methodology](benchmarks/progenitor-builds/README.md) for the
+relevant tradeoffs and limitations.
+
 ## Install
 
 Rust users with Rust 1.88 or newer can install the CLI from crates.io:

benchmarks/progenitor-builds/README.md

@@ -0,0 +1,51 @@
+# Progenitor build-workflow benchmark
+
+This benchmark compares the normal ahead-of-time `openapi-to-rust` workflow
+with Progenitor's default-enabled `generate_api!` procedural-macro workflow.
+It is a workflow benchmark, not a claim that the two tools generate equivalent
+clients.
+
+The harness creates the same synthetic OpenAPI 3.0.3 document for both tools,
+pins Progenitor to 0.14.0 and the direct consuming dependencies used by the
+checked-in result, downloads dependencies before timing, and measures:
+
+- a clean `cargo check` with fresh Cargo build and target directories;
+- an unchanged, no-op `cargo check`;
+- `cargo check` after touching ordinary Rust source; and
+- the complete spec-change loop after changing a schema description: regenerate
+  then check for `openapi-to-rust`, and check (including macro expansion) for
+  Progenitor.
+
+## Run
+
+```bash
+cargo build --release --bin openapi-to-rust
+node benchmarks/progenitor-builds/run.mjs \
+  --samples 5 \
+  --operations 120 \
+  --output /tmp/progenitor-build-results.json
+```
+
+The first run needs network access to resolve and fetch the two dependency
+graphs. Timed Cargo commands run offline. The harness uses Cargo's
+`build.build-dir` setting, so reproduce the checked-in result with Cargo 1.97
+or newer.
+
+## Interpretation limits
+
+- Clean-build measurements include each generated client's current dependency
+  graph (`reqwest` 0.12 for `openapi-to-rust`, 0.13 for Progenitor). They do not
+  isolate procedural-macro overhead.
+- Filesystem and Cargo registry caches are warm, but every clean sample uses
+  fresh compiled-artifact directories.
+- The fixture is synthetic and intentionally limited to features both tools
+  accept. It measures scaling with operations and models, not compatibility
+  with difficult real-world schemas.
+- No-op builds should be effectively equal because Cargo does no generation
+  work when the consuming crate is fresh.
+- Results should always be published with tool versions, hardware, raw samples,
+  and a verification date. Do not generalize one machine's result into a
+  universal build-time guarantee.
+
+The dated result used for the initial comparison is in
+[`results/2026-07-28-m4-max.json`](results/2026-07-28-m4-max.json).

benchmarks/progenitor-builds/results/2026-07-28-m4-max.json

@@ -0,0 +1,58 @@
+{
+  "schemaVersion": 1,
+  "recordedAt": "2026-07-29T01:45:00Z",
+  "fixture": {
+    "kind": "synthetic-openapi-3.0.3",
+    "operations": 120,
+    "schemas": 120,
+    "bytes": 151254
+  },
+  "versions": {
+    "openapiToRust": "openapi-to-rust 0.11.0",
+    "progenitor": "0.14.0",
+    "rustc": "rustc 1.97.0 (2d8144b78 2026-07-07)",
+    "cargo": "cargo 1.97.0 (c980f4866 2026-06-30)",
+    "aotDirectDependencies": "reqwest 0.12.28, reqwest-middleware 0.4.2, serde 1.0.229, serde_json 1.0.151, thiserror 1.0.69",
+    "macroDirectDependencies": "progenitor 0.14.0, reqwest 0.13.4, serde 1.0.229"
+  },
+  "machine": {
+    "platform": "darwin",
+    "release": "25.3.0",
+    "arch": "arm64",
+    "cpu": "Apple M4 Max",
+    "logicalCpus": 16,
+    "memoryBytes": 137438953472
+  },
+  "methodology": {
+    "cleanBuild": "Fresh Cargo target and build directories; dependencies already downloaded; five alternating-order samples.",
+    "noOp": "Unchanged cargo check using a warmed target; six samples.",
+    "rustTouch": "Touch src/lib.rs, then cargo check; six alternating-order samples.",
+    "specChangeAot": "Change a schema description, run the release generator, then cargo check; six alternating-order samples.",
+    "specChangeMacro": "Change the same schema description, then cargo check including macro expansion; six alternating-order samples."
+  },
+  "samplesSeconds": {
+    "clean:aot": [5.84, 5.88, 8.17, 5.91, 6.12],
+    "clean:macro": [8.04, 7.85, 11.08, 8.56, 8.03],
+    "noop:aot": [0.08, 0.08, 0.08, 0.08, 0.08, 0.08],
+    "noop:macro": [0.08, 0.08, 0.08, 0.08, 0.08, 0.08],
+    "rust-touch:aot": [0.45, 0.44, 0.45, 0.44, 0.44, 0.45],
+    "rust-touch:macro": [0.59, 0.58, 0.57, 0.59, 0.58, 0.58],
+    "spec-change:aot": [1.26, 0.54, 0.53, 0.54, 0.54, 0.54],
+    "spec-change:macro": [0.66, 0.64, 0.64, 0.63, 0.64, 0.64]
+  },
+  "medianSeconds": {
+    "clean:aot": 5.91,
+    "clean:macro": 8.04,
+    "noop:aot": 0.08,
+    "noop:macro": 0.08,
+    "rust-touch:aot": 0.445,
+    "rust-touch:macro": 0.58,
+    "spec-change:aot": 0.54,
+    "spec-change:macro": 0.64
+  },
+  "aotMedianReductionPercent": {
+    "clean": 26.5,
+    "rustTouch": 23.3,
+    "specChange": 15.6
+  }
+}