command reference generator from package source — no editor, no dump - #3
command reference generator from package source — no editor, no dump#3ArkTarusov wants to merge 1 commit into
Conversation
The reference was produced in two manual steps: a `unity --json command` dump from a live editor plus a hand-extracted RuntimeOnly section, so a routine update needed a running Editor with the package installed, and the split left a stale preamble to re-splice by hand every time. tools/CommandRefGen reads the com.unity.pipeline package straight off the public UPM registry instead. It resolves a version (or `latest` by semver precedence), unpacks the tarball, and parses `[CliCommand]`/`[CliArg]` through the Roslyn syntax tree — attributes span lines, descriptions are concatenated or verbatim, and a `[CliArg]` default falls back to the C# parameter default, none of which survives a regex. The package source is a strict superset of the live listing: it carries the RuntimeOnly commands the listing filters out and the version-gated ones that only exist on newer Unity. The generator owns the whole output file — banner, preamble, Contents, every category section, RuntimeOnly section. Field notes that are true but absent from the source live in annotations.json and are merged into the matching entry, so hand-maintained prose sits in a file the generator reads rather than the file it writes. Every run prints a diff summary of added, removed and changed commands and arguments. Details worth naming: - Descriptions are never shortened. Newlines and whitespace runs collapse to single spaces so a multi-line description cannot break the list layout; that is the only edit. --max-description is an emergency ceiling that warns by command name when hit, never a silent cut. - A command under `#if UNITY_6000_7_OR_NEWER` is marked with its Unity floor rather than presented as universally available. Each file is parsed twice, once with every `#if` symbol defined and once with none, so `#else` and `#if !SYMBOL` branches are reached too; a command declared in both branches is reported ungated, because it exists either way. - The package's Tests/ assembly is excluded — it registers throwaway commands (log_editor, test_types, test_structured), and a warning fires if one ever reaches the document. A `Commands/Tests/` category directory is kept: that is where run_tests and friends live. - float.MinValue, the package's "leave Unity's own value alone" sentinel, renders as -3.40282347e+38, matching the live listing exactly. - Categories come from the directory under `Commands/`; an unmapped directory still gets a section, humanized and named in a warning. tests/run.sh checks the generator against a checked-in fixture package that exercises each rule, plus the warnings it should raise and both --check outcomes. No registry access and no Unity needed. CI runs it before the generator touches the real file. The checked-in reference still holds the old pipeline's content; its banner now points at the generator and says so, and the first run replaces the file wholesale. Replaces tools/gen-commands-md.js.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c74309e2a9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // A release outranks any prerelease of the same core version. | ||
| if (_pre.Length == 0 || other._pre.Length == 0) | ||
| return (other._pre.Length == 0 ? 1 : 0) - (_pre.Length == 0 ? 1 : 0); |
There was a problem hiding this comment.
When latest compares a stable version such as 0.4.0 with 0.4.0-exp.2, this subtraction returns -1 for the stable version, so OrderBy(...).Last() selects the prerelease. This reverses SemVer precedence and can make the default workflow document an experimental package even when the corresponding stable release exists.
Useful? React with 👍 / 👎.
| // Pass A defines every symbol seen in an #if, so version-gated commands | ||
| // (`#if UNITY_6000_7_OR_NEWER`) land in the tree instead of in disabled text. | ||
| foreach (var c in ReadCommands(ParseTree(text, symbols), rel, regions)) |
There was a problem hiding this comment.
Parsing only with every symbol defined and then with none does not cover all conditional branches. For example, in #if A ... #elif B ... #else, the all-defined pass selects A and the none-defined pass selects else, so a command declared only under #elif B is silently omitted; mixed conditions such as A && !B are likewise false in both passes. This can make the generated reference incomplete for valid package sources.
Useful? React with 👍 / 👎.
| sb.Append("The `[CliCommand]` attribute of every command below sets `RuntimeOnly = true`, so ") | ||
| .Append("`unity command` neither lists them nor shows their schemas — these entries, read out of the ") | ||
| .Append("`com.unity.pipeline` `").Append(version).Append("` package source, are the only schema source ") | ||
| .Append("for them. They execute against the editor server despite the flag, and against Player ") | ||
| .Append("connections (`--runtime <process>` / `--runtime-path <port file>`)."); |
There was a problem hiding this comment.
When capture_runtime_element is present, the generated section says all RuntimeOnly commands execute against the editor server, while annotations.json appends that this particular command does not. The resulting reference contradicts itself and may lead users to invoke this player-only command against an editor; retain the existing exception or avoid making the universal claim.
Useful? React with 👍 / 👎.
2 participants
Closes #1.
tools/CommandRefGen(net8, Roslyn as its only external dependency) generatesskills/unity-pipeline/references/editor-commands.mdfrom thecom.unity.pipelinepackage sources on the public UPM registry.tools/gen-commands-md.jsis gone.A version bump is a rerun with a different argument. No code edits, no Unity, no license, no
unity --json commanddump.How it works
--versionagainsthttps://packages.unity.com/com.unity.pipeline;latestpicks the highest entry inversionsby full semver precedence (a release outranks its prereleases,exp.2outranksexp.1).dist.tarballand unpacks it (HttpClient,GZipStream+System.Formats.Tar).[CliCommand]/[CliArg]through the Roslyn syntax tree.Roslyn rather than regexes because attributes span lines, descriptions are built by concatenation or written as verbatim strings, and a
[CliArg]default falls back to the C# parameter default when the attribute sets noDefaultValue.Single ownership of the output file
The generator owns the whole file: banner, preamble, Contents list, every category section, RuntimeOnly section. Nothing in it is hand-maintained, so there is no stale preamble to re-splice.
The field notes that are true but not in the source —
simulate_pointerfeeding a virtual device,capture_runtime_elementnot executing against the editor,quitversuseval EditorApplication.Exit(0)— moved totools/CommandRefGen/annotations.json, which the generator merges into the matching entry. Hand-maintained prose now lives in a file the generator reads, never in the file it writes. A note whose command disappeared is reported as unused.Requirements from the review comment
--max-description(1000) is an emergency ceiling that warns by command name when hit — never a silent cut.#if UNITY_6000_7_OR_NEWERis marked with its Unity floor instead of being presented as universally available. Non-version gates (#if ENABLE_INPUT_SYSTEM) are reported as conditional. Each file is parsed twice — once with every#ifsymbol defined, once with none — so#elseand#if !SYMBOLbranches are reached too; a command declared in both branches is reported ungated, because it exists either way.Other details
Tests/exclusion. The package's test assembly is skipped, and a warning fires iflog_editor/test_types/test_structuredever reaches the document anyway. ACommands/Tests/category directory is kept — that is whererun_testsand friends live, and a blanket "any segment named Tests" rule would have dropped them.Commands/. Display names and section order live inCategories.cs; an unmapped directory still gets a section, humanized, with a warning naming it.float.MinValue, the package's "leave Unity's own value alone" sentinel, renders as-3.40282347e+38— byte-identical to the current file.Required/IsRequiredon[CliArg]decides it; otherwise an argument with no default of any kind is required. The current file has arguments that are both required and defaulted, so the explicit property is load-bearing.Verification
tools/CommandRefGen/tests/run.shruns the generator against a checked-in fixture package and diffs against a golden file. No registry access, no Unity. The fixture exercises multi-line attributes, both default sources, an explicitRequiredon a defaulted argument, thefloat.MinValuesentinel, a verbatim multi-line description, a concatenated one, a#ifversion gate, a compound gate, a command declared in both#ifbranches,RuntimeOnly,MainThreadRequired = false, a parameter with no[CliArg], an unmapped category directory, and aTests/assembly whose commands must not appear. The script also asserts the warnings the fixture should provoke and both--checkoutcomes. It passes.The diff reporter was additionally pointed at the real 150-command reference file, which it parses correctly.
.github/workflows/command-ref.ymlruns the fixture test, then regenerates, onworkflow_dispatchorworkflow_call— the latter is the hook for the release-watch workflow from #2. It puts the diff summary and warnings in the step summary and uploads the file plus the log.What is not done here
The reference file itself is not regenerated in this PR.
packages.unity.comis blocked by this environment's egress policy (403 on CONNECT), so the registry path could not run here — the code path is exercised, and fails with a clear message and a pointer to the offline flags, but no real package was downloaded:So the checked-in
editor-commands.mdstill holds the old pipeline's content. Its banner now points at the generator and states that plainly; the first real run replaces the file wholesale, and its diff summary is the review artifact for that change.--tarballand--source-direxist for exactly this situation and produce output identical to the registry path — I verified--tarballagainst a packed fixture.Two consequences worth expecting on that first run, both reported in the diff summary rather than silently applied:
Generated by Claude Code