gdscript highlighting and packaging

[DavidCooperWCU]

A new prism format was added for gdscript 2.0
A new component was added to generate gdscript packs as zip files The documentation was updated to describe the new gdscript highlighting and active code capabilities 1 program example and 3 activecode examples were added to the Runestone Testing section of the sample book.

[DavidCooperWCU]

To test this:

1. checkout the pr with gh pr checkout 3024
2. cd examples/sample-book
3. mkdir out-test
4. python ../../pretext/pretext -vvv -d out-test -f html -c all -p publication-structural.xml sample-book.xml
5. mkdir out-test/generated/gdscript
6. python ../../pretext/pretext -vvv -d out-test/generated/gdscript -f html -c gdscript -p publication-structural.xml sample-book.xml

Then serve the book and look at Listing 5.1.3 to see the godot highlighting.

Activities 5.2.1, 5.2.2, and 5.2.3 are examples how the ActiveCode will look, but it won't work fully until rs - PR 1233 has been accepted.

This is a circular dependency, but this PR is the first step.

[DavidCooperWCU]

@rbeezer I worked with @ascholerChemeketa to make sure that things were working properly without the pretext cli, and I made some additional changes based on reviews by @oscarlevin

I think it's ready for you to review.

[rbeezer]

This is the resubmission of #2892 (closed in favor of this PR, and described here as cleaned up). Before a detailed pass, the PR needs to meet conventions that were already stated explicitly in the #2892 review. A number of them recur unchanged, and several new ones appear. The consistent issue is code that does not follow PreTeXt's established conventions.

Raised on #2892, still present here

• f-strings. GDScript code highlighting #2892: "We like to use .format(), not 'f-strings'." godot_helper.py still contains log.info(f"Export templates for {GODOT_VERSION} already installed at {template_dir}.") — the only f-string in a file that otherwise uses .format().
• log.error for normal output. GDScript code highlighting #2892: "log.debug/log.info are used for progress, not log.error." gd_pack calls log.error(result.stderr) on the success branches of both the trial and export runs (returncode == 0). Godot writes to stderr routinely, so successful builds will report errors.
• Dead commented code and trailing whitespace. GDScript code highlighting #2892: "Remove the dead commented line … (and its trailing whitespace)." This PR still ships commented-out code — #_, external_dir = common.get_managed_directories(...) in gdscript_pck, and a commented-out template match in extract-gdscript.xsl — and adds trailing whitespace on many lines.
• .gitignore to suppress stray files. GDScript code highlighting #2892: ".DSStore … should not be precluded by .gitignore, it is up to an author … to clean up that sort of stuff before it becomes part of source." This PR adds examples/sample-book/gdpractice/.gitignore containing *.uid, *.import, .godot — the same pattern that was asked against.
• XSL formatting in pretext-runestone.xsl. GDScript code highlighting #2892 itemized "stray blank lines and inconsistent indentation inside <xsl:attribute>" and "leftover commented-out lines." The data-wasm/data-pck/data-scene construction here still has inconsistent indentation and stray blank lines.

New convention problems in this submission

• Tabs. godot_helper.py is indented with tabs throughout (111 lines); the codebase uses four spaces. gd_pack additionally mixes in three-space indentation.
• A committed empty file: …/L2.P9.visual_2d/practices/L2.P9.visual_2d_input_motion/practice.gd is 0 bytes.
• A committed binary blob: …/first_script/project.binary (binary "blobs make my ears prick up" was already noted on GDScript code highlighting #2892).
• Stale copied comments. extract-gdscript.xsl's header says it "prepares a Python dictionary" and refers to a "YouTube ID … comma-separated pair," but it emits a comma-separated triple; pretext.py carries "# no format … (*.jpg)". Typos: "providews", "activcode".
• Grammar in the Guide: "GDScript has it's own way" → "its".

Correctness and substance

• Cleanup bug in gd_pack. In the finally block, the project-file removal is guarded by config_already_existed where it must be project_already_existed. As written, a pre-existing project.godot may not be restored, or the template copy may be left behind in the author's source.
• New attributes are not in the schema. @pck and @scene on <program> are used by the XSL and Python, but <program> has a closed attribute list, so documents using them do not validate. The schema source needs these attributes added. (Deriving the schema files is a maintainer step — no need to touch the generated files.)
• The Godot version is pinned in two places — $godot.version in pretext-runestone.xsl and GODOT_VERSION in godot_helper.py — which can drift.
• gd_pack writes into the author's source directory (project.godot, export_presets.cfg) and relies on best-effort restoration in finally; an interrupted run leaves the source tree modified. Work from a temporary copy instead.
• New third-party dependency. godot_helper.py imports certifi; the toolchain is standard-library only. And the download-and-run of the Godot engine/templates carries no integrity check — both are design questions to settle (the pretext-dev thread is the place for that) before code review continues.

For the next round: bring the Python and XSL into line with the conventions already given on #2892 (four-space indentation, .format(), log.info/log.debug for progress, no commented-out code, no trailing whitespace, no .gitignore/binary/empty files in the examples), fix the cleanup bug, and add the schema entries. The build behavior (source-tree mutation, engine download) should be resolved on the dev list.

Claude Opus 4.8, acting as a review assistant for Rob Beezer

[DavidCooperWCU]

@rbeezer I did my best to address all of the notes. A discussion on pretext-dev took place. It looks like certifi is already a dependency of another library, and I added a checksum integrity check for the download of godot and the export templates.

[rbeezer]

The "programming languages" lookup table has been edited. It may just be a cosmetic edit that affects this @ptx renamed to @name for consistency, that will require an edit or will cause a merge conflict. The infrastructure changes should not be an issue, I expect.

[DavidCooperWCU]

I thought the only difference in the conflicting section was the gdsript programming language was added. So, I merged it. Then I saw what you meant about the @ptx -> @name, and I fixed that.

[DavidCooperWCU]

I corrected the merge I did 2 days ago, and then tested it.

[rbeezer]

Thanks for the continued work — a solid round. Confirming what's now resolved:

gd_pack builds in a disposable copy, which fixed both the source-tree mutation and the config_already_existed/project_already_existed mix-up at once. The engine and export templates are SHA512-verified. The f-string is gone. The 111 tab-indented lines in godot_helper.py are now four-space, and the three-space block with them. Stdlib modules import normally instead of through try/except ImportError. Both dead commented-out blocks are gone. The copyright header is correct — Beezer, and no remaining "PreTexT". The binary blob, the 0-byte practice.gd and the .gitignore are all out of the examples. The prism grammar lives in its own file loaded by a script tag, and the five stray blank lines around it in pretext-html.xsl are gone. @pck and @scene are in the schema source, which is the right half of that job — the derived files are mine to regenerate. The "Python dictionary", "pair" and "providews" comment errors are corrected, as is "it's" → "its" in the Guide. Python no longer carries its own copy of the Godot version.

Blockers

1. <program> inside <p> — examples/sample-book/rune.xml:211 and :222. Both non-interactive GDScript programs sit inside the <statement><p> opened at line 209, which doesn't close until line 235. program is block-level and the schema forbids it there, so the sample book no longer validates:

error: element "program" not allowed here; expected the element end-tag, text or
element "abbr", "acro", ... "xref"

2. Attributes that do nothing, in the sample book — rune.xml:78-80. @pck and @scene are on a non-interactive <program>. Both extract-gdscript.xsl and the detection XPath in pretext.py require @pck and @interactive='activecode', so no pack is built and no data-pck is emitted. Its @scene also points at res://practices/L1.P1.example/..., and L1.P1.example exists nowhere in the committed scaffold; and it uses pck="ch01_p1.pck", a filename, where every working example uses a directory. The sample book is what authors copy from.

3. js/dist/prism/gdscript-prism.js should come out of the diff. Dist files are regenerated by the maintainer. The jsbuilder.mjs target addition is correct and should stay.

Repeated or ignored, most significant first

1. $b-has-gdscript. 2026-06-05 on #2892: "We could have a boolean variable like $b-has-gdscript that is a stricter test than $b-has-program and only load the JS as needed." The guard in pretext-html.xsl is still $b-has-program. Building the sample article, which contains no GDScript, puts gdscript-prism.js on all 62 of its pages — so every PreTeXt book with a single code listing now fetches a grammar it can't use. The prism autoloader two lines above is the mechanism for exactly this.

2. certifi. 2026-07-04 on #2892 asked for standard-library only; 2026-07-16 called the dependency and the download behavior "design questions to settle (the pretext-dev thread is the place for that) before code review continues." The import is still there, and it is worse than it looks: pretext.py has a bare from . import godot_helper, godot_helper.py has a bare import certifi, and certifi.where() runs at import time to build _SSL_CONTEXT. With certifi unavailable the module fails to import at all —

ImportError: No module named 'certifi'

— so this is every command and every output format for every user, not just GDScript builds. requests does pull certifi in, but transitively and undeclared. Importing godot_helper lazily inside gd_pack, and moving import certifi and the context construction into the download helper, confines the failure to GDScript work.

3. The Godot version is still pinned in two places. 2026-07-16: "The Godot version is pinned in two places — $godot.version in pretext-runestone.xsl and GODOT_VERSION in godot_helper.py — which can drift." Python is fixed. The literal '4.6.3' is now an xsl:param default in both pretext-runestone.xsl and extract-gdscript.xsl, each under a verbatim copy of the same three-line comment. Override the string parameter on one conversion and not the other and the WASM shell and the .pck are built for different versions, silently.

4. Merge commits. 2026-06-05: "we would prefer you rebase your contribution on master privately before making a PR... usually we do not want them in the way." The branch now carries a merge of master into it plus a further commit fixing that merge. Commit subjects also want the Topic: short description form, and the multi-line bodies can go.

5. Building the attribute values. 2026-07-04: "Build each value cleanly (a variable or concat()) and drop the blank lines." The whitespace-only line inside data-wasm and the inconsistent indentation are fixed — but the values are still assembled across separate xsl:value-of/xsl:text lines, and there are now two consecutive blank lines after the new godot.version/godot.shell params. The data-pck branch is also an xsl:choose holding a single xsl:when with no xsl:otherwise, which is an xsl:if.

6. The listings="Python" comment. 2026-06-05: "Let's add a code comment before we use Python for LaTeX listings - just so there is no confusion in the future." The row in pretext-common.xsl reads listings="Python" with no comment near it.

7. Trailing whitespace — third time. Asked on 2026-07-04, repeated 2026-07-16. Thirteen lines, mostly rune.xml, plus a new blank line at end of file in js/prism/gdscript-prism.js. git diff --check lists them all.

8. Tabs, fixed in Python and reintroduced in the sample book. rune.xml:215 and :226 begin with a literal tab followed by spaces, while their neighbours are space-indented, and the second block then indents deeper lines with spaces alone. GDScript is indentation-sensitive, so beyond the convention this code won't run as written.

9. log.error for what isn't an error. 2026-07-04: "log.debug/log.info are used for progress, not log.error." The success branches are properly fixed. gd_pack still catches a missing godot executable, logs it at ERROR, then falls back to "godot" and continues — an ERROR line for a condition it recovers from.

10. Generated files in the examples. 2026-07-04: "Any filename with a generated hash looks suspect here... it is up to an author... to clean up that sort of stuff before it becomes part of source." .gitignore is gone, but .file_checksums.json is still committed, and only under L1.P1.double_value — the other two practices have none, while two of the three carry an icon.svg the third lacks. The scaffolds look hand-assembled rather than exported.

11. activcode typo in the extract-gdscript.xsl header, flagged 2026-07-16. The header also still describes the match as gdscript/@pck where it is program[@pck].

Documentation

@pck and @scene are new author-facing attributes and are documented nowhere, and the same is true of the godot.version and godot.shell publisher parameters. New options need a Guide reference entry and a conversational mention. The one new sentence on GDScript testing doesn't say what the GDPractice addon is or where it comes from, hardcodes a count of examples that will rot, doesn't link the sample book the way the paragraph above it does, and writes the language as "gdscript" where the same sentence uses "GDScript".

On the language table

Following up my 2026-07-24 note — the @ptx → @name resolution came through correctly. One thing to undo: the "Procedural" block was re-aligned by a column to fit gdscript, which didn't need it, since every entry already sat at the same column and name="gdscript" is shorter than the name="javascript" that fit there. That turns a one-row addition into 19 lines of whitespace diff and leaves the table aligned in two halves.

Claude Opus 5, acting as a review assistant for Rob Beezer

[rbeezer]

A review from Claude Pro is not without substantial effort, and we are not even getting past that stage. When a review asks for changes, please make them. Not everything here may be 100% accurate, but there is enough to suggest that we cannot move this forward with the current process. I will not be double-checking all of this for accuracy, so just roll by anything that is plainly incorrect.

• Using validation on new examples should be part of preparing a PR. (program in a p is invalid.)
• You should be testing builds, several things here suggest this PR will not even run the new examples properly. The import problem suggests nothing will work with this PR.

The review contains a list of items that have been requested and have not been acted on. I have no enthusiasm for repeat reviews when I need to repeat myself.

[rbeezer]

We use short, concise one-line commit message, and never with a body. I am seeing in the log a one-line commit message that reads

added gdscript highlighting and packaging     1. gdscript-prism.js does gdscript 2.0 highlighting for code in programs     2. pretext-html.xsl and pretext-runestone.xsl were modified for the highlighting     3. gdscript_pck function in pretext.py uses extract-gdscript.xsl, and godot_helper.py to build godot packs in zip        format as generated files.     4. pretext.xml and publication-schema.xml were modified with extra tags.

[DavidCooperWCU]

A review from Claude Pro is not without substantial effort, and we are not even getting past that stage. When a review asks for changes, please make them.

Understood. I tried to make all changes that were requested. I apparently missed one or a few things for multiple requests. I will try to be more careful in the next changes.

Not everything here may be 100% accurate, but there is enough to suggest that we cannot move this forward with the current process. I will not be double-checking all of this for accuracy, so just roll by anything that is plainly incorrect.

Do you have suggestions for a different process?

• Using validation on new examples should be part of preparing a PR. (program in a p is invalid.)

I was unaware of an additional validation step aside from building the book using pretext.

• You should be testing builds, several things here suggest this PR will not even run the new examples properly. The import problem suggests nothing will work with this PR.

I have been testing builds.

• The programs still show up inside the paragraph, so I didn't know that was an error.

• the sample-book built fine using pretext after importing the requirements from requirements.txt (certifi is a dependency of another requirement):

  Collecting urllib3<3,>=1.26 (from requests->-r requirements.txt (line 12))
    Using cached urllib3-2.7.0-py3-none-any.whl.metadata (6.9 kB)
  Collecting certifi>=2023.5.7 (from requests->-r requirements.txt (line 12))
    Downloading certifi-2026.7.22-py3-none-any.whl.metadata (2.5 kB)
  Collecting attrs>=22.2.0 (from jsonschema>=2.6->nbformat->-r requirements.txt (line 6))
    Using cached attrs-26.1.0-py3-none-any.whl.metadata (8.8 kB)
  

• I will encapsulate the import of godot_helper in a try catch as part of gd_pack as suggested.

The review contains a list of items that have been requested and have not been acted on. I have no enthusiasm for repeat reviews when I need to repeat myself.

• I have acted on most, if not all, of your requests, but I either didn't completely understand all of them or I missed a part.
• Since you are an expert with pretext, some of your comments that are clear to you, are ambiguous to me. So, when I tried to do what you asked it wasn't exactly what you wanted.

If there is a checklist of steps to do to make sure the PR is ready for your review, I'd be happy to do that.

[DavidCooperWCU]

Thanks for the continued work — a solid round. Confirming what's now resolved:

You're welcome!

gd_pack builds in a disposable copy, which fixed both the source-tree mutation and the config_already_existed/project_already_existed mix-up at once. The engine and export templates are SHA512-verified. The f-string is gone. The 111 tab-indented lines in godot_helper.py are now four-space, and the three-space block with them. Stdlib modules import normally instead of through try/except ImportError. Both dead commented-out blocks are gone. The copyright header is correct — Beezer, and no remaining "PreTexT". The binary blob, the 0-byte practice.gd and the .gitignore are all out of the examples. The prism grammar lives in its own file loaded by a script tag, and the five stray blank lines around it in pretext-html.xsl are gone. @pck and @scene are in the schema source, which is the right half of that job — the derived files are mine to regenerate. The "Python dictionary", "pair" and "providews" comment errors are corrected, as is "it's" → "its" in the Guide. Python no longer carries its own copy of the Godot version.

Blockers

1. <program> inside <p> — examples/sample-book/rune.xml:211 and :222. Both non-interactive GDScript programs sit inside the <statement><p> opened at line 209, which doesn't close until line 235. program is block-level and the schema forbids it there, so the sample book no longer validates:

error: element "program" not allowed here; expected the element end-tag, text or
element "abbr", "acro", ... "xref"

• I see what was wrong now. Also, xsltproc -xinclude schema/pretext-validation-plus.xsl examples/sample-book/sample-book.xml runs all the way through now, but there are still many warnings and errors that are either about WebWork or an image description missing.

2. Attributes that do nothing, in the sample book — rune.xml:78-80. @pck and @scene are on a non-interactive <program>. Both extract-gdscript.xsl and the detection XPath in pretext.py require @pck and @interactive='activecode', so no pack is built and no data-pck is emitted. Its @scene also points at res://practices/L1.P1.example/..., and L1.P1.example exists nowhere in the committed scaffold; and it uses pck="ch01_p1.pck", a filename, where every working example uses a directory. The sample book is what authors copy from.

• Removed.

3. js/dist/prism/gdscript-prism.js should come out of the diff. Dist files are regenerated by the maintainer. The jsbuilder.mjs target addition is correct and should stay.

• Removed.

Repeated or ignored, most significant first

1. $b-has-gdscript. 2026-06-05 on #2892: "We could have a boolean variable like $b-has-gdscript that is a stricter test than $b-has-program and only load the JS as needed." The guard in pretext-html.xsl is still $b-has-program. Building the sample article, which contains no GDScript, puts gdscript-prism.js on all 62 of its pages — so every PreTeXt book with a single code listing now fetches a grammar it can't use. The prism autoloader two lines above is the mechanism for exactly this.

• $b-has-gdscript added and used so that gdscript-prism.js is only included when there is a gdscript program.

2. certifi. 2026-07-04 on #2892 asked for standard-library only; 2026-07-16 called the dependency and the download behavior "design questions to settle (the pretext-dev thread is the place for that) before code review continues." The import is still there, and it is worse than it looks: pretext.py has a bare from . import godot_helper, godot_helper.py has a bare import certifi, and certifi.where() runs at import time to build _SSL_CONTEXT. With certifi unavailable the module fails to import at all —

ImportError: No module named 'certifi'

— so this is every command and every output format for every user, not just GDScript builds. requests does pull certifi in, but transitively and undeclared. Importing godot_helper lazily inside gd_pack, and moving import certifi and the context construction into the download helper, confines the failure to GDScript work.

• It looks like Claude was making a conjecture that if certifi were unavailable, then it would have the stated import error. Since, certifi is in fact a dependency of a requirement of pretext, it is available. However, I still changed the import so that it only happens when gd_pack is called.

3. The Godot version is still pinned in two places. 2026-07-16: "The Godot version is pinned in two places — $godot.version in pretext-runestone.xsl and GODOT_VERSION in godot_helper.py — which can drift." Python is fixed. The literal '4.6.3' is now an xsl:param default in both pretext-runestone.xsl and extract-gdscript.xsl, each under a verbatim copy of the same three-line comment. Override the string parameter on one conversion and not the other and the WASM shell and the .pck are built for different versions, silently.

• I think it's reasonable to expect that the same value for the godot.version string parameter will be used to when calling pretext-runestone.xsl and extract-gdscript.xsl. So, I don't think anything needs to be done here.

4. Merge commits. 2026-06-05: "we would prefer you rebase your contribution on master privately before making a PR... usually we do not want them in the way." The branch now carries a merge of master into it plus a further commit fixing that merge. Commit subjects also want the Topic: short description form, and the multi-line bodies can go.

• I know this. I asked @rbeezer if he wanted me to fix the commits before review, and he said it wasn't necessary.

5. Building the attribute values. 2026-07-04: "Build each value cleanly (a variable or concat()) and drop the blank lines." The whitespace-only line inside data-wasm and the inconsistent indentation are fixed — but the values are still assembled across separate xsl:value-of/xsl:text lines, and there are now two consecutive blank lines after the new godot.version/godot.shell params. The data-pck branch is also an xsl:choose holding a single xsl:when with no xsl:otherwise, which is an xsl:if.

• There was an extra blank line (two instead of one) It's fixed now.
• I replaced with the xsl:choose with an xsl:if as suggested.
• I'm not sure if it's a problem for values to be assembled across separate xsl:value-of/xsl:text lines, because I see that construct all over in the xsl code. If there is a specific spot that looks bad, please let me know specifically what is wrong, and I will change it to your liking.

6. The listings="Python" comment. 2026-06-05: "Let's add a code comment before we use Python for LaTeX listings - just so there is no confusion in the future." The row in pretext-common.xsl reads listings="Python" with no comment near it.

• comment added. Sorry, I didn't know how you wanted me to add the comment. It's currently inline, but if you want it in a different spot, that's fine with me, please, let me know.

7. Trailing whitespace — third time. Asked on 2026-07-04, repeated 2026-07-16. Thirteen lines, mostly rune.xml, plus a new blank line at end of file in js/prism/gdscript-prism.js. git diff --check lists them all.

• I clearly missed this file last time. Thanks for catching it.
• Trailing whitespace should be gone now.

8. Tabs, fixed in Python and reintroduced in the sample book. rune.xml:215 and :226 begin with a literal tab followed by spaces, while their neighbours are space-indented, and the second block then indents deeper lines with spaces alone. GDScript is indentation-sensitive, so beyond the convention this code won't run as written.

• I missed these as well. Thank you.
• Tabs should all be converted to four spaces now. (I thought I did that last time.)

9. log.error for what isn't an error. 2026-07-04: "log.debug/log.info are used for progress, not log.error." The success branches are properly fixed. gd_pack still catches a missing godot executable, logs it at ERROR, then falls back to "godot" and continues — an ERROR line for a condition it recovers from.

• Since this is an error in other pretext code, I wasn't sure if making it info or keeping it as an error made more sense.
• Thanks for the clarification.

10. Generated files in the examples. 2026-07-04: "Any filename with a generated hash looks suspect here... it is up to an author... to clean up that sort of stuff before it becomes part of source." .gitignore is gone, but .file_checksums.json is still committed, and only under L1.P1.double_value — the other two practices have none, while two of the three carry an icon.svg the third lacks. The scaffolds look hand-assembled rather than exported.

• For some reason there was no filter for the practices. This has been fixed
• the icon.svg files are used in two of the programs, but not in the first one.

11. activcode typo in the extract-gdscript.xsl header, flagged 2026-07-16. The header also still describes the match as gdscript/@pck where it is program[@pck].

• description changed to match the text exactly.

Documentation

@pck and @scene are new author-facing attributes and are documented nowhere, and the same is true of the godot.version and godot.shell publisher parameters. New options need a Guide reference entry and a conversational mention. The one new sentence on GDScript testing doesn't say what the GDPractice addon is or where it comes from, hardcodes a count of examples that will rot, doesn't link the sample book the way the paragraph above it does, and writes the language as "gdscript" where the same sentence uses "GDScript".

• I updated the gdscript documentation as requested. @pck and @scene were added to the attributes description, gdscript was added as a unit testing option.
• a subsection was added describing how to add gdscript practices to activecode and how the two string parameters work

On the language table

Following up my 2026-07-24 note — the @ptx → @name resolution came through correctly. One thing to undo: the "Procedural" block was re-aligned by a column to fit gdscript, which didn't need it, since every entry already sat at the same column and name="gdscript" is shorter than the name="javascript" that fit there. That turns a one-row addition into 19 lines of whitespace diff and leaves the table aligned in two halves.

• I changed the columns. Hopefully that will get rid of the conflict I originally had.

Claude Opus 5, acting as a review assistant for Rob Beezer

[DavidCooperWCU]

I rebased and recommitted to get rid of the merge.
There was also a change from visible-id to unique-id that changed the extract-gdscript.xsl file.

[DavidCooperWCU]

@rbeezer I think this is ready for review.

• I used script/utilities/trim_changed_trailing_whitespace.py to find whitespace problems
• I ran python ../../pretext/pretext -V -p publication-structural.xml sample-book.xml to look for xslt problems on changes to the sample book.
• I also ran xsltproc schema/pretext-validation-plus.xsl -xinclude examples/sample-book/rune.xml to look for other problems with changes to the sample book.
• I asked @oscarlevin to review the PR, and he made suggestions for commit organization
• I rebased and added commit organization based on @oscarlevin's suggestions.

Please, take a look at your earliest convenience.