Skip to content

REPORT.md

Latest commit

 

History

History
1226 lines (1027 loc) · 68 KB

REPORT.md

File metadata and controls

1226 lines (1027 loc) · 68 KB
Raw

Apple Pages IWA reverse-engineering report

This document is a development-time reverse-engineering record. It preserves the format evidence, implementation rationale, and validation findings.

Date: 2026-07-19 (Asia/Tokyo)

Progress

Stage Status Result / evidence
A. Paragraph formatting COMPLETE API, synthesis, tests, decoded-IWA audit, and output/verify_a.pages complete
B. Document settings COMPLETE Single-section geometry and six header/footer variants; output/verify_b.pages complete
C. Text insertion/deletion COMPLETE / MANUAL PASS Control-character page breaks, paragraph/run deletion, and style-boundary regression fixes passed Pages PDF rendering
D. Creation/properties COMPLETE Bundled empty template, Document(), core-properties facade, verify and audit complete
E. Tables COMPLETE / MANUAL PASS Existing table read/write passed rendering; full graph-cloning add_table() and verify_h_add_table.pages are structurally complete
F. Pictures COMPLETE / MANUAL PASS Corrected full/thumbnail replacement passed rendering; graph-cloning add_picture() and verify_h_add_picture.pages are structurally complete
G. Remaining API/ledger COMPLETE / MANUAL PASS Hyperlink/style verify passed rendering; exhaustive 59-file/662-method upstream ledger complete
H. Final graph construction CORRECTED add_picture passed; stale 2x2 UID/sidecar/header dimensions in add_table are corrected and verify_h_add_table.pages regenerated
I. Scripts/highlight CORRECTED Bisect proved type-2021/field-8 invalid; final verify matches the real type-2022/field-5 paragraph-background graph
J. Comments/annotation COMPLETE Real type-2013/3056/212 graph parsed; ranges, author color, replies, and safe text edits exposed
K. OSS packaging COMPLETE Distribution renamed to python-pages, import package to pages, public docs scrubbed, and private fixtures excluded

add_picture, subscript, and superscript passed PDF rendering checks. Bisect A (text insertion only) preserves page-7 decoration while negative-control B (type-2021/field-8 background) makes Pages discard that page's character decoration. The final verify_i_highlight.pages was rebuilt with the Pages-authored type-2022/field-5 paragraph representation. New-comment graph creation remains explicitly excluded for the reasons in Stage J.

Outcome

This project now has a dependency-free, three-layer Python library validated against three private modern .pages fixtures. The low layer performs lossless protobuf/IWA/ Snappy operations, the middle layer manages containers, package metadata, stylesheets, and text run tables, and api.py provides a python-docx-shaped Document -> paragraphs -> runs -> font API. Bold, italic, underline, strikethrough, subscript, superscript, font name, point size, sRGB font color, standard-palette paragraph background, and existing comments are readable. Character formatting is writable with tri-state inheritance where applicable. Stage A additionally implements paragraph alignment, line spacing, before/after spacing, three indents, and tab stops through a python-docx-shaped ParagraphFormat API. Stage B adds a singleton sections collection with page geometry, margins, orientation, and first/even/primary header/footer access. Existing comment and reply text can be safely edited in place. The final graph cloner adds fresh object/component IDs, UUID-map entries, cross-component links, attachment runs, and package data references for new tables and inline PNG images based on the Pages-authored feature fixture.

The previously missing renderer requirement is now established: a synthesized style is added to TSS.StylesheetArchive.styles, to its parent's children in parent_to_children_style_map, and to the stylesheet MessageInfo's packed object_references. The user manually confirmed in Pages on 2026-07-19 that a file patched with these three links renders the synthesized bold style. Pages was not launched or automated during this implementation.

OSS package identity

The distribution is python-pages, in homage to python-docx, while the import package is pages. Source modules live under src/pages, the console command is python-pages, and setuptools includes the bundled minimal template through the pages package-data declaration. samples/, output/, reference clones, and crash reports remain excluded from Git and distributions.

Core-property writes now use Metadata/PythonPagesCoreProperties.plist. The reader retains an explicit compatibility fallback for the former extension member, but no old project-name import package or CLI alias is shipped. Public documentation contains no private fixture body text; exact text assertions remain only in local tests because the underlying fixtures are not distributed. An unrelated PyPI distribution already owns the pages distribution/import name, so public instructions require an isolated environment and warn against co-installing it with python-pages.

At that stage, rename verification covered the then-current 116 local tests, a clean import pages, Document() loading the relocated bundled template, CLI help and fixture inspection, TOML metadata parsing, and Git ignore checks for private/generated packages. A development-environment wheel build was attempted with build isolation disabled, but that host Python had no setuptools backend installed and external dependency fetches were disabled.

The bundled default.pages was audited separately because it is package data. Its body and three previews are blank, but an unreachable legacy image object still carried a QR-code PNG. That payload was replaced with a deterministic 216×216 white RGB PNG, its DataInfo digest/length were updated, and both stored filenames were changed to template-placeholder.png variants. The sanitized template remains an empty 7-IWA/636-segment package and passes both ZIP checks.

Rendering correction after the first output

A Pages rendering check showed that the first generated fixture A opened normally but the target text did not render bold. That version referenced named style 1731516, whose character properties contained only bold=1. This is consistent with a document-specific font family that does not style-link its regular face to its bold face.

The current compatibility helper no longer relies on trait-only font resolution. It synthesizes an anonymous style with field 5 explicitly set to the document's bold PostScript face, while retaining field 1 bold=1, and performs all stylesheet and PackageMetadata registration in the same transaction.

Archive and framing findings

Both samples are single ZIP files with direct Index/*.iwa members. Neither contains a nested Index.zip; the loader and writer nevertheless support that older package layout too.

Each IWA member is a sequence of custom iWork Snappy frames:

  1. byte 0x00;
  2. a 24-bit little-endian compressed byte count;
  3. one raw Snappy block (not the standard Snappy framed stream).

The concatenated decoded bytes contain repeated archive segments. Each segment starts with a protobuf varint header length, followed by a TSP.ArchiveInfo, then the payload bytes described by its repeated TSP.MessageInfo records. MessageInfo.type selects the schema and MessageInfo.length gives each payload length. Object dependencies are packed uint64 values in MessageInfo.object_references field 5.

Implemented components:

  • src/pages/snappy.py: all four raw Snappy tag types, custom frame decode, and a correctness-first literal-only encoder.
  • src/pages/protobuf.py: lossless protobuf wire parser/encoder for varint, fixed32, fixed64, and length-delimited fields. Untouched fields retain their exact original encoding and order.
  • src/pages/iwa.py: ArchiveInfo/MessageInfo segmentation, payload-length updates, and IWA reassembly.
  • src/pages/pages.py: outer ZIP/direct Index/nested Index.zip handling.

Parsing inventory:

Sample IWA files Archive segments Body storage ID Body code points
private_fixture_a.pages 7 636 1732514 5,901
private_fixture_b.pages 7 606 1732514 10,148

Schema extraction and reconstruction

The binary in a locally installed Pages application bundle was read statically only. It was never launched, loaded, automated, or addressed through AppleScript. Its static Info.plist identifies Pages 15.1.1, build 7044.0.273. The relevant generated protobuf metadata and Objective-C archiving strings are in:

Contents/Frameworks/TSText.framework/Versions/A/TSText

Static string extraction confirmed the current binary contains TSWP.StorageArchive, TSWP.CharacterStylePropertiesArchive, TSWP.CharacterStyleArchive, TSWP.ParagraphStyleArchive, and fields including bold, font_name, font_name_null, and the related undefined markers. tools/inspect_schema.py reproduces this static-only inspection.

Field numbers were reconstructed by combining the binary names, protobuf-wire inspection of both samples, and comparison with the open-source iWork schemas used by numbers-parser and other iWork readers. The useful partial schemas are checked in as schema/TSPArchiveInfo.proto and schema/TSWP_partial.proto. They cover the messages used by the editor and intentionally mark unneeded submessages as opaque bytes.

Primary external format references consulted:

Stage A — paragraph formatting

Analysis

Three sources were compared before implementation:

  • real type-2022 paragraph styles and the body field-5 attribute table from both supplied Pages packages;
  • static strings in the Pages TSText framework, including ParagraphStylePropertiesArchive, LineSpacingArchive, TabArchive, and each property name below;
  • ref/numbers-parser/src/protos/TSWPArchives.proto and its alignment mapping in numbers_parser/cell.py. No ref/keynote-parser checkout is present in this workspace, so it could not supply an additional local schema source.

TSWP.ParagraphStyleArchive is IWA type 2022. Its field 11 is optional direct character formatting and field 12 is ParagraphStylePropertiesArchive:

Field Wire form Meaning
1 varint alignment: left=0, right=1, center=2, justified=3, auto=4
4 fixed32 default tab interval (decoded, not yet exposed)
7 fixed32 first-line indent in points
11 fixed32 left indent in points
12 varint line-spacing-null marker
13 message line spacing: mode field 1, amount fixed32 field 2, baseline rule field 3
19 fixed32 right indent in points
20 fixed32 space after in points
21 fixed32 space before in points
24 varint tabs-null marker
25 message repeated tabs; position field 1, alignment field 2, leader field 3

Line-spacing modes are relative=0, minimum=1, exact=2, maximum=3, and space-between=4. Tab alignment is left=0, center=1, right=2, decimal=3. Observed sample tabs include a 57.5-point stop and a 451-point right-aligned stop. Named Body and Caption styles contain complete inherited defaults; anonymous styles contain direct subsets and refer to their named parent.

The field-5 paragraph attribute table can contain more than one boundary inside one newline-delimited paragraph (the first sample paragraph has entries at UTF-16 offsets 0 and 1). A paragraph edit therefore preserves every offset and synthesizes a corresponding derivative for each active parent/character context. It must not apply the adjacent-style compaction used for character runs. The decoded-IWA audit caught and prevented that loss before completion.

Implemented behavior

  • ParagraphProperties, LineSpacing, and ParagraphTab retain unknown paragraph-property fields while exposing the known values.
  • Anonymous type-2022 styles are synthesized from a real paragraph template, retain their source parent and direct character bytes, compute dynamic override_count, and use a (parent, character bytes, properties) cache.
  • Every new paragraph style receives the same stylesheet three-point registration and PackageMetadata UUID/external-reference bookkeeping as a character style.
  • WD_ALIGN_PARAGRAPH, WD_LINE_SPACING, WD_TAB_ALIGNMENT, WD_TAB_LEADER, Length, Pt, Inches, and Cm are exposed by api.py.
  • Paragraph.alignment and Paragraph.paragraph_format support alignment, fixed or multiple line spacing, spacing before/after, left/right/first-line indents, and sorted tab-stop addition/clearing. None removes a direct override and restores inheritance.

Deliberately deferred Stage-A properties are pagination controls, keep-with- next/keep-together/widow control, contextual spacing, borders/background, list formatting, default tab intervals, mutable individual TabStop objects, and indexed tab deletion. Their schemas or APIs are not required by the master's Stage-A surface; they remain candidates for Stage G. DOCX XML mutation assertions are excluded because Pages has no WordprocessingML representation.

Verification

tests/test_paragraph_format.py adds thirteen behaviors adapted from python-docx paragraph-format and tab-stop expectations, including all tri-state setters, save/reload, cache and three-link registration, preservation of direct character properties, and preservation of explicit paragraph-table boundaries and opaque-field retention. The complete suite passes 38/38 under unittest discover.

output/verify_a.pages centers the second paragraph, uses exact 24-point line spacing, 12/8-point before/after spacing, 18-point left/right indents, a 9-point first-line indent, and a 72-point centered dot-leader tab. Reparse and ZIP validation passed (zipfile.testzip() == None, unzip -t clean, seven IWA files and 637 segments).

The decoded-IWA diff changed only the intended members and objects:

Member Decoded before Decoded after Objects
Index/Document.iwa 39,133 39,138 storage 1732514 only
Index/DocumentStylesheet.iwa 340,187 340,302 stylesheet 1732588 + new type-2022 style 44990410
Index/Metadata.iwa 22,381 22,422 PackageMetadata object 2 only

Body text and all paragraph-table boundary offsets are unchanged. The storage delta is only the replacement style reference plus its MessageInfo object reference; the stylesheet and metadata deltas are the required registration.

Stage B — document settings and headers/footers

Analysis

The Pages executable's embedded TPArchives.proto descriptor was decoded statically from disk; the application was not launched. It identifies the package root as TP.DocumentArchive (IWA type 10000) and gives the following current fields, all observed in both samples:

Field Wire form Meaning Fixture A value
30 fixed32 float page width 499 pt
31 fixed32 float page height 709 pt
32 fixed32 float left margin 11.9055 pt
33 fixed32 float right margin 11.9055 pt
34 fixed32 float top margin 14.1732 pt
35 fixed32 float bottom margin 11.9055 pt
36 fixed32 float header distance 35.4331 pt
37 fixed32 float footer distance 42.5197 pt
38 fixed32 float page scale 1.0
42 varint orientation: portrait=0, landscape=1 0

The BODY storage's MessageInfo references one TP.SectionArchive (type 10011, object 1732271). Its fields 23, 24, and 25 reference first/even/odd TP.SectionTemplateArchive objects (type 10143). Each template has three header references in repeated field 1 and three footer references in repeated field 2. These are the Pages left/center/right fragments. All 18 referenced objects are existing kind=1 TSWP.StorageArchive objects with a paragraph style boundary at UTF-16 offset zero; the supplied documents leave their text fields empty.

numbers-parser does not contain the Pages-only TPArchives.proto; its TN.DocumentArchive and sheet print schema provide analogous size/margin and header/footer structures but are not used as field-number authority here. No local keynote-parser checkout exists. The embedded descriptor and both real sample graphs agree exactly. The reconstructed subset is checked in as schema/TP_partial.proto.

Implemented behavior

  • New middle layer document.py resolves the type-10000 root, the section dependency, all three templates, and their six groups of storage fragments.
  • Document.sections returns one Section, matching Pages' package-wide page setup. Width, height, four margins, header/footer distances, and orientation are readable and writable as point-based Length values.
  • WD_ORIENTATION and WD_ORIENT expose portrait/landscape enum values.
  • header, footer, first_page_header/footer, and even_page_header/footer expose their three Pages fragments through .paragraphs; each paragraph supports text reading and writing.
  • different_first_page_header_footer reads/writes SectionArchive field 18.

Pages multiple sections are deliberately represented as one API section because page geometry lives on TP.DocumentArchive, as permitted by the master plan. Previous-section linking, section start kinds, gutters, mirrored margins, and header/footer creation/deletion have no implemented Pages mapping and are deferred/excluded. Arbitrary replacement of a header/footer that already contains nonzero attribute boundaries is deferred to Stage C's general offset engine; the current writer handles the supplied existing zero-boundary storages and rejects unsafe cases explicitly.

Verification

Ten Stage-B tests adapted from python-docx section behaviors cover the singleton collection, all geometry values and setters, orientation, first/even/primary variants, header/footer text, section flags, real object resolution, ZIP save, and reload. The full dependency-free suite passes 48/48.

output/verify_b.pages changes the document to 709×499-point landscape, 36-point left/right margins, 42-point top/bottom margins, 18-point header/footer distances, and adds visible primary header/footer strings. Self-reparse reports seven IWA files and 636 segments; zipfile.testzip() and unzip -t both pass.

The decoded-IWA diff changes only Index/Document.iwa, from 39,133 to 39,189 bytes, with the same 102 segments. Object-level changes are exactly:

  • object 1/type 10000: eight geometry/orientation fields;
  • object 1739944/type 2001: primary header text field 3;
  • object 1740019/type 2001: primary footer text field 3.

No body text, attribute table, stylesheet, PackageMetadata, or unrelated object changed.

Stage C — text insertion/deletion

Static TSWP.StorageArchive schema extraction and the numbers-parser proto agree on four offset-bearing table shapes. Object tables are fields 5, 7–9, 11–12, 15–18, 21–23, and 27–28; ParaData tables are 6, 14, and 24; string tables are 19–20; overlapping TSP.Range tables are 25–26. All use UTF-16 code-unit positions. Both supplied BODY objects currently use fields 5–9, 12, 14, 17–19, 24, and 28.

The offset engine rewrites every indexed table after a text replacement, preserves a boundary exactly at the replacement start, removes interior boundaries, shifts the end and all later boundaries, and adjusts both ends of overlapping ranges. Astral Unicode is covered by tests. Text remains in the existing first field-3 chunk; the supplied samples each have one chunk.

Completed and audited sub-stages:

  • C1 middle-layer arbitrary-length replacement: output/verify_c1_replace.pages;
  • C2 public Run.text setter: output/verify_c2_run_text.pages;
  • C3 Paragraph.add_run(): output/verify_c3_add_run.pages;
  • C4 Document.add_paragraph(text, style) with named style resolution: output/verify_c4_add_paragraph.pages;
  • C5 Document.add_heading(text, level) using Title, Heading, Heading 2, and Heading 3: output/verify_c5_add_heading.pages;
  • C6 Document.add_page_break() mapped to the Pages body control character 0x05: output/verify_c6_page_break.pages;
  • C7 Run.clear() / Run.delete() and Paragraph.clear() / Paragraph.delete(): output/verify_c7_delete.pages.

C1–C7 decoded diffs change only Index/Document.iwa, object 1732514/type 2001, and retain 636 total segments. C6 no longer synthesizes a paragraph style or changes the stylesheet/PackageMetadata.

page_break_before field 14 exists in the static Pages 15.1.1 protobuf descriptor and numbers-parser schema, but user PDF rendering proved that it is not the representation used by Pages' Insert Page Break operation. The feature fixture contains 0x05 at UTF-16 offset 74 with no special character or paragraph-style boundary at that offset; this is the effective page-break encoding. The supplied theme defines Title and Heading levels 1–3. Since Pages has no fixed nine-level docx hierarchy, API levels 4–9 deliberately fall back to Heading 3; levels outside 0–9 are rejected like python-docx.

The first C6 implementation and C7 deletion could create two field-5 entries at the same UTF-16 offset. Pages treated the resulting paragraph-style table as invalid, flattening existing headings outside the edited range. The indexed table rewriter now keeps the later restore boundary when deletion produces a collision, and C6 does not touch field 5. Regression tests assert unique offsets and preservation of existing heading style IDs.

All seven verify packages reparse as seven IWA files, pass ZipFile.testzip() and unzip -t, and retain 636 segments. The corrected full suite passes 74/74. Run.text continues to reject embedded paragraph separators; paragraph insertion and page breaks have explicit APIs, avoiding ambiguous attribute-table behavior.

Stage D — document creation and core properties

Schema and package analysis

The smaller package is private_fixture_a.pages (106,152 uncompressed package bytes versus 427,302 for private_fixture_b.pages), so it is the template source. Its body was reduced to one empty paragraph using the Stage-C UTF-16 offset engine, saved, reloaded, and saved again as the no-op-normalized src/pages/data/default.pages. The 51,611-byte template remains a complete seven-IWA Pages package with 636 segments. Relative to its source, only body StorageArchive object 1732514/type 2001 changes; all interior attribute-table boundaries are removed with the text.

Pages' native metadata was checked in all three requested sources: both sample packages, the static Pages/TSKit protobuf strings, and numbers-parser's TSPArchiveMessages.proto. Type 11011 is not a Dublin Core record:

Location / field Meaning
TSP.DocumentMetadata 1 collaboration-mode bool
TSP.DocumentMetadata 3 data digest/version records
Metadata/Properties.plist package UUIDs, revision, format version, multipage/external-data flags
Metadata/DocumentIdentifier native package document UUID

The exact type-11011 subset is recorded in schema/TSPDocumentMetadata.proto. Neither native location has fields for title, subject, author, keywords, category, comments, or the python-docx date properties. Annotation authors describe comment/activity identities and are not document authors, so mapping core_properties.author there would be semantically wrong.

Implemented behavior and explicit compatibility extension

  • Document() opens the bundled empty template; package-data configuration includes it in installed distributions.
  • Document.core_properties exposes all python-docx string fields, created, modified, last_printed, and positive integer revision.
  • Strings retain python-docx's 255-character limit. Datetimes normalize to UTC and revisions use the same positive-integer setter rule.
  • Because Pages has no native equivalent, values are stored in the explicit Metadata/PythonPagesCoreProperties.plist extension. Pages-native UUID/revision metadata is not overloaded. This member is library metadata: Pages itself neither displays nor edits these Dublin Core values.
  • tools/diff_iwa.py now reports added and removed ZIP members as well as changed existing members; previously an added metadata member was omitted from its audit set.

output/verify_d.pages was built from Document(), adds a visible heading and body paragraph, and sets representative core properties. Against the bundled template its decoded diff changes only body object 1732514/type 2001 and adds the named extension plist. It retains seven IWA files and 636 segments. The package self-reparses, ZipFile.testzip() returns no bad member, and unzip -t passes. A no-op Document().save() has an empty member diff. The full suite passes 72/72.

Excluded as having no Pages counterpart: native Pages display/edit behavior for Dublin Core properties. The compatibility facade and round-trip format are implemented, but claiming that these values are Pages-native would be incorrect. The template intentionally follows the requested package-clone model and retains its native document UUID; unique native identity generation would require coordinated revision-identifier rewrites across Properties, PackageMetadata, and view/history archives and is deferred rather than making an inconsistent partial rewrite.

Stage E — tables

The initial inventory compared both supplied packages with the static TSText schema strings and numbers-parser's TSTArchives.proto / table model code. Both Pages packages contain exactly the same unused table-theme defaults:

Type Count per sample Meaning
6003 6 table styles
6004 102 cell styles
6008 6 table style presets
6247 12 table-related style network defaults
6366 1 table-related default container

Neither sample contains any type 6000 TableInfoArchive, 6001 TableModelArchive, 6002 Tile, 6005 TableDataList, 6006 HeaderStorageBucket, 6007 WPTableInfoArchive, or 6011 TableDataListSegment. Consequently there is no real row/column count, tile storage buffer, string table, rich-text payload, WP text-flow owner, or table object reference to dump or mutate.

The static schema does establish the expected read chain: type 6007 wraps type 6000 and a TSWP storage reference; type 6000 field 2 points to type 6001; TableModel fields 6/7 are row/column counts and field 4 is DataStore; DataStore fields 3/4 point to tile storage and the string table; type-6002 row info uses fields 6/7 for the packed cell buffer and offsets. numbers-parser can decode that binary cell format, but it operates on real model/tile objects and cannot manufacture the missing Pages WP ownership graph from style defaults.

The former fixture blocker is resolved by samples/fixture_features.pages, which contains a 2x2 table with one populated, bold B2 cell. The earlier empty-sample inventory remains useful as proof that table theme defaults are not themselves document tables.

Fixture ownership and field map

Pages does not wrap this inline table in type 6007. Body StorageArchive field 9 has an object run at UTF-16 offset 73 to attachment 1734178/type 2003, whose field 1 points directly to TableInfo 1733304/type 6000. Its resolved graph is:

1734178 -> 1733304 (TableInfo) -> 1733299 (TableModel) -> embedded DataStore.

Archive Field Meaning in fixture
TableInfo/type 6000 2 TableModel reference
TableModel/type 6001 4 embedded DataStore
TableModel/type 6001 6 / 7 / 8 rows=2 / columns=2 / Table 1
TableModel/type 6001 9 / 10 / 11 header-row/header-column/footer-row counts
TableModel/type 6001 24–27 default body/header/footer text styles
DataStore 3 embedded TileStorage: repeated field 1 tile links, field 2 tile size
DataStore 4 / 5 string/style TableDataList references
Tile/type 6002 5 repeated TileRowInfo
TileRowInfo 1 / 2 tile-local row index / stored cell count
TileRowInfo 5–8 storage version, cell bytes, signed int16 offsets, wide-offset flag
TableDataList/type 6005 1 / 2 / 3 list type, next key, repeated keyed entries

Only B2 is materialized. Tile 1733269 has tile row index 1, offsets [-1, 0, -1 ...], and one 28-byte version-5 text-cell buffer. Its flags are 0x21048: string key 1, text-style key 1, suggestion key 5, and text-format key 1. String key 1 resolves through DataStore field 4 to the populated B2 cell. Text-style key 1 resolves through field 5 to anonymous paragraph style 1734152/type 2022. That style has direct CharacterProperties bold=1 and inherits the remaining HelveticaNeue 10 pt properties from parent 1731527. This agrees with numbers-parser's version-5 cell decoder and effective style-parent lookup.

API and write path

New tables.py is the middle-layer TST graph/tile/data-list reader. The public API now exposes Document.tables, Table.rows, Table.columns, Table.cell(row, col), TableCell.text, TableCell.paragraphs/runs, and the effective read-only cell run font properties. Omitted cells read as empty. The populated string cell can be changed without rewriting its tile: the setter updates only its uniquely referenced string TableDataList entry. A save/reload test proves that the new text and the B2 effective bold property both survive.

output/verify_e.pages changes B2 to a neutral replacement string while retaining its bold style. Its decoded diff changes exactly Index/Tables/DataList-1733266-2.iwa, object 1733266/type 6005, from 43 to 65 decoded bytes. It retains 37 IWA files and 639 segments; self-reparse, zipfile.testzip(), and unzip -t all pass.

Creating storage for an omitted cell in an existing sparse table remains separate from table construction. Final Stage H implements add_table() by cloning the complete fixture graph instead of manufacturing a partial graph; see the Stage-H audit below.

Stage F — inline pictures

The fixture body attachment table has a second run at UTF-16 offset 75: attachment 1734500/type 2003 field 1 points to ImageArchive 1734421/type 3005. Its MessageInfo references two geometry helpers (1734422/1734423, type 3097) and image style 1731560/type 3016. The relevant data fields agree exactly with TSDArchives.proto:

ImageArchive field Meaning Fixture value
3 image style reference 1731560
4 original logical size 481.894 × 481.894
9 natural pixel size 1024 × 1024
11 full-size TSP.DataReference DataInfo 24
12 thumbnail TSP.DataReference DataInfo 25

PackageMetadata repeated field 4 holds the two DataInfo records. DataInfo uses field 1 identifier, field 2 SHA-1 digest, fields 3/4 preferred/stored filename, field 10 image attributes, and field 18 materialized byte length. DataInfo 24 maps to the 1,303,354-byte 1024px PNG; DataInfo 25 maps to the 98,264-byte 256px editing thumbnail.

New pictures.py resolves only body-attached type-3005 images and exposes the full/thumbnail package data. Document.inline_shapes (with pictures alias) provides filename, bytes, natural width/height, and replace(path, thumbnail_path=None). A PNG replacement updates the full DataInfo digest/length, its pixel-size attribute, and ImageArchive natural size. For sources no larger than 256px the same PNG is also a valid thumbnail. Larger sources either take an explicitly supplied thumbnail matching Pages' 256px maximum scaling rule, or preserve the original Pages thumbnail unchanged.

The first Stage-F verify passed ZIP/IWA validation but hung or crashed Pages twice during the user's manual test. Binary comparison identified a concrete contract violation missed by structural checks: both DataInfo 24 and thumbnail DataInfo 25 contained the same 1024×1024 PNG, whereas the Pages-authored graph uses 1024×1024 and 256×256 respectively. They consequently also had the same SHA-1 digest and materialized length under two distinct data identifiers. The supplied crash summary places the SIGABRT assertion on TSPDataManager.Datas, while QuickLook thumbnail generation was active; this strongly ties the failure to that duplicated data identity/invalid thumbnail pair rather than the IWA container. The corrected writer validates the expected thumbnail dimensions before mutating any object, cannot put a large source into the small slot, and writes distinct full/thumbnail digests. Regression tests also prove that omitting a thumbnail for a large source leaves DataInfo 25 and its package member intact.

output/verify_f.pages replaces the fixture image with a visible cyan/magenta checkerboard with a corresponding 256px checkerboard thumbnail, both retaining the Pages-authored RGB/non-interlaced PNG shape and 72-dpi pHYs chunk. Its decoded/member diff is limited to the two expected Data/*.png members and PackageMetadata object 2/type 11006 in Index/Metadata.iwa; ImageArchive is byte-identical because the natural dimensions did not change. It retains 37 IWA files and 639 segments, and self-reparse, zipfile.testzip(), and unzip -t pass. A separate mutation test uses a 512×256 PNG plus 256×128 thumbnail and proves that only ImageArchive 1734421/type 3005 is additionally changed for the updated natural size.

Final Stage H implements add_picture() with independent DataInfo identifiers, ImageArchive/geometry identities, UUID registration, attachment insertion, and UTF-16 table shifting. Existing-image replacement and new-image insertion now share the same validated full/thumbnail metadata rules.

Stage G — hyperlinks and named styles

The fixture hyperlink is encoded in body StorageArchive field 11, an ObjectAttributeTable used for smart fields. Its runs are (0, nil), (53, 1734811), (71, nil), precisely covering the fixture hyperlink in UTF-16 coordinates. Object 1734811/type 2032 is TSWP.HyperlinkFieldArchive: field 1 is its SmartField UUID and field 2 is the UTF-8 URL.

hyperlinks.py resolves these ranges, while Document.hyperlinks and Paragraph.hyperlinks expose label text, url/address, and contained runs. Both label and address are writable. Label replacement uses the central text offset engine, so the end boundary in field 11 and every other downstream body table shift together. URL replacement changes only the type-2032 object.

The stylesheet facade now indexes all registered named paragraph and character styles. Document.styles supports iteration and name lookup; Style exposes name, stable Pages identifier as style_id, and WD_STYLE_TYPE. Paragraph.style and Run.style read their effective named ancestor and apply an existing named style with type validation. Paragraph application preserves explicit field-5 boundaries and retains the duplicate-offset regression guard.

output/verify_g.pages applies Heading 2 to a fixture paragraph and changes the hyperlink to a neutral visible label with target https://example.com. Its decoded diff changes only Index/Document.iwa, body Storage 1732539/type 2001 and HyperlinkField 1734811/type 2032. It retains 37 IWA files and 639 segments, and self-reparse, zipfile.testzip(), and unzip -t pass.

Stage H — new component graphs and comment investigation

Shared graph-cloning layer

components.py performs schema-light cloning without aliasing the source objects. It follows MessageInfo object references, assigns a new ID to each owned object, recursively remaps embedded TSP.Reference messages, and remaps the packed MessageInfo object/data reference lists. Standalone Tables/* components receive a new component ID, locator, and IWA member; objects that live in Document or the existing CalculationEngine are appended there. PackageMetadata external-reference and object-UUID-map membership is mirrored exactly from the Pages-authored source graph. Shared stylesheet objects remain shared.

Cloned table identities also receive deterministic fresh TSP.UUID/CFUUID values and a new uppercase table UUID string. The body insertion is a new paragraph separator plus U+FFFC, with a field-9 attachment boundary at that character. The central text replacement engine shifts every pre-existing offset table before the new boundary is added.

add_table(rows, cols)

The source attachment 1734178 reaches a 50-object owned closure: 19 objects in CalculationEngine, 30 standalone Tile/DataList/HeaderStorageBucket components, and the attachment itself. All 50 are cloned. The cloned Tile records are initialized as blank version-5 wide-offset rows, row/column header buckets are resized, TableModel fields 6/7/8 become the requested dimensions and a unique table name, and the complete secondary calculation/UID graph is retained. The current one-tile implementation accepts 1–256 rows. Named table style selection is excluded because the public style facade has no table-style objects; the Pages-authored template style is retained.

output/verify_h_add_table.pages appends a labelled blank 3x4 table. Its audit is intentionally expansive but closed: 30 new Index/Tables/*.iwa members, 19 added CalculationEngine segments, one added type-2003 attachment, the body Storage mutation, and PackageMetadata registration. No pre-existing table, style, or media object changes. The result has 67 IWA files and 689 segments; self-reparse, zipfile.testzip(), and unzip -t pass.

The first manual render exposed a second dimension source: although TableModel fields 6/7 and the tile row buffers said 3x4, Pages rendered the cloned source's 2x2 grid. The stale values were the type-6267 ColumnRowUIDMapArchive, type-6305 StrokeSidecarArchive column/row counts, and header field-4 cell counts. The corrected writer rebuilds a sorted set of four column UIDs and three row UIDs with both inverse index arrays, sets the sidecar to columns=4/rows=3, and writes three row headers with four cells plus four column headers with three cells. Tile field 4 is 3, its three RowInfo records each have four wide offsets, and all seven dimension representations now agree. A dedicated regression test asserts each one. The corrected verify package retains the same closed 67-file/689-segment graph shape.

add_picture(path_or_stream, width=None, height=None)

The image closure clones the type-2003 attachment, type-3005 ImageArchive, and two type-3097 geometry helpers while sharing the existing type-3016 image style. It allocates DataInfo 26/27, unique stored filenames, two ComponentData references to the new ImageArchive, and independent SHA-1/materialized-length records. A dependency-free PNG decoder supports non-interlaced 8-bit RGB/RGBA sources and generates a nearest-neighbor thumbnail with a 256-pixel maximum; recognized color/DPI chunks are retained. Logical dimensions preserve aspect ratio and honor optional width/height.

output/verify_h_add_picture.pages appends a labelled 512x320 cyan/magenta checkerboard and an automatically generated 256x160 thumbnail. Its decoded diff is limited to added Data members, four added Document segments, the body Storage mutation, and PackageMetadata. It retains 37 IWA files and has 643 segments; full/thumbnail digests differ, and self-reparse, zipfile.testzip(), and unzip -t pass.

Comments: real graph and implemented read/edit path

The updated fixture_features.pages contains one root comment and one reply. Its real graph agrees with the static Pages strings and numbers-parser schemas:

Archive/type Fields established from real data
TSWP.StorageArchive/2001 field 25 overlapping range table; location=0, length=26, highlighting object 1734975
TSWP.HighlightArchive/2013 field 1 CommentStorage reference; field 2 text-attribute UUID string
TSD.CommentStorageArchive/3056 field 1 text; field 2 creation date; field 3 author; field 4 replies; field 5 storage UUID
TSK.AnnotationAuthorArchive/212 field 1 name; field 2 color; fields 3–5 public identities
TSK.AnnotationAuthorStorageArchive/213 repeated field 1 author references

The range selects exactly the first 26 UTF-16 units of neutral fixture text. Highlight 1734975 references root storage 1734976; its repeated field 4 references reply 1734977. Both point to the fixture's private author record. TSP.Date field 1 is an IEEE-754 double in seconds from the 2001-01-01 Apple epoch; both root and reply timestamps decode correctly.

Object 1731480/type 10001 is not an annotation graph. Its field 1 is a TSS.ThemeArchive; extension field 110 is TSWP.ThemePresetsArchive, whose repeated field 6 is character_style_presets. References 1731545/1731546 are the named theme presets Red Bold and White. Neither is referenced by the body Storage or defines a text range. This is established by the proto extension declaration, the archive's Blank theme identifier/stylesheet reference, the preset names, and a regression test. Treating path 1→110→6 as review annotation state was an incorrect interpretation.

Document.comments implements collection length, iteration, indexing, get, root text, author, timestamp, comment ID, reply traversal, UTF-16 anchor positions, and anchor text. Existing root and reply text have safe setters. Document.annotation_highlights exposes the actual review ranges with their selected text, linked Comment, and author color; fixture author QA decodes to FAEF5A. Review highlights are UI annotations and are not expected in PDF output. Comments.add_comment() remains explicitly excluded: current Pages adds calculation/view-state/activity objects (types 4008, 10131–10133, and related metadata) alongside the five essential annotation objects. Cloning only the visible graph would risk collaboration-state corruption; one sample is not enough to distinguish required state from transient UI state.

Stage I — subscript, superscript, and highlight

Schema analysis

The numbers-parser proto and static strings in the Pages 15.1.1 TSText framework agree on the following CharacterStylePropertiesArchive fields:

Field Wire form Meaning
10 varint enum script: baseline=0, superscript=1, subscript=2
14 fixed32 float explicit baseline shift in points
25 varint bool background-color-null marker
26 message background TSP.Color
43 / 44 bool / message modern tsd_stroke_null / TSD.StrokeArchive
45 / 46 bool / message modern tsd_fill_null / TSD.FillArchive
47 varint bool apply the TSD fill to the text container rather than glyphs

No supplied fixture contains a non-default script. The latest feature fixture does contain a real format-menu character background. Pages created anonymous type-2022 style 1735176; its field-11 character properties contain exactly one field: field 26 with a green TSP.Color (88FA4E after 8-bit conversion). There is no field-25 null marker and no field 7, 46, or 47. Body paragraph-style table field 5 applies this style to the background paragraph range. It is not referenced from character-style table field 8. The outer style fields are exactly (1, 10, 11, 12): super points to named Body, override count is 1, field 11 contains field 26, and field 12 is an explicitly present empty ParagraphStylePropertiesArchive.

Although type 2021 and type 2022 reuse the same character-properties message, their placement is semantic. The user-rendered bisect proves that putting field 26 in an anonymous type-2021 style referenced from field 8 is invalid: Pages fails character-style resolution for the affected page rather than treating it as a run background.

By contrast, theme styles 1731545/1731546 with (7,46) and (1,7,46) are foreground-color presets, not review highlights or character backgrounds. Field 7 is font color and field 46 mirrors it as a TSD fill. Field 47 occurs only in full paragraph-style defaults in these fixtures.

Implemented behavior

CharacterProperties includes vertical_align, baseline_shift, and background_color. They participate in signatures, style caching, dynamic override counts, synthesis, and decoding. Font.subscript and Font.superscript implement python-docx's shared tri-state vertical-alignment behavior: setting either True replaces the other, False removes only the matching state, and None removes the direct script override.

WD_COLOR_INDEX and its WD_COLOR alias expose the complete python-docx highlight palette. The native API is Paragraph.background_color (arbitrary RGB) and Paragraph.highlight_color (indexed palette). It synthesizes an anonymous type-2022 style, writes field 26 only inside field 11, retains the required empty field 12, and references the style through Storage field 5. None and INHERITED remove it. Font.highlight_color remains available as a python-docx-shaped facade, but Pages has no observed run-level equivalent: its getter and setter operate on the run's entire containing paragraph. Foreground font.color.rgb remains the separate field-7 plus field-46 representation. Pages has no distinct automatic indexed-color token, so AUTO is written as black and reparses as BLACK. Arbitrary Pages colors that are not an exact palette member return None from the indexed-color facade.

Low-level type-2021 synthesis rejects CharacterProperties.background_color by default. The only escape hatch is the explicitly named experimental_allow_paragraph_background=True, retained so the known-bad bisect B can be reproduced; it is not used by the public formatting API.

The prior claims that field 47 or the (7,46) theme fill caused the page-7 shift were disproved and are withdrawn. (7,46) is foreground/theme color. The actual failure is the style class and reference table: field-26 background was synthesized as type 2021 and installed in field 8, while Pages-authored data uses type 2022 and field 5.

The requested controlled bisect now uses identical appended text and operation order. A adds only the plain line and changes body Storage 1732514. C applies a known-safe bold preset; it also changes only Storage 1732514 because style 1731516 is reused. B applies the now-known-invalid direct character background and differs from A only by the four appended field-8 run boundaries, two new type-2021 styles, stylesheet 401 registration, and PackageMetadata. Its styles contain field 26 only. The user's PDF result was decisive: A preserved all page-7 bold and underline; B removed all of it and rendered no background. This reconciles the binary invariant (all earlier objects and 16 existing field-8 runs unchanged) with the visual failure: a newly registered style graph can invalidate Pages' resolver without mutating the earlier style data.

The final verification file is no longer B. It appends two paragraphs and applies yellow/green backgrounds using two anonymous type-2022 styles through field 5. The character-style table is byte-for-byte unchanged from the source.

The verification generator also exposed an insertion-boundary regression: formatting a newly added run and then adding another run extended the first style across the second text. Paragraph.add_run() now explicitly restores inherited character formatting over an unstyled inserted run, with a regression test.

Verification

Six upstream text/test_font.py behavior groups for script and highlight are now explicitly marked migrated in test_python_docx_ledger.py. API, wire, save/reparse, mutual-exclusion, palette, appended-run isolation, preservation of all pre-existing run boundaries, exact Pages-authored style bytes, and the comment/annotation and real-background tests bring the dependency-free suite to 116 passing tests after the package-rename compatibility test.

output/verify_i_subsuper.pages adds an H₂O/x² line. It contains exactly two new type-2021 styles, each with override count 1 and only field 10 (values 2 and 1 respectively). output/verify_i_highlight.pages adds separate yellow- and bright-green-background paragraphs. Its two new styles are type 2022 with outer fields (1,10,11,12), override count 1, field-11 content (26), and an empty field 12, exactly matching fixture style 1735176 apart from identifiers, parent, and color. They appear at new field-5 positions 5902 and 5929. The complete character-style table remains byte-for-byte identical to the source, including all page-7 bold/bold-off ranges.

For each file the decoded diff changes only Index/Document.iwa (body Storage 1732514), Index/DocumentStylesheet.iwa (stylesheet 1732588 plus styles 44990410/44990411), and Index/Metadata.iwa (PackageMetadata 2). The final highlight stylesheet grows from 340,187 to 340,377 decoded bytes and the package has 638 segments. Self-reparse, ZipFile.testzip(), and unzip -t all pass. Output packages are ignored by Git as required.

The bisect outputs are:

  • verify_i_highlight_bisect_a.pages: identical line, no style; 636 segments, source diff limited to Storage 1732514.
  • verify_i_highlight_bisect_b.pages: known-invalid type-2021/field-8 field-26 negative control; 638 segments. The user confirmed it destroys page-7 text decoration and renders no background.
  • verify_i_highlight_bisect_c.pages: identical line with bold style 1731516; 636 segments and only Storage 1732514 changed.
  • verify_i_highlight.pages: corrected type-2022/field-5 implementation; not byte-identical to B.

output/control_fixture.pages is SHA-256-identical to the updated fixture (a7a9ab44…9000f), reparses as 37 IWA files/644 segments, and passes both ZIP checks. If its genuine Pages-authored background is absent from PDF, PDF export is not a valid background-color oracle; use Pages screen rendering for the corrected final verify. This control does not affect the already conclusive A/B decoration regression result.

Stage J — comments

src/pages/comments.py implements the real field-25/type-2013/type-3056 graph described above, including reply recursion and Apple-epoch dates. Public Comment/Comments proxies are exported from pages, and Document.comments exposes them. Eight applicable upstream tests/test_comments.py behaviors moved from fully excluded to partially migrated; Pages-specific reply/anchor behavior and the safe text setter have additional local tests.

output/verify_j_comments.pages changes root text to Stage J: root comment edited and its reply to Stage J: reply edited. Its decoded diff contains only Index/Document.iwa objects 1734976 and 1734977; no reference, range, author, UUID, metadata, or collaboration object changes. The package remains 37 IWA files/643 segments, and self-reparse, zipfile.testzip(), and unzip -t pass.

Body text and style location

The main body is object 1732514, archive type 2001 (TSWP.StorageArchive) in Index/Document.iwa.

Relevant StorageArchive fields:

Field Meaning
1 storage kind; 0 is BODY
2 stylesheet reference
3 repeated UTF-8 text chunks
5 paragraph-style attribute table
8 character-style attribute table

table_char_style is an ObjectAttributeTable. Each repeated field-1 entry has character_index in field 1 and an optional TSP.Reference in field 2. The indices are UTF-16 code-unit offsets, matching Cocoa NSRange. A missing object reference is a run boundary that reverts to paragraph styling.

The referenced style objects have archive type 2021 (TSWP.CharacterStyleArchive). Their field 11 contains CharacterStylePropertiesArchive, whose key fields are:

Field Meaning
1 bold boolean
2 italic boolean
3 font size, fixed32 float
4 font-name-null marker
5 PostScript font-name string
6 font-color-null marker
7 TSP.Color font color
10 baseline/superscript/subscript enum
11 underline enum
12 strikethrough enum
14 baseline shift, fixed32 float
25 background-color-null marker
26 TSP.Color background/highlight color
43 / 44 modern TSD stroke null/value
45 / 46 modern TSD fill null/value
47 fill text container rather than glyphs

The color field was confirmed from both the extracted schema and real sample objects 1731520/1731521/40613724/40613726. RGB colors use TSP.Color.model=1, fixed32 floats in fields 3/4/5, alpha in field 6, and RGB color space in field 12. Pages 15 also writes fixed32 field 13 as 1.0; the writer follows that observed shape.

Paragraph styles are archive type 2022. They use the same field 11 character properties and additionally use field 12 for paragraph properties.

Concrete sample correlations:

  • Fixture A character style 42372726 has only bold=1; it is active over UTF-16 [5183,5202) and [5416,5434).
  • Fixture A character style 40613725 has only bold=0; it is active over [5107,5108) and [5260,5314).
  • Fixture A style 40613723 and fixture B style 66597713 have bold=1 plus underline=1 at their referenced runs.
  • The reusable named character style 1731516 (Emphasis) contains only bold=1. The editor deliberately uses this style because it changes only weight and leaves font, size, and color inherited.

Font names are direct strings, not indexes into a separate font table in these messages. Examples found in each sample include paragraph style 1731492 with bold=1 and the document-specific bold face; fixture A style 1731479 has bold=1 with the corresponding regular face, and anonymous paragraph styles 39858906/43092615 carry that regular face without an explicit bold field. The samples demonstrate that the bold flag and selected PostScript face are independent properties and can sometimes look contradictory. Run-level bold-only character styles omit field 5 and rely on Pages' inherited-font/trait resolution.

Writer behavior

PagesDocument.make_bold(start, length) accepts Python code-point coordinates and converts them to UTF-16. make_substring_bold(text, occurrence=1) performs the search and then calls the range API. The fixture-specific compatibility default selects its explicit bold face; passing font_name=None retains the trait-only mode.

For the selected interval the generalized editor:

  1. determines the style active at the end boundary;
  2. removes run boundaries inside the interval;
  3. selects an anonymous type-2021 style that has a parent reference as the structural template;
  4. allocates max(PackageMetadata.last_object_identifier, all object IDs)+1;
  5. clones the template ArchiveInfo, version, anonymous TSS.StyleArchive, and base-style dependency;
  6. encodes the requested CharacterProperties; override_count is the number of non-None properties rather than a constant;
  7. appends the new segment to Index/DocumentStylesheet.iwa;
  8. adds the new style reference to stylesheet field 1, adds it below the template's parent in stylesheet field 5, and adds it to the stylesheet MessageInfo field-5 reference list;
  9. increments PackageMetadata field 1, adds an object-ID/UUID entry to the DocumentStylesheet component's field-11 object UUID map, and adds a strong (DocumentStylesheet component, new style object) reference to the Document component's field-6 external-reference map;
  10. inserts the new style boundary at the start and a restore boundary at the end, and adds the object ID to the storage MessageInfo reference list;
  11. updates payload and ArchiveInfo lengths and writes the archive in 64-KiB IWA frames.

The object UUID is deterministic UUIDv5 over document revision, object ID, and the complete property signature. The current fixture A output uses 2f1dd3a3-f46f-5e7b-8a45-5a2592893b74.

No original sample is modified in place. Generated examples are under output/.

Actual validation results

Commands used:

PYTHONPATH=src python3 -m unittest discover -s tests -v
PYTHONPATH=src python3 tools/diff_iwa.py samples/private_fixture_a.pages output/private_fixture_a_bold.pages
PYTHONPATH=src python3 tools/diff_iwa.py samples/private_fixture_b.pages output/private_fixture_b_bold.pages
unzip -t output/private_fixture_a_bold.pages
unzip -t output/private_fixture_b_bold.pages

All 96 automated tests pass. They cover the two end-to-end edits, no-op byte preservation, all three stylesheet links, PackageMetadata registration, property-sensitive style caching, dynamic override_count, real sample color decoding, every requested API property, save/reload, paragraph/run derivation, multi-MessageInfo object retention, Stage-A paragraph formatting, Stage-B section/header-footer behavior, Stage-C text insertion and deletion, existing and newly cloned tables/images, hyperlinks, styles, and the upstream ledger.

End-to-end edits:

Output Selected text New UTF-16 run Result
output/private_fixture_a_bold.pages a 13-code-point body substring [24,37) -> new style 44990410 PASS
output/private_fixture_b_bold.pages a 9-code-point body substring [100,109) -> new style 66610570 PASS

Direct inspection of the final fixture A output confirmed:

  • Python ZipFile.testzip() returned no bad member;
  • unzip -t returned “No errors detected”;
  • all seven IWA members re-parsed with this implementation;
  • the segment total changed from 636 to 637, exactly accounting for the new type-2021 object;
  • body text was byte/Unicode-identical after the style-only edit;
  • the target run at UTF-16 24 references object 44990410 and the restore run at 37 has no object reference;
  • object 44990410 is type 2021 in Index/DocumentStylesheet.iwa, has override_count=2, and its character properties are exactly field 1 bold=1 and field 5 set to the fixture's explicit bold face;
  • its ArchiveInfo retains the template base-style reference 1731514;
  • stylesheet field 1 contains 44990410 exactly once;
  • the field-5 parent entry for 1731514 contains child 44990410 exactly once;
  • stylesheet MessageInfo field 5 contains 44990410 exactly once;
  • storage object 1732514 references new style 44990410 exactly once;
  • PackageMetadata last_object_identifier, the maximum parsed object ID, and the new ID all equal 44990410;
  • the DocumentStylesheet UUID map grew from 507 to 508 entries and contains the new object;
  • the Document component external-reference map contains exactly one strong (1732588, 44990410) entry.

Only three outer ZIP members changed, all expected:

Member Decoded before Decoded after Delta Purpose
Index/Document.iwa 39,133 39,152 +19 run entries and object reference
Index/DocumentStylesheet.iwa 340,187 340,270 +83 type-401 three-point registration and one new type-2021 segment
Index/Metadata.iwa 22,381 22,422 +41 last ID, UUID map, external reference

The fixture B audit has the same semantic shape: only storage object 1732514, stylesheet object 1732588 plus new style 66610570, and PackageMetadata object 2 changed. Its decoded deltas are 50,485→50,505, 339,193→339,276, and 21,709→21,751 respectively. SequenceMatcher byte opcodes are included by the reproducible tools/diff_iwa.py report. No unrelated object changed.

The new fixture A object differs from structural template 40613723 only where required: ArchiveInfo identifier, payload lengths, exact character properties, and override_count. The anonymous super style, its parent reference 1731514, version, and header base-style references are retained. Untouched IWA members retain their exact original compressed bytes.

numbers-parser comparison

The requested paths were reviewed directly:

  • containers.py:new_message_id() increments its maximum and writes PackageMetadata.last_object_identifier; allocate_object_identifier() does the same and also guards against a stale metadata maximum by comparing all parsed object IDs.
  • model.py:add_component_reference() writes the same strong component/object pair used here for Document -> (DocumentStylesheet, style ID).
  • The object UUID map is updated here because the sample metadata and the successful diagnostic require a UUID-addressable object in the existing DocumentStylesheet component.
  • add_component_metadata() creates an entirely new component/IWA locator. A character style is a segment in the existing DocumentStylesheet component, so adding another component record would be incorrect.
  • iwafile.py refreshes MessageInfo payload lengths and writes at most 64-KiB uncompressed chunks. The local writer already refreshed lengths; it now also uses the same 64-KiB chunk boundary after a changed file can no longer reuse the original frame sizes.

No additional component metadata or weak reference is warranted for this write path.

python-docx test migration

All seven files under ref/python-docx/tests/text/ were scanned: 94 behavior methods total. The original character/run behavior categories are adapted in tests/test_python_docx_compat.py: run text, font access, tri-state bold/italic, underline, font name, size, color, strike, paragraph text, and paragraph runs. Stage A adds thirteen paragraph-format/style-synthesis cases in tests/test_paragraph_format.py, covering the applicable behavioral surface of test_parfmt.py and test_tabstops.py. Parameter variants are also covered in the broader API tests.

Stage B also scanned ref/python-docx/tests/test_section.py. Ten applicable Pages-backed behavior groups are adapted in tests/test_sections.py. Its WordprocessingML element/relationship assertions, multiple-section linkage, and header/footer-part lifecycle tests are excluded because Pages uses shared TP section templates and TSWP fragment storages rather than DOCX parts.

The final scan is enforced by tests/test_python_docx_ledger.py: all 59 test_*.py files and all 662 upstream it_*/test_* behavior methods must be accounted for exactly once. PARTIAL means the public behavior relevant to a Pages counterpart has tests in this project while DOCX XML/relationship variants or unsupported properties in the same file are excluded. EXCLUDED means the whole file tests a DOCX container/internal layer or an API with no implemented Pages counterpart.

Disposition Upstream files Reason / local coverage
PARTIAL (20) dml/test_color.py; styles/test_latent.py; styles/test_style.py; styles/test_styles.py; test_api.py; test_blkcntnr.py; test_comments.py; test_document.py; test_enum.py; test_section.py; test_shape.py; test_shared.py; test_table.py; text/test_font.py; text/test_hyperlink.py; text/test_pagebreak.py; text/test_paragraph.py; text/test_parfmt.py; text/test_run.py; text/test_tabstops.py RGB/length/enums, document/paragraph/run/font, comments, paragraph format, single section/header/footer, existing and newly inserted tables/images, hyperlinks, and named styles are adapted in API-level tests. Unsupported DOCX-only properties and XML assertions inside these files are excluded.
EXCLUDED — image codecs (7) image/test_bmp.py; image/test_gif.py; image/test_helpers.py; image/test_image.py; image/test_jpeg.py; image/test_png.py; image/test_tiff.py These test python-docx's independent image header/DPI codec. python-pages preserves and replaces package PNG bytes; it is not a general image codec.
EXCLUDED — OPC (10) opc/parts/test_coreprops.py; opc/test_coreprops.py; opc/test_oxml.py; opc/test_package.py; opc/test_packuri.py; opc/test_part.py; opc/test_phys_pkg.py; opc/test_pkgreader.py; opc/test_pkgwriter.py; opc/test_rel.py DOCX OPC parts, relationships, content types, PackURI, and physical-package rules have no IWA counterpart. Core-properties public behavior is separately adapted.
EXCLUDED — OXML (11) oxml/parts/test_document.py; oxml/test__init__.py; oxml/test_comments.py; oxml/test_document.py; oxml/test_ns.py; oxml/test_section.py; oxml/test_styles.py; oxml/test_table.py; oxml/test_xmlchemy.py; oxml/text/test_hyperlink.py; oxml/text/test_run.py WordprocessingML element classes, namespaces, XPath, and XMLchemy mutation are format-specific. Equivalent public behavior is tested against IWA wire objects.
EXCLUDED — DOCX parts (8) parts/test_comments.py; parts/test_document.py; parts/test_hdrftr.py; parts/test_image.py; parts/test_numbering.py; parts/test_settings.py; parts/test_story.py; parts/test_styles.py Relationship/part lifecycle and Word numbering/settings internals do not map to Pages components. Public document, header/footer, image, and style behavior is covered elsewhere.
EXCLUDED — unavailable public areas (3) test_drawing.py; test_package.py; test_settings.py Settings have no implemented Pages facade. Drawing/package tests are DOCX inline-XML and package-part behavior, while public PNG insertion is covered separately.

Method-level exclusions within PARTIAL files cover DOCX XML/OPC mutation, multiple sections, table merge/row-column mutation, non-PNG image codecs, remaining paragraph pagination/tab mutation, and font properties without an implemented Pages counterpart. The six subscript, superscript, and highlight behavior groups are now explicitly migrated rather than excluded. Highlight uses the documented paragraph-granularity mapping because the only Pages-authored counterpart is a paragraph style, not a character run style.

Final aggregate: 20/59 upstream files are partially migrated at their public Pages-backed behavior layer and 39/59 are excluded with the category reasons above; the ledger accounts for all 662 upstream behavior methods. The current standalone suite discovers 81 tests under unittest discover; nine optional fixture or ledger tests skip when their local inputs are absent. Pytest is not required.

Limitations and feasibility assessment

The requested explicit-face foundation and object model are implemented for the supplied samples. The editor performs full stylesheet and component-map bookkeeping, not merely a run-table patch.

Known limitations:

  • The library did not launch Pages. Rendering checks confirmed add_picture, subscript, and superscript. Screen-level rendering was not recorded for the corrected add_table grid dimensions, type-2022/field-5 paragraph background, or comment text editing.
  • New comment creation is excluded because Pages writes collaboration, calculation-engine, and view-state objects beyond the visible annotation graph. Existing root/reply reading and text editing are implemented.
  • API setters merge all ten known character properties on the selected run. A source style containing additional, unsupported character properties can lose those unknown direct overrides when replaced.
  • The compatibility helper's default explicit font is fixture-specific. Callers editing other fonts must supply the appropriate PostScript name.
  • The generic wire layer preserves unknown fields but does not implement the semantic merge algorithm for ArchiveInfo.should_merge patch payloads. The edited body storage here is a normal full payload.
  • Encrypted documents, malformed ZIP filename encodings on older Python, and every historical IWA/package variant are outside this project's coverage.
  • ZIP container bytes, compression ratios, and timestamps are not preserved; uncompressed no-op members are preserved exactly.
  • Named paragraph-style values are exposed as inherited (None) rather than flattened computed values, matching python-docx direct-format semantics.
  • Pages page setup is package-wide, so Document.sections intentionally has one entry. Multiple DOCX sections and previous-header/footer linking are not emulated.
  • Pages headers and footers are exposed as their three left/center/right fragments. Their specialized setter is not yet routed through the complete body offset engine for styled or attachment-bearing fragment text.
  • add_table() and add_picture() require the target document to contain a Pages-authored table or image graph respectively, used as the structural template. A new table uses one 256-row tile and retains the template table style; named table styles, more than 256 rows, merge operations, and adding content to a sparse omitted cell are excluded.
  • Automatic thumbnails accept non-interlaced 8-bit RGB/RGBA PNG. Other image codecs and PNG color models require a caller-provided preprocessing step.
  • Pages exposes the observed field-26 background at paragraph granularity. Run.font.highlight_color therefore affects its containing paragraph; a run-granular mapping is excluded because the type-2021/field-8 experiment provably breaks character-style rendering.
  • Screen-level rendering was not recorded for the corrected paragraph- background verification file. PDF evidence is usable only if the byte-identical control fixture's genuine background also appears in PDF export.

Within these explicit format-specific exclusions, the requested python-docx public surface is implemented or fully accounted for by the final ledger.