Skip to content

perf(image): atomically adopt a verified OCI qcow2 into the cloudimg store #20

Description

@CMGS

Evidence

The OCI pull currently downloads into a caller-owned temporary file and hashes it once:

The pinned Cocoon import then hashes the same file, copies it into another temporary file while hashing again, and finally renames a final-form qcow2:

For an already-final qcow2, the path performs approximately 3N reads, 2N writes, and 3N SHA-256 work. A store-owned verified adopt path needs 1N write and 1N verification read. For a 35 GB image, that avoids approximately 70 GB of reads and 35 GB of writes. This byte accounting is static; it has not yet been benchmarked.

Proposed scope

Add a narrow store-owned import capability rather than a general trust caller verification bypass:

  1. The cloudimg store creates an unguessable temporary file inside its own TempDir, on the blob filesystem.
  2. The OCI downloader writes ranges directly to that file and retains the OCI descriptor's expected size and SHA-256.
  3. A store commit operation closes the writer, verifies size and digest exactly once, inspects the image, and atomically renames it when it is qcow2 compat 1.1 with no backing file.
  4. Non-final images continue through the existing safe conversion path.
  5. Index update, duplicate-digest handling, permissions, and cleanup remain store-controlled.

Do not accept arbitrary caller paths or a boolean already verified flag. Handle cross-device rename by falling back to the existing copy-and-rehash path.

Acceptance criteria

  • The final-form OCI path performs one payload write and one full verification read.
  • Digest mismatch, size mismatch, truncated ranges, backing-file qcow2, and unsupported format all fail closed.
  • Concurrent imports of the same digest produce one valid blob and consistent refs.
  • Process termination at each commit phase leaves no indexed partial blob and cleans stale temporaries safely.
  • EXDEV uses the safe fallback.
  • Benchmarks for 1/15/35 GB images report wall time, CPU, process read/write bytes, and peak disk usage for cold and warm page cache.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions