Quantization Bootstrapping RFC

[metajack]

This document provides a proposal for quantization bootstrapping, allowing DiskANN to grow incrementally from scratch supporting quantization.

Rendered

[Copilot]

Pull request overview

This PR adds an RFC proposing “quantization bootstrapping” so DiskANN can start from an empty index, operate initially in full precision, then transition into hybrid/quantized operation once enough vectors exist to build quantization tables.

Changes:

• Introduces a 3-phase bootstrapping design (full precision → hybrid/backfill → fully quantized).
• Proposes strategy “wrappers” to orchestrate phase transitions.
• Sketches an API/trait concept (QuantBackfill) for controlling quantization backfill behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.16%. Comparing base (62e201a) to head (7fa2c88).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #848      +/-   ##
==========================================
- Coverage   90.16%   90.16%   -0.01%     
==========================================
  Files         509      509              
  Lines       97234    97238       +4     
==========================================
+ Hits        87672    87674       +2     
- Misses       9562     9564       +2     

Flag	Coverage Δ
miri	90.16% <ø> (-0.01%)	⬇️
unittests	89.83% <ø> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[metajack]

After discussion with Mark, I have reworked this a bit to be contained completely with the DataProvider implementation.

[hildebrandmw]

Thanks Jack.

Making this hidden at the algorithmic level is nice for a couple reasons. First, it cuts down on the sheer amount of code that gets monomorphized. When we start thinking about supporting the entire API surface in both modes: search/insert/filtered-search/inplace-delete, we're talking about a lot of code that is potentially getting compiled just to support bootstrap. Hiding this with some form of dynamic dispatch nicely sidesteps this issue, but introduces dynamic dispatch for vector retrieval, distance computations, and slightly degrades type-safety. Next - it means we don't have to do anything at the library level 😄.

Some of the dynamic dispatch overheads can be mitigated by moving the dispatch higher (e.g., at the ExpandBeam level rather than lower), and if #859 lands, then pretty much all vector element storage needs will be abstracted from the core algorithm.

That said - is there potentially another way to approach this by coupling quantization bootstrap with graph bootstrap? For relatively low vector counts, and exhaustive search is anyways going to be faster. While PQ is an outlier with its training data requirement, other quantizers have modest requirements. I wonder if we could have the best of both worlds with tiering instead: type safety in the core algorithm and no mandatory dynamic dispatch and no redundant monomorphizations. The flat initial portion would obviously need code generation, but I suspect algorithms on it would be far simpler than the graph and generate less bloat.

[metajack]

Ok, I've done a quick update to add the number and address the outstanding suggestions. I think this should be ready to go.

[metajack]

Another pass complete.

[harsha-simhadri]

Could you please update the "updated date"