Skip to content

Add a tool for computing groundtruth for a runbook#1263

Draft
magdalendobson wants to merge 3 commits into
mainfrom
users/magdalen/runbook_gt_tool
Draft

Add a tool for computing groundtruth for a runbook#1263
magdalendobson wants to merge 3 commits into
mainfrom
users/magdalen/runbook_gt_tool

Conversation

@magdalendobson

Copy link
Copy Markdown
Contributor

The groundtruth computation tool in the deprecated C++ repository seems to have some bugs, so I wrote a new one in Rust using our Runbook abstractions. To avoid making diskann-tools take a dependency on diskann-benchmark-core, I moved some of the shared runbook code to diskann-utils.

Once this PR lands I will update the streaming groundtruth tool in big-ann-benchmarks to use the Rust code instead.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 49.07652% with 193 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.91%. Comparing base (62e201a) to head (bea772b).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...ann-tools/src/bin/compute_streaming_groundtruth.rs 0.00% 152 Missing ⚠️
...n-utils/src/streaming/executors/bigann/validate.rs 80.35% 33 Missing ⚠️
diskann-utils/src/lib.rs 53.33% 7 Missing ⚠️
...nn-utils/src/streaming/executors/bigann/runbook.rs 80.00% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (49.07%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1263      +/-   ##
==========================================
- Coverage   90.16%   89.91%   -0.26%     
==========================================
  Files         509      509              
  Lines       97234    96788     -446     
==========================================
- Hits        87672    87027     -645     
- Misses       9562     9761     +199     
Flag Coverage Δ
miri 89.91% <49.07%> (-0.26%) ⬇️
unittests 89.58% <49.07%> (-0.26%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
diskann-tools/src/utils/ground_truth.rs 42.28% <100.00%> (ø)
diskann-utils/src/streaming/api.rs 100.00% <100.00%> (ø)
...nn-utils/src/streaming/executors/bigann/parsing.rs 83.68% <100.00%> (ø)
...nn-utils/src/streaming/executors/bigann/runbook.rs 79.13% <80.00%> (ø)
diskann-utils/src/lib.rs 74.07% <53.33%> (-25.93%) ⬇️
...n-utils/src/streaming/executors/bigann/validate.rs 80.35% <80.35%> (ø)
...ann-tools/src/bin/compute_streaming_groundtruth.rs 0.00% <0.00%> (ø)

... and 9 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants