👷 [#771] Change Bencher threshold test to use IQR

[stevenbal]

Fixes #771

Changes

• Add script to analyze Bencher historical data
• Change Bencher threshold test to use IQR
• Document benchmarking for performance tests

Checklist

• Commit hygiene

  • Commit messages refer to the relevant Github issue
  • Commit messages explain the "why" of change, not the how

• Architectural design record

  • If a design decision was made that changes functionality, create an architectural design record for it here

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.77%. Comparing base (d65c0b7) to head (e55d44c).

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #783   +/-   ##
=======================================
  Coverage   86.77%   86.77%           
=======================================
  Files         145      145           
  Lines        3161     3161           
  Branches      253      253           
=======================================
  Hits         2743     2743           
  Misses        370      370           
  Partials       48       48           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Bencher Report

Branch	chore/fix-bencher-alarms
Testbed	ubuntu-24.04

Click to view all benchmark results

Benchmark	Latency	Benchmark Result milliseconds (ms) (Result Δ%)	Upper Boundary milliseconds (ms) (Limit %)
performance_test/tests/test_objects_list.py::test_objects_api_list_filter_by_object_type	📈 view plot 🚷 view threshold	96.64 ms (-22.86%) Baseline: 125.28 ms	137.26 ms (70.41%)
performance_test/tests/test_objects_list.py::test_objects_api_list_filter_one_result	📈 view plot 🚷 view threshold	15.51 ms (-22.33%) Baseline: 19.98 ms	24.13 ms (64.31%)
performance_test/tests/test_objects_list.py::test_objects_api_list_large_page_size_page_1	📈 view plot 🚷 view threshold	239.94 ms (-27.71%) Baseline: 331.91 ms	357.03 ms (67.20%)
performance_test/tests/test_objects_list.py::test_objects_api_list_large_page_size_page_5	📈 view plot 🚷 view threshold	241.07 ms (-28.31%) Baseline: 336.26 ms	355.85 ms (67.75%)
performance_test/tests/test_objects_list.py::test_objects_api_list_small_page_size_page_20	📈 view plot 🚷 view threshold	98.95 ms (-21.95%) Baseline: 126.77 ms	133.28 ms (74.24%)

🐰 View full continuous benchmarking report in Bencher

[CharString]

LGTM. Maybe that little iqr calculation needs find somewhere else to live... Preferably a view on the Bencher website.🙂 It was a quick one of, but it felt wrong to just keep it around on my machine.