ANN tests: make the min_recall check strict #1156
Conversation
…rence against expected recall is not big
achirkin
added
improvement
|
NB: some of the affected algorithms are not deterministic due to concurrency etc.; I've successfully run the test suite on my workstation 35 times, but we should keep an eye on how it behaves on the CI machines as well. |
Codecov ReportBase: 87.99% // Head: 87.99% // No change to project coverage 👍
Additional details and impacted files@@ Coverage Diff @@
## branch-23.02 #1156 +/- ##
=============================================
Coverage 87.99% 87.99%
=============================================
Files 21 21
Lines 483 483
=============================================
Hits 425 425
Misses 58 58 Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
|
My recent merge commit broke the build. Just leaving a note here that I'll fix this. |
|
Did a dozen tests on a100 and v100, seems to be passing with the last adjustments. |
|
/merge |
In #1135, we adjusted the min_recall values to report if any regressions happen in ivf-pq. However,
eval_neighboursfunction, which is used in several ANN test suites, doesn't fail unless the regression is really large (it prints a warning if the calculated recall is "slightly" smaller than the expected recall).In this PR, I make
eval_neighboursalways fail if the calculated recall is smaller than the expected recall. Slightly adjust the tests and do a small refactoring along the way.