Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing Googletests and re-enabling in CI #1904

Merged
merged 20 commits into from
Oct 19, 2023

Conversation

cjnolet
Copy link
Member

@cjnolet cjnolet commented Oct 16, 2023

Recently, the CI script to run the C++ gtests was changed to call ctest but it looks like this command has been failing in CI for quite some time. It's a little scary as most of RAFT's C++ APIs are not exposed through Python and so we cannot rely on those Python tests to guarantee quality of the C++ APIs.

I'm reverting the change back to invoking the C++ binaries directly so that we can get the tests running again. Once we figure out why ctest isn't running the tests then we can make the change to use that again.

@cjnolet cjnolet added bug Something isn't working non-breaking Non-breaking change labels Oct 16, 2023
@cjnolet cjnolet self-assigned this Oct 16, 2023
@cjnolet cjnolet requested a review from a team as a code owner October 16, 2023 14:22
@github-actions github-actions bot added the ci label Oct 16, 2023
@cjnolet cjnolet requested a review from a team as a code owner October 16, 2023 18:46
@github-actions github-actions bot added the cpp label Oct 16, 2023
Copy link
Contributor

@robertmaynard robertmaynard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add cd "$CONDA_PREFIX"/bin/gtests/libraft before the ctest line to have ctest execute the tests instead of reverting the change

@cjnolet cjnolet requested a review from a team as a code owner October 17, 2023 03:27
@cjnolet
Copy link
Member Author

cjnolet commented Oct 18, 2023

It turns out at least the expanded Euclidean issue is related to the precision of clamping from round-off errors between the subtraction of the norms and the dot product (there can be an error of 1e-5, for example, which would end up becoming 0.003 once the sqrt is taken. Thus tiny errors get magnified from the sqrt.

Prior to this change, we were just taking any small value < 1e-4 and clamping it to 0. The problem is that we were limiting the effective range of inputs to the algorithms. This change adds an explicit check that the norms are equal as well. Be cautioned that in order to determine the vectors are equal, we need to test that the dot product = normX = normY. The challenge is that this roundoff error makes normX == normY but the dot product might only be approximately equal to them. Adding this extra check for the equality of the norms should at least narrow down the potential for the clamping to affect other numbers, however, it can still effect things like true orthognality, which could still have equal norms.

@cjnolet cjnolet changed the title Reverting ci logic to run gtests. Fixing Googletests and re-enabling in CI Oct 18, 2023
Copy link
Member

@benfred benfred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

python/pylibraft/pylibraft/test/test_distance.py Outdated Show resolved Hide resolved
python/pylibraft/pylibraft/test/test_distance.py Outdated Show resolved Hide resolved
@cjnolet
Copy link
Member Author

cjnolet commented Oct 19, 2023

/merge

@rapids-bot rapids-bot bot merged commit b5e14e1 into rapidsai:branch-23.12 Oct 19, 2023
57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ci cpp non-breaking Non-breaking change python
Projects
Development

Successfully merging this pull request may close these issues.

5 participants