-
Notifications
You must be signed in to change notification settings - Fork 339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Catch warnings in test_proton_radiography.py #1050
Catch warnings in test_proton_radiography.py #1050
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #1050 +/- ##
=======================================
Coverage 96.86% 96.86%
=======================================
Files 70 70
Lines 6864 6864
=======================================
Hits 6649 6649
Misses 215 215 ☔ View full report in Codecov by Sentry. |
|
||
sim = prad.SyntheticProtonRadiograph(grid, source, detector, verbose=False) | ||
# Catch warnings because these fields aren't well-behaved at the edges | ||
with warnings.catch_warnings(): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it not, instead, be better to use pytest.warns to also check for the eventuality that we broke the functionality and warnings are not raised anymore? As I understand it this is deterministic enough that they'll always get raised.
I probably forgot to mention pytest.warns in the issue and definitely should have, sorry!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, yes, it is pretty much deterministic and should get always get raised. My thinking was that was better to rely on the other places where these warnings are explicitly tested (in situations designed to create them). The warnings fixed in this PR all just occur in setup lines preparing for other tests. So, for the purpose of being organized, I thought it was better to just blanket filter the warnings in that case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, but suppose we had this situation:
- our code throws warning A;
- we don't want to see warning A in tests, so we use
with warnings.catch_warnings()
; - something changes elsewhere (upstream, decorators, etc...) that causes our code, tests or no, to throw hundreds of occurrences of warning B;
- months pass;
- we learn our code has been horribly broken for a few months;
- everyone's sad
And while I can't exactly point you to the exact PR after these couple years, we did have that situation :D It was something with units and our unit checks IIRC.
What would be best, while we don't do it nearly enough, is to catch these warnings like https://docs.pytest.org/en/stable/warnings.html#warns does, with the following:
with pytest.warns(GeometryWarning, match="is not a stellarator, rethink your reactor design or reboot your fusion program")
test_DEMO()
Then if test_DEMO
a) ceases to throw GeometryWarning
b) ceases to throw that particular GeometryWarning
c) starts throwing WallMeltingWarning
or DisruptionWarning
due to some other changes elsewhere
we get to know about in basically in real time!
... I should probably stick that into our testing guide, shouldn't I?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok that makes sense, I'll change over to using pytest.warns then.
I like the idea of checking explicitly to make sure the warning is correct: is there a way to do that while accounting for fstrings in the warning? Many of mine are quasi-deterministic so the warning will always appear but the numbers in the string might not always be identical.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
match
is a regex! if you set it to the constant part, it'll work as you'd like it to :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
AMAZING
regex is just magic
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good changes and the warnings are indeed gone - happy to merge! :)
Addresses issue #844 for
proton_radiography.py