Make test_generate_with_static_cache even less flaky
#34995
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
also fixed
|
| model.generate(**input_text_eng) | ||
| model.generate(**input_text_eng, generate_speech=False) | ||
|
|
||
| def test_speech_generation(self): |
There was a problem hiding this comment.
tests/models/seamless_m4t_v2test_modeling_seamless_m4t_v2.py::SeamlessM4Tv2GenerationTest::test_speech_generation
previously failed 50% of the time. Now running 300 times and all pass.
There was a problem hiding this comment.
Run 500 times
tests/models/musicgen_melody/test_modeling_musicgen_melody.py::MusicgenMelodyDecoderTest::test_eager_matches_sdpa_inference_1_bfloat16
2 failed (before) v.s. 0 failed with PR
tests/models/musicgen_melody/test_modeling_musicgen_melody.py::MusicgenMelodyTest::test_eager_matches_sdpa_inference_1_bfloat16
3 failed (before) v.s. 0 failed with PR
200 runs: 8 failed (before) v.s 0 failed (PR) |
ArthurZucker
left a comment
ArthurZucker
left a comment
There was a problem hiding this comment.
Cool, I like that we have some utils that can be re-used!
What does this PR do?
#34464 make some generate tests less flaky. However, with
it fails 0.6% of the time.
This PR takes the same idea in #34558 to make
test_generate_with_static_cacheeven much less flaky. Running 2000 times and all pass.It also make this process easier to applied to other tests in the future whenever necessary