Replies: 2 comments 2 replies
-
|
I was in there trying to add GrokAdamW and I saw patterns that would force me to import the pytorch version of the optimizer (thus limiting us only to optimizers that are already implemented in pytorch?) |
Beta Was this translation helpful? Give feedback.
-
|
It seems to me, we should not have any kind of dependency (even a test dependency) on pytorch |
Beta Was this translation helpful? Give feedback.
-
|
it does not need to be torch. fine as long as there's a reliable source to compare with. we have LARS and LAMB that compare with tonsorflow version because that's the reference for MLPerf |
Beta Was this translation helpful? Give feedback.
-
|
I don't think we need to rely on an external implementation to compare - as opposed to first party test logic. Why must we rely on an external implementation? What if the optimizer is only implemented in tinygrad and no other third party frameworks? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Do we really need to import torch? It seems like a bad idea.
tinygrad/test/test_optim.py
Line 2 in d051308
Beta Was this translation helpful? Give feedback.
All reactions