Add logsumexp function #491
Conversation
This adds the logsumexp function using the logsumexp trick which is the numerical stable version of applying `log(sum(exp(x)))`. I chose to default to the backend implementation if available (pytorch, numpy/scipy, tensorflow, jax) and implemented the numerical stable computation of `log(sum(exp(x)))` when the backend did not provide a logsumexp function (mxnet, cupy). Additionally, I've added a unit test that checks against the numpy/scipy baseline with some 3d data tensor over all axis.
Codecov Report
@@ Coverage Diff @@
## main #491 +/- ##
==========================================
- Coverage 86.79% 83.12% -3.68%
==========================================
Files 120 120
Lines 7392 7395 +3
==========================================
- Hits 6416 6147 -269
- Misses 976 1248 +272
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
Thanks @braun-steven! Is there anywhere it could be used in the current codebase or is this just for new functionalities? |
|
I haven't found any use cases in the current code base. We need the functionality for our library SPFlow which we are currently rewriting to be used with tensorly as backend. We could have also written it as part of our codebase but thought that pushing the added functionality upstream would be good for tensorly as well. |
|
Indeed, thanks for contributing @braun-steven ! Looks good to me, merging. |
|
As a side note @braun-steven let us know if you need more features on the tensorly side -- we can also showcase SPFlow somewhere on the website, e.g. in an ecosystem page or something! |
Will do! But I think we will probably be able to implement most functionality that we need and is missing ourselves and push it upstream. In case of issues/questions/problems we will definitely get back at you :) The current status is that the rewrite of SPFlow is quite far and we wanted to initially support a numpy and a torch backend which was almost done, with support for TensorFlow and Jax in future (>1.0.0) versions. Then we found out about tensorly and investigated if it is worth the effort to replace the numpy backend with tensorly to get all the backends "for free" (with the exception of some specific modules such as probability distributions etc). So the current status is that we are replacing numpy with tensorly and see how far we come and into how many issues we run on our side.
That sounds awesome! Once the backend rewrite is done and we are ready to release our 1.0.0 version built on top of tensorly, I would gladly come back at you :) |
|
I think this wasn't added to the docs, or did I just miss it? |
|
Indeed, it hasn't been added, good catch @felixdivo! Do you want to make a small PR and add it to the doc (in the API doc)? |
3 participants
This adds the
logsumexpfunction using the logsumexp trick which is the numerical stable version of applyinglog(sum(exp(x))). I chose to default to the backend implementation if available (pytorch, numpy/scipy, tensorflow, jax) and implemented the numerical stable computation oflog(sum(exp(x)))when the backend did not provide a logsumexp function (mxnet, cupy).Additionally, I've added a unit test that checks against the numpy/scipy baseline with some 3d data tensor over all axis.