Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

glum v3.0 #677

Merged
merged 77 commits into from
Apr 27, 2024
Merged

glum v3.0 #677

merged 77 commits into from
Apr 27, 2024

Conversation

MartinStancsicsQC
Copy link
Contributor

@MartinStancsicsQC MartinStancsicsQC commented Aug 14, 2023

Checklist

  • Added a CHANGELOG.rst entry

This PR contains the changes needed to make glum compatible with the new major release of tabmat. Branches and PRs that make use of new functionality in Tabmat 4.0 will be based on this gulm-v3 branch.

Related issues: closes #730, closes #728, closes #583, closes #775.

stanmart and others added 9 commits July 25, 2023 10:18
* Do not use _set_up_... in covariance_matrix

* Add changelog entry
…#661)

* Add option to store covariance matrix during fit

* Fix fitting with variance matrix estimation

`.covariance_matrix()` expects X and weights in a different format than
what we have at the end of `.fit().

* Store covariance matrix after estimation

* Handle the alpha_search and glm_cv cases

* Propagate covariance parameters

* Add changelog

* Slightly more lenient tests
Co-authored-by: quant-ranger[bot] <132915763+quant-ranger[bot]@users.noreply.github.com>
@MartinStancsicsQC MartinStancsicsQC added this to the Glum 3.0 milestone Aug 14, 2023
@MartinStancsicsQC MartinStancsicsQC added the on hold not now, maybe never label Aug 14, 2023
@MartinStancsicsQC MartinStancsicsQC force-pushed the glum-v3 branch 2 times, most recently from edd5745 to 940b260 Compare August 16, 2023 08:25
@MarcAntoineSchmidtQC MarcAntoineSchmidtQC changed the title Compatibility with Tabmat v4.0 glum v3.0 alpha Aug 16, 2023
* Delegate column naming to tabmat

* Add tests

* More tests

* Test for dropping complete categories

* Add docstrings for new argument

* Add changelog entry

* Convert to pandas at the correct place

* Reorganize converting from pandas

* Remove xfail from test
* Add formulaic to dependencies

* Add function for transforming the formula

* Add tests

* First draft of glum formula interface

* Fixes and tests

* Handle intercept correctly

* Add formula functionality to glm_cv

* Variables from local context

* Test predict with formulas

* Add formula tutorial

* Fix tutorial

* Reformat tutorial

* Improve function signatures adn docstrings

* Handle two-sided formulas in covariance_matrix

* Make mypy happy about module names

* Matthias' suggestions

* Improve tutorial

* Improve tutorial
* Add formulaic to dependencies

* Add function for transforming the formula

* Add tests

* First draft of glum formula interface

* Fixes and tests

* Handle intercept correctly

* Add formula functionality to glm_cv

* Variables from local context

* Test predict with formulas

* Add formula tutorial

* Fix tutorial

* Reformat tutorial

* Improve function signatures adn docstrings

* Handle two-sided formulas in covariance_matrix

* Make mypy happy about module names

* Matthias' suggestions

* Add back term-based Wald-tests

* Tests for term names

* Add formula-based Wald-test

* Tests for formula-based Wald-test

* Add changelog

* Fix exception message

* Additional test case

* make docstrings clearer in the case of terms
* Delegate column naming to tabmat

* Add tests

* More tests

* Test for dropping complete categories

* Add docstrings for new argument

* Add changelog entry

* Convert to pandas at the correct place

* Reorganize converting from pandas

* Remove xfail from test

* Implement missing categorical support

* Add test

* Solve adding missing category when predicting

* Apply Matthias' suggestions

* Add changelog entry
* Make tests fail

* Propagate context through methods
@MarcAntoineSchmidtQC MarcAntoineSchmidtQC removed the on hold not now, maybe never label Oct 13, 2023
@MatthiasSchmidtblaicherQC
Copy link
Contributor

MatthiasSchmidtblaicherQC commented Apr 23, 2024

FYI, CI is currently failing because tabmat 4.0.0 is not available on conda-forge yet.

The reported errors are:
- Encountered problems while solving:
-   - nothing provides requested tabmat >=4.0.0

@MatthiasSchmidtblaicherQC
Copy link
Contributor

New CI failure is due to missing libblas >=0 *mkl when solving environment on macos-latest - Py3.12. Same failure occurs on main.

@xhochy
Copy link
Member

xhochy commented Apr 26, 2024

I think they switched macos-latest to being an ARM-based image in the last two days. Thus, you cannot install the Intel-only libblas >=0 *mkl package there.

* try solving on ci with different env for macos

* add missing if

* typo

* try and remove --no-test flag
@MatthiasSchmidtblaicherQC
Copy link
Contributor

I added a separate environment for macos without the libblas >=0 *mkl dependency. This has the additional benefit that it is more prominent to Mac users who might miss the comment in the standard environment.yml IMO.

@MatthiasSchmidtblaicherQC
Copy link
Contributor

MatthiasSchmidtblaicherQC commented Apr 26, 2024

Nightly failures (or at least most of them) should because of shorthand scipy.sparse.*_matrix.A in the simulations, which is deprecated in future scipy versions, see also here. Will replace them on this branch.

@MatthiasSchmidtblaicherQC
Copy link
Contributor

MatthiasSchmidtblaicherQC commented Apr 27, 2024

The remaining 34 failures on the nightly builds are due to incompatibilities between upstream packages and pre-release versions of other dependencies, namely:

  • tabmat calling scipy.sparse.*_matrix.A, which will be deprecated in scipy, issue here,
  • formulaic not yet being compatible with the pandas 3.0.0* development version, issue here.

These should be addressed in the upstream packages.

Unless anyone objects, I will merge the PR and release soon.

@MatthiasSchmidtblaicherQC MatthiasSchmidtblaicherQC changed the title glum v3.0 alpha glum v3.0 Apr 27, 2024
@MatthiasSchmidtblaicherQC MatthiasSchmidtblaicherQC merged commit 653b419 into main Apr 27, 2024
18 of 19 checks passed
@MatthiasSchmidtblaicherQC MatthiasSchmidtblaicherQC deleted the glum-v3 branch April 27, 2024 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
7 participants