I don't think that we are ready to pay for the change of behaviour cost. However, I don't mind that we had a note in the docstring of the estimator.

Just pinging @lorentzenchr if he has additional thoughts.

I'm not speaking on @lorentzenchr's behalf, but I came across a similar discussion on Ridge/Lasso regression where he shared an incredibly helpful comment. I believe it’s relevant to this case as well.

That said, I still think @lorentzenchr is the best person to weigh in on this.

Thanks, @virchan, for your nice words and the link(s). So I don't need to repeat those arguments again.

Documentation:
If someone wants an improvement, then a PR is welcome.

Model Change:

I don't think that we are ready to pay for the change of behaviour cost.

I see it a bit different than @glemaitre. I'm open to change the penalty parameter of all linear models to be consistent. I find the different ways of different estimators quite annoying and error-prone.
This is, however, a decision that needs consensus among core contributors, therefore ping @scikit-learn/Core-devs.

I wouldn't mind the change to make them consistent.

I'm open to change the penalty parameter of all linear models to be consistent. I find the different ways of different estimators quite annoying and error-prone.

To be backward compatible, can we add a new parameter first like scale_regularization = {"n_samples", "none"}? Then we can make everything consistent in terms of this parameter (with a deprecation cycle if it is required for that estimator).

Thank you to all for paying so much attention to this problem! (and congrats for the creation of probabl)

If I may add something, what should the default be? loss.sum() + lambda * reg or loss.mean() + lambda * reg?

I feel like the answer should be, whichever is more stable when using CV to select the final lambda value, but I'm not sure which one it should be, aka which one selects the best lambda when evaluating on subsets.