Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

function^n for iterated functions #39042

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

stevengj
Copy link
Member

@stevengj stevengj commented Dec 30, 2020

As suggested in #34251 (comment) by @StefanKarpinski and motivated more recently by #39031 for dirname, this PR defines function^n as an iterated function.

For example, (dirname^2)("/foo/bar/baz") == dirname(dirname("/foo/bar/baz")) == "/foo", and ((x -> (x + 2/x)/2)^10)(1.234) performs 10 iterations of Newton's method for sqrt(2), giving 1.414213562373095 ≈ sqrt(2).

(Note that you must call f^2 as (f^2)(x), not f^2(x), because the latter is parsed as f^(2(x)).)

Using literal_pow, I made f^0 == identity, f^1 == f, and f^2 == f ∘ f, up to f^4, for improved type inferability and inlining. (Note that f^-1 already calls inv(f), generally giving a MethodError, from literal_pow, and the interpretation as an iterated function is consistent with f^-1 denoting an inverse function.)

Keyword arguments of f^n are passed through to f. I also changed the identity function to ignore keyword arguments, and changed f ∘ g to pass any keywords through to both functions — previously, keywords were not allowed in either case, while the documentation for f ∘ g incorrectly stated that keywords were passed to g, so the new behavior is backwards compatible and more useful.

@stevengj stevengj added speculative Whether the change will be implemented is speculative needs docs Documentation for this change is required needs news A NEWS entry is required for this change labels Dec 30, 2020
@oscardssmith
Copy link
Member

The traditional issue with this has been that sin^2(x) is common notation for sin(x)^2

@stevengj
Copy link
Member Author

stevengj commented Dec 30, 2020

Yes, but the iterated-function notation is far more common and consistent — there are only a handful of functions like sin that commonly use the squaring notation, and even they are inconsistent: sin⁻¹(x) is standard notation for asin(x), not 1/sin(x). Moreover, we already have sin(x)^2, so there would not be much point in defining sin^2 with that meaning.

In consequence, settling on the iterated-function meaning doesn't bother me here; it is more general, more consistent, and far more useful (since it is vastly more concise and clear than any existing alternative) than mere exponentiation of the result.

@simeonschaub
Copy link
Member

I am not a big fan of using ^ for this, since right now, * and are separate operators and there are packages like ApproxFun, where for Fun objects both of these actually have a different meaning that should not be confused. This also limits this syntax for iterated application to callable objects that are <: Function, which is not limited to. What do you think about using ^ + unicode suffix for this instead? is already non-ASCII, so I don't think it would be unreasonable to use Unicode for this.

"""
struct IteratedFunction{F} <: Function
f::F
n::Int
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should be a type parameter instead, so application can be type stable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about that — it depends on how this is used. If n is almost always a hard-coded literal value, then a type parameter is definitely the way to go. On the other hand, if people use an n determined dynamically a lot, it would create a lot of dynamic dispatch.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe one could get the best of both words? Use n::TN where TN is Val{N} if a literal argument was used, and where TN is Int otherwise?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it really that common to construct a lot of these with dynamic n? The uses I had in mind was mostly with higher order functions like map, where the cost of calling the functor is much more important than that of constructing and dynamically passing it. Making this dependent on literal_pow sounds interesting, but I can still imagine use cases, where n is not a literal, but will still pretty much always constant propagate. I am also not 100% sure whether we want to make literal_pow more special, since it's not an uncommon source of confusion already.

@stevengj
Copy link
Member Author

What do you think about using ^ + unicode suffix for this instead?

I can't think of anything offhand that wouldn't be a bit ugly and confusing…

ApproxFun already overloads ^, so they would probably want to keep doing their own behavior. You can always call IteratedFunction(f, n) explicitly if you want to force this behavior or have a non-Function object.

@simeonschaub
Copy link
Member

I think superscript o would make sense for this: f ^ᵒ n.

ApproxFun already overloads ^, so they would probably want to keep doing their own behavior.

I am mainly concerned that x ^ ::Integer now doesn't always mean iterated multiplication anymore. I would be fine with f ^ n if we also defined f * g for function composition, but that's probably too late now.

@@ -940,9 +940,9 @@ function ∘ end
Represents the composition of two callable objects `outer::Outer` and `inner::Inner`. That is
```julia
ComposedFunction(outer, inner)(args...; kw...) === outer(inner(args...; kw...))
ComposedFunction(outer, inner)(args...; kw...) === outer(inner(args...; kw...); kw...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is the right definition. Arguments to the composed function should only be passed to inner. If it were possible for inner to return something that caused keyword arguments to be passed to outer, then that would be the right thing to do. Since that isn't possible, the outer function should never be passed keyword arguments.

The situation is similar to that with varargs: the inner function can take any number of arguments, but the outer function can only accept the inner function's returned value.

@StefanKarpinski
Copy link
Member

Possible Unicode symbol option is ∘̂.

@uniment
Copy link

uniment commented Feb 17, 2023

concerned that x ^ ::Integer now doesn't always mean iterated multiplication anymore. I would be fine with f ^ n if we also defined f * g for function composition

Isn't composition enough of a multiplication-like operator? After all, it has the same precedence:

julia> let prec=Base.operator_precedence; prec(:)  prec(:×)  prec(:)  prec(:*)  prec(:) end
true

The natural question should be: which type of "multiplication" should ^ distribute? Dot product $(\cdot)$? Cross product $(\times)$? Composition $(\circ)$? Convolution $(\ast)$? Cross-correlation $(\star)$? Note that convolution powers seem to be written $f^{*n}$.

I've seen iterated functions written $f^n$ plenty, and many more times I've seen an inverse $f^{-1}$ consistent with this, and constructing such an object is useful. Insisting that ^ distribute specifically the * character seems a rather artificial constraint which mathematical convention doesn't abide by; the world of basic math seems mostly agreed that raising a function to a power should refer to repeated composition $(\circ)$, even though raising a number to a power is repeated multiplication $(\cdot)$.

I'll add that $x\mapsto\cos(x)^2$ can anyway be easily composed as $\left(x\mapsto x^2\right)\circ\cos$, so for the trig folks to steal $\cos^2$ for this is rather rude (and a convention they themselves don't even respect when they write $\cos^{-1}=\arccos$).

FWIW, here's what ChatGPT thinks:

Where f is a function and x is a variable, what is (f^3)(x)? Does this operation have a name?

The notation (f^3)(x) means "the function f applied to x, then applied again to the result, and then applied a third time to the result." In other words, it is the composition of f with itself three times, which can also be written as f(f(f(x))). This operation is commonly referred to as "function composition" or "iterated function application."

Note that the notation (f^3)(x) should not be confused with the notation f(x)^3, which means "the result of applying the function f to x, and then raising the result to the third power."

@LilithHafner LilithHafner added the triage This should be discussed on a triage call label Jan 28, 2024
@jariji
Copy link
Contributor

jariji commented Jan 29, 2024

Insisting that ^ distribute specifically the * character seems a rather artificial constraint which mathematical convention doesn't abide by

I think in programming we need to be a bit more selective about our notation than in math, because while mathematicians have the benefit of context, culture, and critical thinking, computers take the notation literally.

For programming, I prefer to specify relationships between symbols, such as x ^ i means "* iterated". That way we can specify that interface with computer tooling so it's easy to understand and validate that it holds using PropCheck.jl or InterfaceSpecs.jl.

@uniment
Copy link

uniment commented Jan 30, 2024

I prefer to specify relationships between symbols, such as x ^ i means "* iterated".

While I empathize with the sentiment, in the case of * it seems that we already don't follow the policy you propose: matrix-matrix multiplication isn't merely + iteration, and for strings and regular expressions we have no + operator to iterate over. So imposing that ^ should only and always distribute * would be a policy specific to ^.

I think of the ^ character to mean primarily not * iteration but instead superscript so I don't see a problem, but maybe $\LaTeX$ gave me brain damage 😆

@jariji
Copy link
Contributor

jariji commented Jan 31, 2024

Btw, array languages have function iteration syntax, but they also allow n to be negative, which uses the inverse of the function.

Here I'm using https://github.com/JuliaMath/InverseFunctions.jl.

using InverseFunctions, Test
struct IteratedFunction{F}
    f::F
    n::Int
end
((;f,n)::IteratedFunction)(x) =
    if n == 0
        x
    elseif n > 0
        foldl((y,f)->f(y), [f for _ in 1:n]; init=x);
    else # if n < 0
        (inverse(f) ̂ (-n))(x)
    end

f ̂ n = IteratedFunction(f,n)

@test (exp̂1)(1)  2.718281828459045
@test (exp̂2)(1)  15.154262241479262
@test (exp̂3)(1)  3.814279104760214e6
@test (exp̂(-1))(ℯ)  1.0
@test (exp̂(-2))(ℯ)  0

@uniment
Copy link

uniment commented Jan 31, 2024

Yes, I'm enthusiastic about the possibility of f^-1 referring to inverse(f). 🤩

In #24990 I show an example where I write inverse(exp ∘ _^2); with that and this it could be written (exp ∘ _^2)^-1. (contingent on lowering to typed partially applied functions; currently it must be written inverse(exp ∘ Fix2(^, 2)))

I forgot to mention before, but here's another reason that iterated function composition is a coherent use of ^:

Where a linear transformation f(x::Vector)::Vector can be represented by a matrix 𝐀 such that f(x) == 𝐀*x, then repeating the transformation 𝐀^n * x (for example, if 𝐀 is a rotation matrix) can be written (f^n)(x). Indeed in math notation we write $\mathbf A^n\vec x = (f^n)(\vec x)$, since iterated multiplication of a matrix is directly analogous to iterated composition of a function.

@oschulz
Copy link
Contributor

oschulz commented Jan 31, 2024

Regarding f^-1 and inverse (I maintain InverseFunctions, together with @devmotion ):

A while ago I created a package FunctionChains that might be very suitable to host something like this (if it doesn't go into the language itself). FunctionChains supports InverseFunctions.inverse out of the box, so we can already do:

julia> using FunctionChains, InverseFunctions, FillArrays

julia> f = setinverse(x -> x * 2, x -> x / 2)

julia> g = fchain(Fill(f, 3))

julia> g(2)
16

julia> inverse(g)(16)
2.0

So in principle, this

̂(f, n::Integer) = n >= 0 ? fchain(Fill(f, n)) : fchain(Fill(inverse(f), -n))

would already do the trick:

julia> (f̂ 3)(2)
16

julia> (f̂ -3)(16)
2.0

julia> inverse(f̂ 3)(16)
2.0

FunctionChains has a few extra tricks, so this works too then

julia> with_intermediate_results(f̂ 3, 2)
3-element Vector{Int64}:
  4
  8
 16

and also

julia> using ChangesOfVariables

julia> f = setladj(x -> x * 2, _ -> log(2))

julia> with_logabsdet_jacobian(f̂3, 2)
(16, 2.0794415416798357)

It would certainly be better to use a dedicated IteratedFunction (as suggested by @jariji ) instead of fchain(Fill(...)) though - FillArrays would be kind of a heavy dependency. I'd be happy to add something like IteratedFunction to FunctionChains if people are interested - we could also move FunctionChains into JuliaMath or so, to make it more of a community package (I hadn't widely advertised the package so far).

@uniment
Copy link

uniment commented Feb 1, 2024

Point of note, ∘̂ currently has an undesirable operator precedence for this purpose.

julia> let prec=Base.operator_precedence;  prec(:̂), prec(:^)  end
(12, 15)

@oschulz
Copy link
Contributor

oschulz commented Feb 1, 2024

Point of note, ∘̂ currently has an undesirable operator precedence for this purpose.

Ah, yes - also, it's hard to type.

Since ^ is problematich (see discussion above) ... hm, what else might be a good fit?

@jariji
Copy link
Contributor

jariji commented Feb 1, 2024

I like the ∘̂ character for this. I wonder if we could add a new precedence class that requires parentheses to disambiguate. At least for a release or two. That's what Guy Steele advocated in his Juliacon talk: require parentheses unless the operator is learned by first-year university.

@uniment
Copy link

uniment commented Feb 1, 2024

I don't believe ^ is problematic (see discussion above).

@oschulz
Copy link
Contributor

oschulz commented Feb 2, 2024

I don't believe ^ is problematic (see discussion above).

Well, I'll be happy with whatever the language experts on this thread would prefer to go with.

If we do it in a package though, anything like ^(::Function,::Integer) or even ^(::Function,::Integer) would certainly be type-piracy though. So that would have to be done in the language (i.e. this PR).

Question, if we do this in the language, though, not in a package, how do get f^-n to call on InverseFunctions.inverse, though?

@uniment
Copy link

uniment commented Feb 2, 2024

I can imagine three possible paths at the moment:

  1. Include InverseFunctions in the language and have Base.:^(::Function, n::Integer) call InverseFunctions.inverse for n<0.

  2. Don't include InverseFunctions in the language but still have ^ call InverseFunctions.inverse, and if the package isn't loaded throw an error. (Admittedly this creates an interesting codependency between the language and a package, but maybe we can carve out an exemption for this case?)

  3. Declare in the language the function Base.inverse with no methods (leave it up to the user to define methods), and have ^ call inverse. Then, when a user loads InverseFunctions, it could run a loop to check if, for example, hasmethod(Base.inverse, Tuple{typeof(exp)}): if not, then define it as Base.inverse(::typeof(exp)) = InverseFunctions.inverse(exp). Does it count as type piracy if you define a method for a Base function on Base types, but you only define it if it isn't already defined?

@mbauman
Copy link
Member

mbauman commented Feb 2, 2024

Using ^ is problematic for exactly the same reason that Base.:^(::Function, ::Int) is pirate-y and non-ideal — dispatching on ::Function is a code-smell as you'd ideally want to also support iterated constructors and callables. But you can't define it in general because * is not .

Using an alternative like ^ᵒ or ∘̂ is possible right now in any package. Or a package could define its own independent ^ function that people could choose to use (shadowing the base definition) with using InverseFunctions: ^.

@uniment
Copy link

uniment commented Feb 2, 2024

Base.:^(::Function, ::Int) is pirate-y

Not if it's implemented in Base.

support iterated constructors and callables

  1. Is this an actual use case?
  2. Is it not satisfied by calling the IteratedFunction constructor implemented in this PR, as mentioned above by @stevengj?
  3. I understand that duck typing is preferred over dispatching on ::Function for good reasons (such as map), but is this case not an exception to that heuristic?

My suspicion is that a cost/benefit analysis will find the inconveniency of always needing to type ^ᵒ or ∘̂ outweighs that of occasionally needing to call the IteratedFunction constructor (heck, I'd rather type IteratedFunction than ^\^o<tab> or \circ<tab>\hat<tab> 😅). Not to mention, the mental load for f^n is lower because everybody already knows what it means from linear algebra.

Ok I'll step back now.

@oschulz
Copy link
Contributor

oschulz commented Feb 28, 2024

I'll add this to FunctionChains (oschulz/FunctionChains.jl#13) as f ∘̂ n so people can play with it. Question is, must negative n be supported? It's pretty much impossible to do in a type-stable fashion.

@oschulz
Copy link
Contributor

oschulz commented Dec 4, 2024

FunctionChains v0.1.6 supports this now:

julia> using FunctionChains

julia> (sqrt ̂  2)(16)
2.0

julia> (sqrt ̂  -2)(2)
16

Note that when chained this does not behave like exponentiation, and f ∘̂ n ∘̂ -n != f :

(sqrt ∘̂ -2 ∘̂ 2)(2) == ((sqrt ∘̂ -2) ∘̂ 2)(2) == 65536 == (sqrt ∘̂ -4)(2)

@LilithHafner LilithHafner removed the triage This should be discussed on a triage call label Dec 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs docs Documentation for this change is required needs news A NEWS entry is required for this change speculative Whether the change will be implemented is speculative
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants