Default env & startup.jl loading pattern can cause excessive precompilation #56766
Description
Problem
A common usage pattern is for a user to have, say Revise.jl in their default env, and loaded in their startup.jl
So
$ julia
# Revise & deps are loaded from the @1.11 Manifest
pkg> activate Foo
pkg> add Bar
# Bar & deps are precompiled based on Foo/Manifest.toml only, ignoring already loaded deps
julia> using Bar
# Because some deps were already loaded by Revise, the only way to load Bar here is to precompile a new ad-hoc version with the different loaded deps
[ Info:Precompiling Bar [xxx] (cache misses: wrong dep version loaded (1))
To the user they have just had to sit through their newly added packages precompile twice, without clear explanation for why.
The same can be true if the user is simply switching env after startup, but the fix there is easier to teach: start julia with --project
to make loading Revise use the common dep versions from the target env.
Potential mitigations
-
Julia could inform the user why the package is being precompiled again.
It already does this with the debug info in serial precompile logs i.e.wrong dep version loaded (1)
above, but that's not the clearest messaging as it's often accompanied with other cache miss reasons that aren't clear to people unfamiliar with loading mechanics. So julia should probably add a clearer message. -
Provide a mechanism for packages like Revise to absorb deps as internal code, rather than a regular shared dep.
Something like@internalize using OrderdCollections
which would expand to
Base.allow_load_indirect_deps(true) # new function
include(Base.find_package("OrderedCollections"))
using .OrderedCollections
Base.allow_load_indirect_deps(false)
but that would be harder to do for the deps of deps.
Also if the packages people are loading in their startup.jl are more mixed usage, i.e. Plots.jl, then it's not a complete solution
-
Unload & reload packages when needed. Perhaps prompt the user first?
-
Some loading multiverse where sessions can fork into different loading states and have different versions loaded at the same time. Seems hard to do while maintaining interop that a user might expect.