Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-1.9: Backports for julia 1.9.0-alpha2 / 1.9.0-beta1 #47602

Merged
merged 57 commits into from
Dec 27, 2022

Conversation

KristofferC
Copy link
Member

@KristofferC KristofferC commented Nov 17, 2022

Backported PRs:

Non-merged PRs with backport label:

staticfloat and others added 4 commits November 17, 2022 07:46
* Limit initial OpenBLAS thread count

We set OpenBLAS's initial thread count to `1` to prevent runaway
allocation within OpenBLAS's initial thread startup.  LinearAlgebra will
later call `BLAS.set_num_threads()` to the actual value we require.

* Support older names

(cherry picked from commit 58b559f)
This has probably been wrong for a long time (since being introduced in 7908246).

(cherry picked from commit b369511)
@KristofferC KristofferC added the release Release management and versioning. label Nov 17, 2022
@DilumAluthge

This comment was marked as outdated.

@KristofferC KristofferC changed the title release-1.9: Backports for julia 1.9.0-alpha1 / 1.9.0-beta1 release-1.9: Backports for julia 1.9.0-alpha2 / 1.9.0-beta1 Nov 17, 2022
@KristofferC

This comment was marked as outdated.

vchuravy and others added 13 commits November 28, 2022 21:10
* Update doc/src/base/sort.md

* Update docs: The default sorting alg. is stable

* Compat 1.9 for QuickSort to be stable

* Specify the default algorithm

* Use example from InlineStrings.jl

* Change example to jldoctest

* Remove "*appear* to be stable." as slightly misleading.

Co-authored-by: Lilith Orion Hafner <[email protected]>
(cherry picked from commit c5fe17b)
checksums updated via approach described in #47174.

(cherry picked from commit 5996520)
Fixes `make -C deps getall`

(cherry picked from commit 3200219)
* Remove typeinfer lock altogether

* Don't remove the typeinf lock functions

* Track reentrancy in current task state

* Fix up some git status

* Initialize task variables

* Promise that jl_typeinf_func is rooted somewhere

(cherry picked from commit 113efb6)
The command-line program `rm` has no problem deleting an empty directory
that we do not have listing permissions on, so we should follow suit.

Example:

```
mktempdir() do dir
    mkpath("$(dir)/foo")
    chmod("$(dir)/foo", 0o200)
    rm(dir; recursive=true)
end
```

(cherry picked from commit d0a211a)
Fixup for #46764

(cherry picked from commit 02aa0b0)
* Add test/gcext to out-of-tree
* Disable gcext test that uses jl_gc_internal_obj_base_ptr

(cherry picked from commit 5495b8d)
Co-authored-by: N5N3 <[email protected]>
(cherry picked from commit 902e8a7)
@giordano
Copy link
Contributor

I don't understand why #46976 is in the list of PRs, eb708d6 is already in v1.9.0-alpha1

KristofferC and others added 8 commits December 8, 2022 08:04
* Add support for "glue packages" to code loading

This allows packages to define "glue packages" which
are modules that are automatically loaded when
a set of other packages are loaded into the Julia
session.

(cherry picked from commit 495a004)
Co-authored-by: Dilum Aluthge <[email protected]>
(cherry picked from commit db00cc1)
* Fix and simplify inference timing logic

* Reduce task struct size

(cherry picked from commit 88a0627)
This code was introduced by me back in #31025 to speed up evaluation
of generated functions that didn't make use of all of their arguments to
make generation decisions. However, it neglected to take into account the
possibility that the generator could be varargs. As a result, an unfortunate
coincidence of an unused slot in the correct position could have allowed
expansion of generators that were not supposed to be expandable. This can
cause incorrect inference with all the usual consequences. However, fortunately
this coincidence appears to be pretty rare.

Fixes JuliaDebug/CassetteOverlay.jl#12

(cherry picked from commit 328dd57)
Adapts to the final version of the constrained/available memory APIs.

(cherry picked from commit 41b73e2)
In looking at a TSAN report recently, I noticed that globals were
getting stored as atomic-unordered (since c92ab5e #44182), instead
of atomic-release as intended (since
46135df #45484).

(cherry picked from commit f4534d1)
KristofferC and others added 7 commits December 16, 2022 10:18
When doing intersection, we might end up with a value in `env` (as the
only possible *value* for that parameter) without properly considering
that the parameter might be a TypeVar.

(cherry picked from commit 26a7dbb)
fixes #46635

co-authored-by: Jameson Nash <[email protected]>
(cherry picked from commit b6f32bc)
@KristofferC KristofferC force-pushed the backports-release-1.9 branch from 51f1df2 to 5a684f0 Compare December 20, 2022 16:37
KristofferC and others added 4 commits December 20, 2022 17:37
We cache only those external CodeInstances that link back to the
package being precompiled. Formerly we required a backedge;
this PRs adds any whose `specTypes` could only link back to the
package. This scoops up a few runtime-dispatched CodeInstances
and their callees.

(cherry picked from commit 1f0700a)
…s... (#47788)

...as they were in 1.8 and rename the new PartialQuickSort to QuickerSort
Also improve the documentation and API for constructing QuickerSort and test
the API

Co-authored-by: Lilith Hafner <[email protected]>
(cherry picked from commit 8cdb17b)
Co-authored-by: Simon Byrne <[email protected]>
(cherry picked from commit d7363d8)
@KristofferC
Copy link
Member Author

@nanosoldier runbenchmarks(ALL, vs = ":release-1.8")

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

@KristofferC
Copy link
Member Author

@nanosoldier runbenchmarks(ALL, vs = ":release-1.9")

@KristofferC
Copy link
Member Author

Some observations:

  • Sort slower and allocates a lot more. Probably fixed to some extent by Apply InitialOptimizations more consistently in sorting #47946 (@LilithHafner
  • Sparse matmul looks slower across the board. (@dkarrasch)
  • ["linalg", "arithmetic", ("-", "Matrix", "Matrix", 1024)] 2x slower.
  • ["collection", "set operations", ("Set", "Int", "union!", "BitSet")] started allocating
  • ["inference", "abstract interpretation", "many_global_refs"] got 10x slower (vs 1.8)

@LilithHafner
Copy link
Member

Note that BaseBenchmarks measures sort(x; alg=XXX) but does not measure sort(x) so the sorting regressions are worth addressing but should not effect most user code. sort(x) is often much faster.

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

@KristofferC
Copy link
Member Author

@nanosoldier runbenchmarks("sort", vs = ":release-1.8")

@nanosoldier
Copy link
Collaborator

Your benchmark job has completed - possible performance regressions were detected. A full report can be found here.

pchintalapudi and others added 4 commits December 27, 2022 15:01
(cherry picked from commit 09a6ff8)
This implements caching native code in "package images" (pkgimages).
We now write two serialization files, one ending in `*.ji` and the other with
the platform dynamic library extension (e.g., `*.so`). The `*.ji` contains
"extended" header information (include the source-code dump for Revise),
whereas the dynamic library includes the Julia objects, including LLVM-generated
native code.

Native code is compiled once during precompilation and then a second time
to build a "clean" module. When we find an edge to an external function (already
cached in anloaded pkgimage), we emit a global variable which we will patch during
loading with the address of the function to call. This allows us to leverage the
standard multiversioning capabilities.

Co-authored-by: Tim Holy <[email protected]>
Co-authored-by: Kristoffer Carlsson <[email protected]>
Co-authored-by: Mosè Giordano <[email protected]>
Co-authored-by: Ian Butterworth <[email protected]>
Co-authored-by: Max Horn <[email protected]>
Co-authored-by: Michael Schlottke-Lakemper <[email protected]>
Co-authored-by: Alex Ames <[email protected]>
(cherry picked from commit a2db90f)
…tch bug (#47946)

* Apply InitialOptimizations by default in several cases when it was previously present

* fixup for MissingOptimization

* fix stability in the sortperm union with missing case

(cherry picked from commit 12e679c)
Commit c8b72e2 completely removed libgcc_s symlinking (I assume unintentionally) in !macOS.

(cherry picked from commit ea13810)
@KristofferC
Copy link
Member Author

Let's do a beta now after CI goes through.

@oscardssmith
Copy link
Member

We probably should backport #47184 first.

@KristofferC
Copy link
Member Author

KristofferC commented Dec 27, 2022

It's backported already. (I mean, I wouldn't suggest a beta just after that got merged without it being backported heh).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release Release management and versioning.
Projects
None yet
Development

Successfully merging this pull request may close these issues.