Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add waitany and waitall functions to wait multiple tasks at once #53341

Merged
merged 30 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
1fb6a95
Add waitany and waitall to wait multiple tasks
mrkn Jan 22, 2024
7482838
Reduce the number of working vectors
mrkn Feb 4, 2024
5a30999
Rewrite with using Channel
mrkn Feb 8, 2024
664f8a3
Add test cases to Cover all argument types
mrkn Feb 15, 2024
8ac6ddc
Remove return type of _wait_multiple
mrkn Feb 27, 2024
f459e8a
Remove 1st argument type of _wait_multiple
mrkn Feb 27, 2024
eb2bafd
Specify type of element that comes from iteration for type stability
mrkn Feb 27, 2024
d7cf9dc
Support inputs that can be iterated only once
mrkn Mar 1, 2024
9159247
Delete waiters from waitq of each remaining task
mrkn Mar 1, 2024
573ee9f
Fix for performance
mrkn Mar 1, 2024
1175779
Split type checking and examining loops
mrkn Mar 4, 2024
633cb58
Stop using needless enumerate
mrkn Mar 5, 2024
4bec8cd
Optimize for waitall with failfast=false
mrkn Mar 5, 2024
b9dd9e6
Use vector for managing waiters
mrkn Mar 5, 2024
ae0ca9d
Add channel emptiness check
mrkn Mar 5, 2024
93057e6
Insert done check in waiter creation loop
mrkn Mar 5, 2024
ed58eda
Stop using kwargs in _wait_multiple
mrkn Mar 6, 2024
f1f400e
Add throw keyword argument in waitall
mrkn Mar 6, 2024
4d8e137
Add throw keyword argument in waitany
mrkn Mar 6, 2024
1a55697
Add docstrings of waitany and waitall
mrkn Mar 7, 2024
22646dd
Wait single task synchronously
mrkn Mar 7, 2024
505d476
Use TaskFailedException
mrkn Mar 7, 2024
1c9adbf
Stop using sleep in test
mrkn Mar 7, 2024
58d1e02
Remove needless yield call
mrkn Mar 7, 2024
3c9a9c8
Wait all three tasks in teardown function in test
mrkn Mar 7, 2024
4dd8862
Use consistent declarative tense in docstring
mrkn Mar 7, 2024
34e3d41
Add waitany and waitall in doc/src/base/parallel.md
mrkn Mar 7, 2024
8a7683f
Add waitany and waitall in NEWS.md
mrkn Mar 7, 2024
d30c9c0
Change default argument values
mrkn Mar 8, 2024
0a382a3
Add usage note of waitall in docstring
mrkn Mar 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Change default argument values
  • Loading branch information
mrkn committed Mar 9, 2024
commit d30c9c0766be40db124e3407a48457320299fdc3
8 changes: 4 additions & 4 deletions base/task.jl
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ end
# Wait multiple tasks

"""
waitany(tasks; throw=false) -> (done_tasks, remaining_tasks)
waitany(tasks; throw=true) -> (done_tasks, remaining_tasks)

Wait until at least one of the given tasks have been completed.

Expand All @@ -378,10 +378,10 @@ completed tasks completes with an exception.
The return value consists of two task vectors. The first one consists of
completed tasks, and the other consists of uncompleted tasks.
"""
mrkn marked this conversation as resolved.
Show resolved Hide resolved
waitany(tasks; throw=false) = _wait_multiple(tasks, throw)
waitany(tasks; throw=true) = _wait_multiple(tasks, throw)

"""
waitall(tasks; failfast=false, throw=false) -> (done_tasks, remaining_tasks)
waitall(tasks; failfast=true, throw=true) -> (done_tasks, remaining_tasks)

Wait until all the given tasks have been completed.

Expand All @@ -395,7 +395,7 @@ given tasks is finished by exception. If `throw` is `true`, throw
The return value consists of two task vectors. The first one consists of
completed tasks, and the other consists of uncompleted tasks.
"""
waitall(tasks; failfast=false, throw=false) = _wait_multiple(tasks, throw, true, failfast)
waitall(tasks; failfast=true, throw=true) = _wait_multiple(tasks, throw, true, failfast)

function _wait_multiple(waiting_tasks, throwexc=false, all=false, failfast=false)
tasks = Task[]
Expand Down
14 changes: 7 additions & 7 deletions test/threads_exec.jl
Original file line number Diff line number Diff line change
Expand Up @@ -1217,11 +1217,11 @@ end

for tasks_type in (Vector{Task}, Set{Task}, Tuple{Task})
@testset "waitany" begin
@testset "no options" begin
@testset "throw=false" begin
tasks, event = create_tasks()
wait(tasks[1])
wait(tasks[2])
done, pending = waitany(convert_tasks(tasks_type, tasks))
done, pending = waitany(convert_tasks(tasks_type, tasks); throw=false)
@test length(done) == 2
@test tasks[1] ∈ done
@test tasks[2] ∈ done
Expand Down Expand Up @@ -1260,13 +1260,13 @@ end
@test length(pending) == 0
end

@testset "failfast=true" begin
@testset "failfast=true, throw=false" begin
tasks, event = create_tasks()
push!(tasks, Threads.@spawn error("Error"))

wait(tasks[1])
wait(tasks[2])
waiter = Threads.@spawn waitall(convert_tasks(tasks_type, tasks); failfast=true)
waiter = Threads.@spawn waitall(convert_tasks(tasks_type, tasks); failfast=true, throw=false)

done, pending = fetch(waiter)
@test length(done) == 3
Expand All @@ -1279,22 +1279,22 @@ end
teardown(tasks, event)
end

@testset "throw=true" begin
@testset "failfast=false, throw=true" begin
tasks, event = create_tasks()
push!(tasks, Threads.@spawn error("Error"))

notify(event)

@test_throws CompositeException begin
waitall(convert_tasks(tasks_type, tasks); throw=true)
waitall(convert_tasks(tasks_type, tasks); failfast=false, throw=true)
end

@test all(istaskdone.(tasks))

teardown(tasks, event)
end

@testset "failfast=true and throw=true" begin
@testset "failfast=true, throw=true" begin
tasks, event = create_tasks()
push!(tasks, Threads.@spawn error("Error"))

Expand Down