Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redesign dictionary: use Vector{DictSlot{K,V}} #39

Merged
merged 1 commit into from
Sep 28, 2021
Merged

Conversation

tkf
Copy link
Member

@tkf tkf commented Sep 28, 2021

Since Julia does not have atomics for arrays, supporting boxed Julia objects in a hash table requires indirection (i.e., a mutable struct) somewhere at the moment because no atomic operation can be defined on Vector{Any}. Previously, LinearProbingDict was using a mutable struct only for "storage" and the actual linearization was done on a dense array using 128 bit CAS (based on Maier et al. (2019)). However, it required complex key/value storage management to actually store objects in the hash table. It turned out that just using Vector{DictSlot{K,V}} (where DictSlot{K,V} is a mutable struct) is more straightforward and beneficial in terms of performance. The important optimization to make it practical is to avoid copying DictSlot{K,V} during migration.

A nice side-effect of this approach is that it gets rid of the "torn" 64-bit load which is not likely to be supported in Julia. It is now possible to separate key and value to distinct Vectors if atomics on Vector{Any} is supported at some point. This is more likely to be supported than updating key/value location directly in Vector{Pair{Any,Any}}.

@tkf tkf enabled auto-merge (squash) September 28, 2021 21:33
@tkf tkf merged commit aed56a7 into master Sep 28, 2021
@tkf tkf deleted the dict-indirection branch September 28, 2021 21:40
@tkf
Copy link
Member Author

tkf commented Sep 28, 2021

-t16

julia> suite = ConcurrentCollectionsBenchmarks.BenchDictHistogram.setup();

julia> result = run(suite; verbose = true)
...
4-element BenchmarkTools.BenchmarkGroup:
  tags: []
  "cdict-ntasks=16" => Trial(91.662 ms)
  "base-seq" => Trial(120.610 ms)
  "cdict-seq" => Trial(320.926 ms)
  "cdict-ntasks=2" => Trial(243.681 ms)

-t1

3-element BenchmarkTools.BenchmarkGroup:
  tags: []
  "cdict-ntasks=1" => Trial(259.050 ms)
  "base-seq" => Trial(118.540 ms)
  "cdict-seq" => Trial(274.011 ms)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant