分散ストレージに使えるかもしれないアルゴリズム
•
19 likes•2,637 views
The document discusses EpiChord, an enhancement of the Chord distributed hash table. It presents EpiChord's lookup algorithm and its division of the address space to reduce routing table sizes while maintaining low diameter. Bloom filters are also summarized, including how they represent sets using bit vectors and can check for set membership with possible false positives. Techniques like vector clocks for maintaining causality in distributed systems are also briefly covered.
![Bloom filter
BF
m
0
0
0
0
0
0
0
0
ZF
m
hi
i 0<i k
[0,m-1] hi(x)
0
1
0
0
1
0
0
0
BF = ZF
BF[ hi(x) ] = 1 hi(y)
0
1
0
0
1
1
0
0
Bloom filter m=8, k=2 x, y](https://image.slidesharecdn.com/20100226-erlang-100226103058-phpapp02/85/-31-320.jpg)
![Bloom filter
membership query
F = ZF
hi(x)
included
F[ hi(x) ] = 1
BF & F == F 0
1
0
0
1
1
0
0
hi(z)
not included
0
1
0
0
1
1
0
0
false positive!
hi(w)
included
0
1
0
0
1
1
0
0](https://image.slidesharecdn.com/20100226-erlang-100226103058-phpapp02/85/-32-320.jpg)
![Bloom filter
k p
1
p= 0
1
0
0
1
1
0
1
2
k
Bloom filter
[mitzenmacher02]
€
k
0 1
](https://image.slidesharecdn.com/20100226-erlang-100226103058-phpapp02/85/-39-320.jpg)
分散ストレージに使えるかもしれないアルゴリズム
- 1. 2010-02-26 4 Erlang
- 2. 1/2 No Erlang 3 Chord VectorClocks …
- 3. 2/2 Chord EpiChord BASE Sinfonia: VectorClocks Bloom ZDD EpiChord
- 4. Lookup 1 0-10 1 5? 11-20 2 2 21-30 3 3 ! ?
- 5. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Address Space 0-63
- 6. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Mapping Keys to Nodes 1-6 6 : : 52-57 57 : :
- 7. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf Chord 35 36-40 40 41-47 47 48-50 49 51-2 57 3-30 6 • log • log
- 8. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
- 9. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
- 10. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
- 11. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
- 12. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
- 13. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
- 14. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
- 15. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Lookup Algorithm
- 16. B. Leong et al. http://www.comp.nus.edu.sg/~bleong/slides/icon-epichord-slides.pdf EpiChord Division of Address Space ?
- 17. J. Xu et al., On the Fundamental Tradeoffs between Routing Table Size and Network Diameter in Peer-to-Peer Networks Lookup (Dynamo?) EpiChord?
- 18. ACID: 1. SELECT … 2. INSERT … 3. : DB 1 ? ! ?
- 19. S. Shinohara http://www.slideshare.net/shino/conflict-resolution-in-kai-presentation VectorClocks “happens-before” (->) BASE VectorClocks
- 20. S. Shinohara http://www.slideshare.net/shino/conflict-resolution-in-kai-presentation VectorClocks “concurrent” (||) BASE VectorClocks
- 21. T.Yamamuro (postgresql’s) 2-phase commit
- 22. T.Yamamuro Sinfonia 2-phase commit’s optimization ?
- 23. T.Yamamuro Sinfonia mini-transaction primitives
- 24. T.Yamamuro Sinfonia Sinfonia’s 2-phase commit =
- 25. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
- 26. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
- 27. T.Yamamuro Sinfonia Sinfonia 2-phase commit 5
- 28. T.Yamamuro Sinfonia Evaluation: Scalability NFS, 250 2 100
- 29. VectorClocks Sinfonia 2-phase commit BASE ACID
- 30. {apple, banana} + {orange} Is apple a member? = {apple, banana, orange} {apple, banana, orange} ! ? Q. ? A. BigTable Bloom
- 31. Bloom filter BF m 0 0 0 0 0 0 0 0 ZF m hi i 0<i k [0,m-1] hi(x) 0 1 0 0 1 0 0 0 BF = ZF BF[ hi(x) ] = 1 hi(y) 0 1 0 0 1 1 0 0 Bloom filter m=8, k=2 x, y
- 32. Bloom filter membership query F = ZF hi(x) included F[ hi(x) ] = 1 BF & F == F 0 1 0 0 1 1 0 0 hi(z) not included 0 1 0 0 1 1 0 0 false positive! hi(w) included 0 1 0 0 1 1 0 0
- 33. Bloom filter O k n
- 34. Bloom filter OR AND m, hi 0 1 0 0 1 1 0 0 0 0 0 1 1 0 0 1 0 1 0 1 1 1 0 1
- 35. Bloom filter false positive p 1 kn kn p = 1− ≈ exp − m m f € k f = (1− p) € m = 256, n = 32
- 36. Bloom filter false positive f k f k m k = ln2 ⋅ n floor k € f m f ≈ 0.6185 n k
- 37. Bloom filter n, f m m = −1.44n log 2 f k €
- 38. Bloom filter b b = −1.44 log 2 f k b = −log 2 f 9.6 1% € 1.44 € 9.6 Bloom filter b 1%
- 39. Bloom filter k p 1 p= 0 1 0 0 1 1 0 1 2 k Bloom filter [mitzenmacher02] € k 0 1
- 40. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD (Binary Decision Diagram) {abc, ab, ac, b, } trie 0 (false) 1 (true)
- 45. S. Minato http://www-alg.ist.hokudai.ac.jp/~minato/alg2009-j.html ZDD BDD ZDD: Zero-suppressed BDD
- 49. EpiChord B. Leong et al., “EpiChord: Parallelizing the Chord lookup algorithm with reactive routing state management,” Comput. Commun., vol.29, no.9, pp.1243-1259, 2006. Sinfonia M.K. Aguilera et al., “Sinfonia: a new paradigm for building scalable distributed systems,” Proc. of SOSP’07, pp.159-174, 2007. ZDD S. Minato, “Zero-suppressed BDDs for set manipulation in combinatorial problems,” Proc. of DAC’93, pp. 272-277, 1993.