[B! lsh] pcodã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

pcod id:pcod

lshã«é–¢ã™ã‚‹pcodã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (4)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

LSH (Locality Sensitive Hashing) ã‚’ç”¨ã„ãŸé¡žä¼¼ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ãƒšã‚¢ã®æŠ½å‡º - mixi engineer blog
GW ä¸ã®é•·è·é›¢ç§»å‹•ã®ãŸã‚ã«ä½“èª¿ãŒå„ªã‚Œãªã„ takahi-i ã§ã™. ä»Šå›žã¯å·¨å¤§ãªãƒ‡ãƒ¼ã‚¿ã‚’ãƒžã‚¤ãƒ‹ãƒ³ã‚°ã™ã‚‹ä¸€ã¤ã®æŠ€è¡“ã¨ã—ã¦ LSH (Localtiy Sensitive Hashing) ã‚’ç´¹ä»‹ã•ã›ã¦ã„ãŸã ãã¾ã™. LSH ã¨ã¯ LSH ã¯å¤§é‡ãªãƒ‡ãƒ¼ã‚¿ã‹ã‚‰é¡žä¼¼åº¦ãŒé«˜ã„ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã®ãƒšã‚¢ã‚’é«˜é€Ÿã«æŠ½å‡ºã—ã¦ãã‚Œã‚‹ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã§ã™. ã“ã“ã§ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã¯ãƒ‡ãƒ¼ã‚¿é›†åˆã®ä¸€ã¤ã®è¦ç´ ã‚’è¡¨ã—ã¾ã™. ãŸã¨ãˆã°æ‰±ã†ãƒ‡ãƒ¼ã‚¿ãŒ E-ã‚³ãƒžãƒ¼ã‚¹ã‚µã‚¤ãƒˆã®è³¼è²·ãƒã‚°ã§ã‚ã‚Œã°, ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã¯å„ãƒ¦ãƒ¼ã‚¶ã§ã™ã—, ç”»åƒãƒ‡ãƒ¼ã‚¿é›†åˆã§ã‚ã‚Œã°, ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ã¯å€‹ã€…ã®ç”»åƒãƒ‡ãƒ¼ã‚¿ã§ã™. LSH ã®è©³ã—ã„è§£èª¬ã«ã¤ã„ã¦ã¯ä»¥ä¸‹ã®ã‚µã‚¤ãƒˆãŒã‚ã‚Šã¾ã™. Wikipedia ã®ã‚¨ãƒ³ãƒˆãƒª LSH ã«é–¢ã™ã‚‹è«–æ–‡ãŒã¾ã¨ã‚ã‚‰ã‚Œã¦ã„ã‚‹ãƒšãƒ¼ã‚¸ æœ¬ç¨¿ã§ã¯E-ã‚³ãƒžãƒ¼ã‚¹ã‚µã‚¤ãƒˆã®è³¼è²·å±¥æ´ãƒ‡ãƒ¼ã‚¿ã‚’åŸºã« LSH ã®æ©Ÿèƒ½ã«ã¤ã„ã¦è¿°ã¹ã¦ã‚†ãã¾ã™. ä»¥ä¸‹ã®ã‚ˆã†ãª E-ã‚³ãƒžãƒ¼ã‚¹ã‚µã‚¤ãƒˆã®
pcod 2010/05/07
lsh

algorithm

hadoop
ãƒªãƒ³ã‚¯
Learning to Hash! æœ€æ–° Locality sensitive hashing äº‹æƒ… - æ¦è”µé‡Žæ—¥è¨˜
é«˜é€Ÿã«é¡žä¼¼åº¦è¨ˆç®—ã‚’ã—ãŸã„å ´åˆã€å…¸åž‹çš„ã«ä½¿ã‚ã‚Œã‚‹ã®ã¯ Locality sensitive hashing (LSH)ã¨ã„ã†æŠ€è¡“ã§ã‚ã‚Šã€å…ƒã€…è·é›¢ãŒè¿‘ã„ã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹åŒå£«ã¯ãƒãƒƒã‚·ãƒ¥å€¤ãŒè¿‘ããªã‚‹ã‚ˆã†ã«ãƒãƒƒã‚·ãƒ¥é–¢æ•°ã‚’ä½œã‚‹ã“ã¨ã§é«˜é€Ÿã«é¡žä¼¼åº¦ã‚’è¨ˆç®—ã—ãŸã‚Šã§ãã‚‹ã¨ã„ã†ãŠè©±ãªã®ã ãŒã€æœ€è¿‘ Semantic hashing ã‚„ Spectral hashingã€ã¾ãŸ Kernelized LSH ã¨ã„ã†æ‰‹æ³•ãŒç™»å ´ã—ã¦ç››ã‚Šä¸ŠãŒã‚Šã¤ã¤ã‚ã‚‹ã¨ã“ã‚ã€åŒã˜ã‚°ãƒ«ãƒ¼ãƒ—ã®äººãŒã‚‚ã£ã¨ã„ã„ã®ã‚’å‡ºã—ã¾ã—ãŸã€ã¨ã„ã†ã“ã¨ã‚‰ã—ã„ã€‚ã¡ãªã¿ã«æƒ…å ±æŽ¨è–¦ã¨ã‹ç”»åƒæ¤œç´¢ã¨ã‹å¤§è¦æ¨¡ã‚¯ãƒ©ã‚¹ã‚¿ãƒªãƒ³ã‚°ã¨ã‹ã€ã„ã‚ã„ã‚ãªåˆ†é‡Žã§é«˜é€Ÿãªé¡žä¼¼åº¦è¨ˆç®—ã®å¿œç”¨ä¾‹ãŒã‚ã‚‹ã€‚ ãã†ã„ã†ã‚ã‘ã§ã€ä»Šæ—¥ã¯ manab-ki ãã‚“ãŒ Brian Kulis and Trevor Darrell. "Learning to Hash with Binary Reconstructive
pcod 2009/11/16
lsh

algorithm
ãƒªãƒ³ã‚¯
æ¬¡å…ƒãŒé«˜ã„å ´åˆã«é–¢ã—ã¦ã®simhashã®è¨ˆç®— - tsubosakaã®æ—¥è¨˜
æœ€è¿‘simhashã®å®Ÿè£…ã‚’è¡Œã£ã¦ã„ã¦ã€ãƒ‡ãƒ¼ã‚¿ã®æ¬¡å…ƒãŒé«˜ã„ã¨simhashã‚’è¨ˆç®—ã™ã‚‹ã®ã«å¿…è¦ãªãƒ©ãƒ³ãƒ€ãƒ ãªãƒ™ã‚¯ãƒˆãƒ«ã‚’ãƒ¡ãƒ¢ãƒªä¸Šã«ä¹—ã‚‰ãªã„ã¨ã„ã†äº‹æ…‹ãŒç”Ÿã˜ãŸã®ã§ad hocãªæ–¹æ³•ã§å›žé¿ã—ã¦ã„ãŸã‘ã©ã€è«–æ–‡[1]ã‚’ã‚ˆãè¦‹ç›´ã™ã¨ã»ã¼åŒã˜æ–¹æ³•ã§ã‚ˆã‚Šè¨ˆç®—ã‚³ã‚¹ãƒˆãŒå°‘ãªã„æ–¹æ³•ãŒç´¹ä»‹ã—ã¦ã‚ã£ãŸã®ã§å°‘ã—è§£èª¬ã‚’è¡Œã£ã¦ã¿ã‚‹ã€‚ã¡ãªã¿ã«ä»¥ä¸‹ã®è§£èª¬ã§ã¯ä½Žæ¬¡å…ƒã®ãƒ“ãƒƒãƒˆãƒ™ã‚¯ãƒˆãƒ«ã«ç¸®ç´„ã—ãŸå¾Œã«ãƒãƒŸãƒ³ã‚°è·é›¢ãŒè¿‘ã„ã‚‚ã®ã‚’ã©ã†ã‚„ã£ã¦æŽ¢ç´¢ã™ã‚‹ã‹ã«ã¤ã„ã¦ã¯è¿°ã¹ãªã„ã§ã™ã€ãã‚Œã«é–¢ã—ã¦ã¯[1],[2]ã‚’å‚ç…§ã—ã¦ãã ã•ã„ã€‚ ã¡ãªã¿ã«è‡ªåˆ†ãŒå®Ÿè£…ã—ãŸã®ã¯å„ãƒ“ãƒƒãƒˆã”ã¨ã«æ¬¡å…ƒã«å¯¾ã™ã‚‹ãƒãƒƒã‚·ãƒ¥é–¢æ•°ã‚’å®šç¾©ã—ã¦è¨ˆç®—ã™ã‚‹æ–¹æ³•ã§ã—ãŸã€‚ã“ã®æ–¹æ³•ã ã¨ä»¥ä¸‹ã§é–‹è¨ã™ã‚‹æ‰‹æ³•ã‚ˆã‚Šã‚‚få€ã®å›žæ•°ãƒãƒƒã‚·ãƒ¥é–¢æ•°ã‚’è¨ˆç®—ã™ã‚‹å¿…è¦ãŒã‚ã‚‹ã®ã§å®Ÿè¡Œæ™‚é–“ãŒå‰²ã¨ã‹ã‹ã‚‹ã€‚ è§£èª¬ simhash[3](æ–‡çŒ®ã«ã‚ˆã£ã¦ã¯LSHã¨å‘¼ã¶ã“ã¨ã‚‚ã‚ã‚‹[2])ã¯æ¬¡å…ƒå‰Šæ¸›ã®æ‰‹æ³•ã®ä¸€ã¤ã§ã€é«˜æ¬¡å…ƒã®ãƒ‡ãƒ¼ã‚¿ã‚’ä½Žæ¬¡å…ƒã®ãƒ“ãƒƒãƒˆ
pcod 2009/10/04
algorithm

lsh

nlp
ãƒªãƒ³ã‚¯
LSHKIT: LSHKIT: A C++ Locality Sensitive Hashing Library
This library is part of the CASS project. Author:Wei Dong wdong [@] cs.princeton.edu 2008-2009 UPDATES lshkit 0.2.1 - 2009-09-17 * A Posterirori Multi-Probe LSH (include/lshkit/apost.h tools/apost-run.cpp) * Spectral hashing (include/lshkit/spectral-hash.h) lshkit pre-0.2 - 2009-03-02 * Introduced experimental scanner interface. Scanner is a unary functional object which is passed into LSH index t
pcod 2009/06/15
lsh
ãƒªãƒ³ã‚¯
1