[B! faiss] deejayrokaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

deejayroka id:deejayroka

faissã«é–¢ã™ã‚‹deejayrokaã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (5)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

ConversationalRetrievalChain ã§ LangChain ã® QA ã«ãƒãƒ£ãƒƒãƒˆå±¥æ´å®Ÿè£…
LangChain ã§ã¯ã€ EmbeddingAPI ã‚’ä½¿ã£ã¦ vector search ã¨ãã®çµæžœã‚’ LLM ã«ä¸Žãˆã¦ QA Bot ã‚’æ§‹ç¯‰ã—ãŸã‚Šã€ã‚ã‚‹ã„ã¯ ChatGPT ã®ã‚ˆã†ãªè¨˜æ†¶ãƒ»å±¥æ´(Memory)ã‚’å®Ÿè£…ã—ã¦ã€è‡ªç„¶ãªå¯¾è©±ã‚’è¡Œã†ä¾¿åˆ©ãªãƒ¢ã‚¸ãƒ¥ãƒ¼ãƒ«ãŒæƒã£ã¦ã„ã¾ã™ã€‚ã—ã‹ã—ã€ã“ã® Memory ã®ã‚ªãƒ–ã‚¸ã‚§ã‚¯ãƒˆã‚’ QA ãƒã‚§ãƒ¼ãƒ³ã«å…¥ã‚Œã¦ã‚‚ã€ãã‚Œã ã‘ã§ã¯ä¸Šæ‰‹ãå‹•ä½œã—ã¾ã›ã‚“ã€‚ ãã“ã§ä»Šå›žã¯ã€ã¾ãš QA ã‚’ä¸€ã¤ã® Chain ã‚ªãƒ–ã‚¸ã‚§ã‚¯ãƒˆã§å®Ÿè¡Œã™ã‚‹æ–¹æ³•ã¨ã€ã•ã‚‰ã« Memory ã‚’å®Ÿè£…ã™ã‚‹ã«ã¤ã„ã¦ã§ã™ã€‚ å‚è€ƒï¼šhttps://python.langchain.com/en/latest/modules/chains/index_examples/vector_db_qa.html å‚è€ƒï¼šhttps://python.langchain.com/en/latest/modules/chains/i
deejayroka 2023/05/25
â€œãã®ä»£ã‚ã‚Šã€ConbersationalRetrievalChain å†…ã§ä½¿ã‚ã‚Œã‚‹ qa_chain ãã®ã‚‚ã®ã‚’è¨å®š / å¤‰æ›´ã™ã‚‹ã“ã¨ã§ã“ã‚Œã‚‰ã®å„ç¨®æ©Ÿèƒ½ã«ã‚¢ã‚¯ã‚»ã‚¹ã—ãŸã‚Šã§ãã¾ã™ã€‚â€

langchain

faiss

ãƒãƒ£ãƒƒãƒˆ
ãƒªãƒ³ã‚¯
è¿‘å‚æŽ¢ç´¢ãƒ©ã‚¤ãƒ–ãƒ©ãƒªFaissã‚’ä½¿ã£ã¦ã¿ãŸ - Re:ã‚¼ãƒã‹ã‚‰å§‹ã‚ã‚‹MLç”Ÿæ´»
æœ€è¿‘ã€ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ç³»ã®ã“ã¨ã‚’ã‚„ã£ã¦ã„ã‚‹é–¢ä¿‚ã§è‰²ã€…èª¿ã¹ã¦ã¾ã™ã€‚ ä»¥å‰ã¯NGTã‚’ä½¿ã£ã¦è¿‘å‚æŽ¢ç´¢ã‚’è©¦ã—ã¦ã„ã¾ã—ãŸã€‚ www.nogawanogawa.com Embeddingã¨ã‹ã‚’ä½¿ã£ãŸæ¤œç´¢ã‚„ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‡ãƒ¼ã‚·ãƒ§ãƒ³ã«ã¯è¿‘å‚æŽ¢ç´¢ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã¯éžå¸¸ã«é‡è¦ã§ã™ã€‚ ã¡ã‚‡ã£ã¨èª¿ã¹ã¦ã¿ã‚‹ã¨ã€ãƒ¡ãƒ«ã‚«ãƒªã•ã‚“ãŒFaissã¨ã„ã†ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã‚’ä½¿ã£ã¦ã„ã‚‹ã‚ˆã†ã§ã—ãŸã€‚ tech.mercari.com ã¡ã‚‡ã£ã¨èˆˆå‘³ãŒå‡ºã¦ä½¿ã£ã¦ã¿ãŸã®ã§ã€ã€ä»Šå›žã¯ãã®ãƒ¡ãƒ¢ã§ã™ã€‚ è¿‘å‚æŽ¢ç´¢å•é¡Œ Faiss ã‚¤ãƒ³ã‚¹ãƒˆãƒ¼ãƒ« ä½¿ã£ã¦ã¿ã‚‹ æ„Ÿæƒ³ è¿‘å‚æŽ¢ç´¢å•é¡Œ è¿‘å‚æŽ¢ç´¢å•é¡Œã¯ã€ãƒ‡ãƒ¼ã‚¿é‡ã«å¿œã˜ã¦è¨ˆç®—é‡ãŒæ€¥æ¿€ã«å¢—å¤§ã™ã‚‹å¤å…¸çš„ãªå•é¡Œã§ã€ä»¥ä¸‹ã®ã‚ˆã†ãªç”¨é€”ã§ä½¿ã‚ã‚Œã¦ã„ã¾ã™ã€‚ æœ€è¿‘å‚æŽ¢ç´¢ã¯ã€ä»¥ä¸‹ã®ã‚ˆã†ãªæ§˜ã€…ãªåˆ†é‡Žã§ä½¿ã‚ã‚Œã¦ã„ã‚‹ã€‚ ãƒ»ãƒ‘ã‚¿ãƒ¼ãƒ³èªè˜ - ç‰¹ã«å…‰å¦æ–‡å—èªè˜ ãƒ»çµ±è¨ˆåˆ†é¡ž- Kè¿‘å‚æ³•å‚ç…§ ãƒ»ã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ã‚¿ãƒ“ã‚¸ãƒ§ãƒ³ ãƒ»ãƒ‡ãƒ¼ã‚¿ãƒ™ãƒ¼ã‚¹ - ä¾‹ãˆã°ã€å†…å®¹ã«åŸºã¥ãç”»åƒ
deejayroka 2023/04/17
faiss

github

js

ãƒ¬ã‚³ãƒ¡ãƒ³ãƒ‰
ãƒªãƒ³ã‚¯
é¡žä¼¼å•†å“APIã§ä½¿ã£ã¦ã„ã‚‹è¿‘å‚æŽ¢ç´¢ã®ãƒ„ãƒ¼ãƒ«ã‚’NGTã‹ã‚‰faissã«åˆ‡ã‚Šæ›¿ãˆãŸãŠè©± - BASEãƒ—ãƒãƒ€ã‚¯ãƒˆãƒãƒ¼ãƒ ãƒ–ãƒã‚°
ã“ã®è¨˜äº‹ã¯BASE Advent Calendar 2019ã®10æ—¥ç›®ã®è¨˜äº‹ã§ã™ã€‚ devblog.thebase.in ãŠä¹…ã—ã¶ã‚Šã§ã™ã€‚ BASEãƒ“ãƒ¼ãƒ«éƒ¨éƒ¨é•·ï¼ˆå…¼Data Strategyãƒãƒ¼ãƒ )ã®æ°åŽŸã§ã™ã€‚ 1å¹´ã¡ã‚‡ã£ã¨å‰ã«Yahoo!ã®è¿‘å‚æŽ¢ç´¢ãƒ„ãƒ¼ãƒ«NGTã‚’ä½¿ã£ã¦é¡žä¼¼å•†å“APIã‚’ã¤ãã‚‹ã¨ã„ã†è¨˜äº‹ã‚’æ›¸ãã¾ã—ãŸã€‚ã‚ã‚Œã‹ã‚‰ã ã„ã¶çµŒã¡ã¾ã—ãŸãŒãã®é–“ã«é¡žä¼¼å•†å“APIã¯ã‚³ãƒ„ã‚³ãƒ„ã¨æ”¹å–„ã‚’ç¶šã‘ã¦ã„ã¾ã™ã€‚ä¾‹ãˆã°ãƒ•ã‚¡ãƒƒã‚·ãƒ§ãƒ³ç³»ä»¥å¤–ã§ç²¾åº¦ãŒè‰¯ããªã„ã¨ã„ã†è©±ã‚’å‰ã®è¨˜äº‹ã«ã‚‚æ›¸ãã¾ã—ãŸãŒã€ç”»åƒä»¥å¤–ã«å•†å“ã®ã‚¿ã‚¤ãƒˆãƒ«ã‚„èª¬æ˜Žæ–‡ã®ç‰¹å¾´é‡ã‚‚ä½¿ã†ã‚ˆã†ã«ã™ã‚‹ãªã©ã—ã¦ãã‚Œã‚‚æ”¹å–„ã•ã‚Œã¦ã„ã¾ã™ã€‚ ãã†ã—ãŸå–ã‚Šçµ„ã¿ã®ãªã‹ã§è¿‘å‚æŽ¢ç´¢ã«ä½¿ã£ã¦ã„ãŸNGTã‚’åˆ¥ã®ã‚‚ã®ã«åˆ‡ã‚Šæ›¿ãˆã¾ã—ãŸã®ã§ã€ãã‚Œã‚’ç´¹ä»‹ã—ãŸã„ã¨æ€ã„ã¾ã™ã€‚ é¡žä¼¼å•†å“APIã«ã¤ã„ã¦ é¡žä¼¼å•†å“APIã¯BASEã®ã‚¢ãƒ—ãƒªã§å„å•†å“ã®è©³ç´°ãƒšãƒ¼ã‚¸ã‚’é–‹ã„ãŸã¨ãã«è¡¨ç¤ºã•ã‚Œã‚‹ã€Œé–¢é€£ã™ã‚‹å•†å“ã€ã¨ã„ã†éƒ¨åˆ†
deejayroka 2023/04/14
faiss
ãƒªãƒ³ã‚¯
Faissã®æ¦‚è¦ | Hakky Handbook
Hakkyã§ã¯ã€Œãƒ‡ãƒ¼ã‚¿ã§ãƒ—ãƒãƒ€ã‚¯ãƒˆã‚’ä¾¡å€¤ã‚ã‚‹ã‚‚ã®ã«ã™ã‚‹ã€ã¨ã„ã†ãƒŸãƒƒã‚·ãƒ§ãƒ³ã®å…ƒã€å¤§è¦æ¨¡è¨€èªžãƒ¢ãƒ‡ãƒ«ã‚’ç©æ¥µçš„ã«æ´»ç”¨ã—ã¦è¨˜äº‹åŸ·ç†ã—ã¦ãŠã‚Šã¾ã™ã€‚ãƒ“ã‚¸ãƒã‚¹ã«ãŠã‘ã‚‹AIæ´»ç”¨ã®ãŸã‚ã€Handbookã‚’ãŠå½¹ç«‹ã¦ã„ãŸã ãã¾ã—ãŸã‚‰å¹¸ã„ã§ã™ã€‚ æ¦‚è¦â€‹ æœ¬è¨˜äº‹ã§ã¯è¿‘ä¼¼æœ€è¿‘å‚æŽ¢ç´¢ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã® Faiss ã«ã¤ã„ã¦è§£èª¬ã—ã¾ã™ã€‚ Faiss ã¨ã¯â€‹ Faiss ã¨ã¯ã€Metaï¼ˆFacebookï¼‰è£½ã®è¿‘ä¼¼æœ€è¿‘å‚æŽ¢ç´¢ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã§ã‚ã‚Šã€é¡žä¼¼ã®ç”»åƒã‚„ãƒ†ã‚ã‚¹ãƒˆã‚’æ¤œç´¢ã™ã‚‹ãŸã‚ã®ã‚¤ãƒ³ãƒ‡ãƒƒã‚¯ã‚¹ã‚’ä½œæˆã™ã‚‹ãƒ„ãƒ¼ãƒ«ã§ã™ã€‚Faiss ã¯ C ++ã§è¨˜è¿°ã•ã‚Œã¦ã„ã¾ã™ãŒã€Python ãƒ©ãƒƒãƒ‘ãƒ¼ã‚’ä½¿ç”¨ã—ã¦ Python ã§é«˜é€Ÿãªå¦ç¿’ãŒå¯èƒ½ã§ã™ã€‚ ä»–ã®è¿‘ä¼¼æœ€è¿‘å‚æŽ¢ç´¢ãƒ©ã‚¤ãƒ–ãƒ©ãƒªâ€‹ Faiss ã®ä»–ã«ã€æœ‰åãªè¿‘ä¼¼æœ€è¿‘å‚æŽ¢ç´¢ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã¨ã—ã¦ã€FLANNã€Annoyã€NMSLIBã€NGT ãªã©ãŒã‚ã‚Šã¾ã™ã€‚ãƒ©ã‚¤ãƒ–ãƒ©ãƒªã®æ¯”è¼ƒã¯ä»¥ä¸‹ã®è¨˜äº‹ãŒå‚è€ƒã«ãªã‚Šã¾ã™ã€‚ è¿‘ä¼¼æœ€è¿‘
deejayroka 2023/04/14
faiss
ãƒªãƒ³ã‚¯
FAISS â€” ðŸ¦œðŸ”— LangChain 0.0.160
deejayroka 2023/04/14
faiss
ãƒªãƒ³ã‚¯
1