[B! æ©Ÿæ¢°å¦ç¿’][LLM] yuki_2021ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

yuki_2021 id:yuki_2021

æ©Ÿæ¢°å¦ç¿’ã¨LLMã«é–¢ã™ã‚‹yuki_2021ã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (6)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

GPTã‚’ç”Ÿã‚“ã å…ˆç«¯æŠ€è¡“ã€ŒTransformerã€ã€ãã®åŸºç¤Žã‚’æ£ã—ãç†è§£ã™ã‚‹
å¯¾è©±åž‹AIã‚µãƒ¼ãƒ“ã‚¹ã€ŒChatGPTã€ã§ä½¿ã‚ã‚Œã¦ã„ã‚‹è‡ªç„¶è¨€èªžãƒ¢ãƒ‡ãƒ«ã®æ ¸ã«ãªã£ã¦ã„ã‚‹æŠ€è¡“ãŒã€ŒTransf ormerã€ã ã€‚è‡ªç„¶è¨€èªžå‡¦ç†ã‚’ä¾‹ã«ã€Transf ormerã®ä»•çµ„ã¿ã‚’è§£èª¬ã™ã‚‹ã€‚ ã™ã£ã‹ã‚Šå¸‚æ°‘æ¨©ã‚’å¾—ãŸå¯¾è©±åž‹AIã‚µãƒ¼ãƒ“ã‚¹ã€ŒChatGPTã€ã€‚ChatGPTã§ç”¨ã„ã‚‰ã‚Œã¦ã„ã‚‹è‡ªç„¶è¨€èªžãƒ¢ãƒ‡ãƒ«ãŒã€ŒGPTã€ï¼ˆGenerative Pre-trained Transf ormerï¼‰ã§ã™ã€‚ã„ã‚ã‚†ã‚‹ã€Œå¤§è¦æ¨¡è¨€èªžãƒ¢ãƒ‡ãƒ«ã€ï¼ˆLLMï¼šLarge Language Modelï¼‰ã®1ã¤ã§ã‚ã‚Šã€æœ¬ç¨¿åŸ·ç†æ™‚ç‚¹ï¼ˆ2023å¹´12æœˆï¼‰ã®æœ€æ–°ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã¯ã€ŒGPT-4 Turboã€ã§ã™ã€‚ãƒãƒ¼ã‚¸ãƒ§ãƒ³ã‚’è¿½ã†ã”ã¨ã«ã€ç²¾åº¦å‘ä¸Šãªã©ã®é€²åŒ–ã‚’é‚ã’ã¦ãã¾ã—ãŸã€‚ ã“ã®GPTã®æ ¸ã¨ãªã‚‹ãƒ‡ã‚£ãƒ¼ãƒ—ãƒ©ãƒ¼ãƒ‹ãƒ³ã‚°ï¼ˆæ·±å±¤å¦ç¿’ï¼‰ã®ãƒ¢ãƒ‡ãƒ«ãŒã€ŒTransf ormerã€ã§ã™ã€‚å‰è¿°ã®GPTã®æ£å¼åç§°ã‚’è¦‹ç›´ã™ã¨ã€ã€ŒTã€ã¯Transf ormerã®Tã§ã‚ã‚‹ã“ã¨ãŒã‚ã‹ã‚Šã¾ã™ã€‚ã“ã®ãƒ
yuki_2021 2024/06/25
AI

LLM

æ©Ÿæ¢°å¦ç¿’
ãƒªãƒ³ã‚¯
RAGã§äººé–“ã®è„³ã‚’å†ç¾ã€‚ã€ŒHippoRAGã€ã‚’ç†è§£ã™ã‚‹
ã¯ã˜ã‚ã¾ã—ã¦ã€‚ãƒŠãƒ¬ãƒƒã‚¸ã‚»ãƒ³ã‚¹ã®é–€è„‡ã§ã™ã€‚ç”ŸæˆAIã‚„RAGã‚·ã‚¹ãƒ†ãƒ ã‚’æ´»ç”¨ã—ãŸã‚µãƒ¼ãƒ“ã‚¹ã‚’é–‹ç™ºã—ã¦ã„ã¾ã™ã€‚æœ¬è¨˜äº‹ã§ã¯ã€RAGã®æ€§èƒ½ã‚’é«˜ã‚ã‚‹æ‰‹æ³•ã§ã‚ã‚‹ã€ŒHippoRAGã€ã«ã¤ã„ã¦ã€ã–ã£ãã‚Šç†è§£ã—ã¾ã™ã€‚ ã“ã®è¨˜äº‹ã¯ä½• ã“ã®è¨˜äº‹ã¯ã€RAGã®æ–°æ‰‹æ³•ã¨ã—ã¦æœ€è¿‘æ³¨ç›®ã•ã‚Œã¦ã„ã‚‹ã€ŒHippoRAGã€ã®è«–æ–‡[1]ã«ã¤ã„ã¦ã€æ—¥æœ¬èªžã§ç°¡å˜ã«ã¾ã¨ã‚ãŸã‚‚ã®ã§ã™ã€‚ ã€Œãã‚‚ãã‚‚RAGã¨ã¯ï¼Ÿã€ã«ã¤ã„ã¦ã¯ã€çŸ¥ã£ã¦ã„ã‚‹å‰æã§é€²ã¿ã¾ã™ã€‚ç¢ºèªã™ã‚‹å ´åˆã¯ä»¥ä¸‹ã®è¨˜äº‹ã‚‚ã”å‚è€ƒä¸‹ã•ã„ã€‚ æœ¬é¡Œ ã–ã£ãã‚Šã‚µãƒžãƒªãƒ¼ HippoRAGã¯ã€RAGã®æ€§èƒ½ã‚’é«˜ã‚ã‚‹ãŸã‚ã®æ–°ã—ã„æ‰‹æ³•ã§ã™ã€‚ã‚ªãƒã‚¤ã‚ªå·žç«‹å¤§å¦ã®ç ”ç©¶è€…ã‚‰ã«ã‚ˆã£ã¦2024å¹´5æœˆã«ææ¡ˆã•ã‚Œã¾ã—ãŸã€‚HippoRAGã‚’ä½¿ã†ãƒ¡ãƒªãƒƒãƒˆã¯ã€è¤‡æ•°ã®çŸ¥è˜ã‚’çµ„ã¿åˆã‚ã›ã¦å›žç”ã™ã‚‹å¿…è¦ãŒã‚ã‚‹ã‚ˆã†ãªã€è¤‡é›‘ãªè³ªå•ã«å¼·ããªã‚‹ã“ã¨ã§ã™ã€‚HippoRAGãŒå¾“æ¥ã®RAGã«æ¯”ã¹ã¦ã€è¤‡é›‘ãªè³ªå•ã«å¼·ã„ç†ç”±ã¯ã€ãƒŠãƒ¬ãƒƒã‚¸ã‚°ãƒ©ãƒ•ã¨ã€ãã‚Œã‚’ç¶™ç¶šçš„ã«
yuki_2021 2024/06/17
LLM

è„³

æ©Ÿæ¢°å¦ç¿’

AI
ãƒªãƒ³ã‚¯
è‡ªç„¶è¨€èªžã¨Vision&Language
æ±äº¬å¤§å¦å¤§å¦é™¢ã®2024å¹´åº¦è¬›ç¾©ã€ŒçŸ¥èƒ½æƒ…å ±è«–ã€ã§ä½¿ç”¨ã—ãŸè³‡æ–™ã§ã™ï¼Ž Vision&Languageé–¢é€£ã®ç ”ç©¶ã«ã¤ã„ã¦ï¼Œæ·±å±¤å¦ç¿’åˆæœŸã‹ã‚‰å¤§è¦æ¨¡ãƒ¢ãƒ‡ãƒ«ã«ã„ãŸã‚‹ã¾ã§ã‚’æ¦‚è¦³ã—ã¦ã„ã¾ã™ï¼Ž ãªãŠï¼Œè³‡æ–™ä½œæˆæ™‚æœŸã¯2024å¹´5æœˆä¸‹æ—¬ã§ã‚ã‚Šï¼Œå†…å®¹ã¯ãã®æ™‚ç‚¹ã§ç™ºè¡¨ã•ã‚Œã¦ã„ãŸç ”ç©¶ç‰ã«åŸºã¥ã„ã¦ã„ã¾ã™ï¼Ž
yuki_2021 2024/06/07
è‡ªç„¶è¨€èªžå‡¦ç†

æ©Ÿæ¢°å¦ç¿’

ç”»åƒ

LLM
ãƒªãƒ³ã‚¯
æœ€è¿‘ãƒãƒ¼ã‚«ãƒ«LLMãŒã‚¢ãƒ„ã„ã‚‰ã—ã„
æœ€è¿‘ã€ãƒãƒ¼ã‚«ãƒ«LLMãŒã‚¢ãƒ„ããªã£ã¦ã„ã‚‹ã¨ã„ã†è©±ã‚’twitterã§ãƒãƒ©ãƒ›ãƒ©è¦‹ã‹ã‘ã‚‹ã€‚ ãƒãƒ¼ã‚«ãƒ«LLMã£ã¦ä½•ã˜ã‚ƒï¼Ÿã¨ã„ã†ã¨ã€ã‚ªãƒ¼ãƒ—ãƒ³ã«å…¬é–‹ã•ã‚Œã¦ã„ã‚‹ãƒ¢ãƒ‡ãƒ«ã®ã‚¦ã‚¨ã‚¤ãƒˆã‚’DLã—ã¦ãã¦æ‰‹å…ƒã®PCä¸Šã§æŽ¨è«–ã•ã›ã‚‹äº‹ã§ã‚ã‚‹ã€‚ ã‚ªãƒ¼ãƒ—ãƒ³ãªAIãƒ¢ãƒ‡ãƒ«ã¨ã¯é€†ã®å˜åœ¨ã¨ã—ã¦ã€ãƒ¢ãƒ‡ãƒ«ãŒDLã§ããªã„ã‚¯ãƒãƒ¼ã‚ºãªAIãƒ¢ãƒ‡ãƒ«ã‚‚ã‚ã‚‹ã€‚ OpenAIã‚„Anthropicã®ã‚ˆã†ãªæœ€å…ˆç«¯AIä¼æ¥ã®ã‚¯ãƒãƒ¼ã‚ºãªAIãƒ¢ãƒ‡ãƒ«ã«æ¯”ã¹ã¦ã€ã‚ªãƒ¼ãƒ—ãƒ³ã«å…¬é–‹ã•ã‚Œã¦ã„ã‚‹ã‚ªãƒ¼ãƒ—ãƒ³ãªAIãƒ¢ãƒ‡ãƒ«ã®æ€§èƒ½ã¯ä»Šã§ã‚‚ã‹ãªã‚Šå¾Œã‚Œã‚’å–ã£ã¦ã„ã‚‹ã€‚ ã ã‹ã‚‰åŽ»å¹´ã®é–“ã¯ã‚ãã¾ã§AIã®æœ¬å‘½ã¯ChatGPTã®ã‚ˆã†ãªã‚¯ãƒãƒ¼ã‚ºãƒ¢ãƒ‡ãƒ«ã§ã‚ã£ã¦ã€ã‚ªãƒ¼ãƒ—ãƒ³ãªAIãƒ¢ãƒ‡ãƒ«ãªã‚“ã¦çœ¼ä¸ã«ã‚ã‚Šã¾ã›ã‚“ã¿ãŸã„ãªé¢¨æ½®ãŒç„¡ãã¯ãªã‹ã£ãŸã€‚ã ãŒæœ€è¿‘ã¯é¢¨å‘ããŒå°‘ã€…å¤‰ã‚ã£ã¦ãã¦ã„ã‚‹ã€‚ GPTã®APIé«˜ã„å•é¡Œ ï¼† OpenAIãŒAIãƒ™ãƒ³ãƒãƒ£ãƒ¼çš†æ®ºã—ã«ã—ã¦ã—ã¾ã†å•é¡Œ ã¾ãšã€Œçµå±€GPTã®APIã‚’å©ã„ã¦ã‚µãƒ¼ãƒ“ã‚¹é‹å–¶ã—ã¦æˆåŠŸã—
yuki_2021 2024/05/21
LLM

AI

æ©Ÿæ¢°å¦ç¿’
ãƒªãƒ³ã‚¯
é©šç•°ã®1ãƒ“ãƒƒãƒˆLLMã‚’è©¦ã™ã€‚æžœãŸã—ã¦æœ¬å½“ã«å¦ç¿’ã§ãã‚‹ã®ã‹?ï½œshi3z
æ˜¨æ—¥è©±é¡Œã«ãªã£ãŸã€ŒBit Netã€ã¨ã„ã†1ãƒ“ãƒƒãƒˆã§æŽ¨è«–ã™ã‚‹LLMãŒã©ã†ã—ã¦ã‚‚è©¦ã—ãŸããªã£ãŸã®ã§æ—©é€Ÿè©¦ã—ã¦ã¿ãŸã€‚ Bit Netã¨ã„ã†ã®ã¯ã€1ãƒ“ãƒƒãƒˆ(-1,0,1ã®ä¸‰çŠ¶æ…‹ã‚’æŒã¤)ã¾ã§æƒ…å ±ã‚’å‰ŠãŽè½ã¨ã™ã“ã¨ã§é«˜é€Ÿã«æŽ¨è«–ã™ã‚‹ã¨ã„ã†ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã€‚ã ã‹ã‚‰æ£ç¢ºã«ã¯0ã‹1ã‹ã§ã¯ãªãã€-1ã‹0ã‹1ã¨ã„ã†ã“ã¨ã€‚ ã“ã®æ‰‹æ³•ã®è¡Œãç€ãã¨ã“ã‚ã¯ã€GPUãŒä¸è¦ã«ãªã‚Šæ–°ã—ã„ãƒãƒ¼ãƒ‰ã‚¦ã‚§ã‚¢ãŒå‡ºç¾ã™ã‚‹ä¸–ç•Œã§ã‚ã‚‹ã¨äºˆè¨€ã•ã‚Œã¦ã„ã‚‹ã€‚ãƒžã‚¸ã‹ã‚ˆã€‚ https://arxiv.org/pdf/2402.17764.pdf ã¨ã„ã†ã“ã¨ã§æ—©é€Ÿè©¦ã—ã¦ã¿ã‚‹ã“ã¨ã«ã—ãŸã€‚ ã‚ªãƒ•ã‚£ã‚·ãƒ£ãƒ«ã®å®Ÿè£…ã¯å…¬é–‹ã•ã‚Œã¦ã„ãªã„ãŒã€ãã‚‚ãã‚‚1ãƒ“ãƒƒãƒˆ(ã¨è¨€ã£ã¦ã„ã„ã®ã‹ã‚ã‹ã‚‰ãªã„,-1,0,1ã®ä¸‰çŠ¶æ…‹ã‚’æŒã¤ã‹ã‚‰ã ã€‚ è«–æ–‡è‘—è€…ã¯log2(3)ã§1.58ãƒ“ãƒƒãƒˆã¨ã„ã†ä¸»å¼µã‚’ã—ã¦ã„ã‚‹)é‡ååŒ–ã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ è‡ªä½“ã®ç ”ç©¶ã®æ´å²ã¯å¤ã„ã®ã§ã€Bit Netã«ã‚ˆã‚‹Transf ormerã®é‡Žè‰¯å®Ÿè£…
yuki_2021 2024/02/29
LLM

æ©Ÿæ¢°å¦ç¿’

AI
ãƒªãƒ³ã‚¯
RAGã®æ€§èƒ½ã‚’æ”¹å–„ã™ã‚‹ãŸã‚ã®8ã¤ã®æˆ¦ç•¥ | Fintan
è¿‘å¹´ã€OpenAIã®GPT-4ã‚„Googleã®Geminiã€Metaã®LLaMAã‚’ã¯ã˜ã‚ã¨ã™ã‚‹å¤§è¦æ¨¡è¨€èªžãƒ¢ãƒ‡ãƒ«ï¼ˆLarge Language Modelï¼šLLMï¼‰ã®èƒ½åŠ›ãŒå¤§å¹…ã«å‘ä¸Šã—ã€è‡ªç„¶è¨€èªžå‡¦ç†ã«ãŠã„ã¦å„ªã‚ŒãŸçµæžœã‚’åŽã‚ã¦ã„ã¾ã™[1][2][3]ã€‚ã“ã‚Œã‚‰ã®LLMã¯ã€è†¨å¤§ãªé‡ã®ãƒ†ã‚ã‚¹ãƒˆãƒ‡ãƒ¼ã‚¿ã§å¦ç¿’ã•ã‚Œã¦ãŠã‚Šã€ã•ã¾ã–ã¾ãªè‡ªç„¶è¨€èªžå‡¦ç†ã‚¿ã‚¹ã‚¯ã«ãŠã„ã¦ã€ã‚¿ã‚¹ã‚¯ã«å›ºæœ‰ãªãƒ‡ãƒ¼ã‚¿ã‚’ç”¨ã„ã¦ãƒ¢ãƒ‡ãƒ«ã‚’ãƒ•ã‚¡ã‚¤ãƒ³ãƒãƒ¥ãƒ¼ãƒ‹ãƒ³ã‚°ã™ã‚‹ã“ã¨ãªãã€ã‚ˆã‚Šæ£ç¢ºã§è‡ªç„¶ãªãƒ†ã‚ã‚¹ãƒˆç”Ÿæˆã‚„ã€è¤‡é›‘ãªè³ªå•ã¸ã®å›žç”ãŒå¯èƒ½ã¨ãªã£ã¦ã„ã¾ã™ã€‚ LLM-jp-eval[4]ãŠã‚ˆã³MT-bench-jp[5]ã‚’ç”¨ã„ãŸæ—¥æœ¬èªžLLMã®è©•ä¾¡çµæžœã€‚Nejumi LLMãƒªãƒ¼ãƒ€ãƒ¼ãƒœãƒ¼ãƒ‰ Neoã‚ˆã‚Šå–å¾—ã€‚ å¤§è¦æ¨¡è¨€èªžãƒ¢ãƒ‡ãƒ«ã¯è¿‘å¹´æ€¥é€Ÿãªé€²æ©ã‚’é‚ã’ã¦ã„ã¾ã™ãŒã€ã“ã‚Œã‚‰ã®é€²æ©ã«ã‚‚ã‹ã‹ã‚ã‚‰ãšã€è£ä»˜ã‘ã®ãªã„æƒ…å ±ã‚„çŸ›ç›¾ã—ãŸå†…å®¹ã‚’ç”Ÿæˆã™ã‚‹ç‚¹ã«ãŠã„ã¦ã¯ä¾ç„¶ã¨ã—ã¦èª²é¡ŒãŒã‚ã‚Šã¾ã™ã€‚ãŸã¨ãˆ
yuki_2021 2024/02/10
LLM

æ©Ÿæ¢°å¦ç¿’
ãƒªãƒ³ã‚¯
1