令å4å¹´ã®ä¸å¹´éã§èµ·ãã£ããå ¨äºæ ããã¤ã¾ã軽å·ã»éå·ã»æ»äº¡ã®ãã¹ã¦ãå«ã交éäºæ ã®çºç件æ°ã¯300,839件ããã®ãã¡æ»äº¡äºæ ã¯2,550件ã§ãï¼è¦å¯åºèª¿ã¹ï¼ããããã¯ãã©ããªç¶æ³ã§çºçãã¦ããã®ã§ãããããã¾ããç¹ã«é大ãªäºæ ã«ã¤ãªãããããã®ã¯ã©ã®ãããªã·ãã¥ã¨ã¼ã·ã§ã³ãªã®ã§ããããã 交éäºæ ã»æ»äº¡äºæ ã®ãã¼ã¿ãèªã¿è§£ããã¨ã§ãé大ãªäºæ ãåé¿ããä¸ã§æ³¨æãã¹ããã¤ã³ããè¦ãã¦ããããããã¾ãããå½å ã§çºçãã交éäºæ ã®æ å ±ãèç©ããç·åçãªèª¿æ»åæãè¡ãã交éäºæ ç·ååæã»ã³ã¿ã¼ãã®ç°ä¹ ä¿å®£æããã«ããã¾ãã¾ãªåæãã¼ã¿ãç´¹ä»ãã¦ãããã¤ã¤ãè¦è§£ã伺ãã¾ãããããã«ã¯ããã³ãã«ãæ¡ã人ããäºæ ãèµ·ããããå®å ¨ã«è»ãé転ããããã®ããã¾ãã¾ãªç¤ºåãããã¾ããã ãã話ãèãã人ã ç°ä¹ ä¿å®£æï¼ããã¼ã»ã®ã¶ããï¼ããï¼å·¥å¦å士ã交éäºæ ç·ååæã»ã³ã¿ã¼ ç 究é¨æ¬¡é·å ¼ç 究第ä¸èª²é·ãè¦å¯
ãæ°ããåæã«ãªãããä¸çã«PRãã¦ãããããã25æ¥ã®ãªã¼ããã³ã°ã¤ãã³ãã§ãå°æ± ç¾ååç¥äºã¯ããè¿°ã¹ãã48é建ã¦ãé«ãç´240ã¡ã¼ãã«ã®ç¬¬1æ¬åºèã®å£é¢ã«ãæ±äº¬ã®è¦³å ã¹ããããªã©ãæ¯æ¥åå¾7æãã30åããã«æå½±ããã é½ã¯ä»ã«ãé½è°ä¼è°äºå ãæ§å°ç°æ¥ç¾è²¨åºãªã©ã§ããã¸ã§ã¯ã·ã§ã³ãããã³ã°ãå±éãããããåãããæ¬å¹´åº¦ã®ä¸è¬ä¼è¨äºç®ããè¨ç´18ååãæ¯åºãã¦ããã24年度äºç®æ¡ã«ãåºèå9å5000ä¸åããã®ä»ç´10ååãè¨ä¸ããæ±äº¬ã®å¤ã«æ°ããªâ楽ããâã¨âè³ï¼ã«ãï¼ããâãããããéè¦ãªãã¼ã«ããæ±äº¬ã®ãã©ã¼ã³ã³ãã³ãã¨ãããã¨ã§ãå¤é観å ã®çãä¸ããããããã
åã¯çµ±å失調çã®è¶£å³äººãã¼ãã§ããããã°ã©ãã³ã°ã趣å³ã¨ãã¦ãã£ã¦ãã¾ãã 競æããã°ã©ãã³ã°ã®åå¼·ã¯å¤§å¤ã§ãããatcoderã§ã¬ããã³ã¼ãã¼ã«ãªãã¾ã§ã®ä¸é£ã®åå¼·ã®æ¹æ³ã«ã¤ãã¦æ¸ããã¨æãã¾ãã å®ã¯ç«¶æããã°ã©ãã³ã°ãå§ããåã¾ã§ã¯ãçµ±å失調çã§ã¯ããã¾ããã§ãããåãªããã¼ãã§ãã åã¯çç£çãªè¶£å³ããããã§æ¢ããDIYã¯ä½åã使ãããå«ã ããæçã¯å¤ªãããå«ã ã¨ããçç±ã§ã競ããã«ãã©ãçããã®ã§ãã æåã¯githubã§å ¬éããã¦ããç·´ç¿åé¡ã®ãªã¹ããå復ãã¦è¡ãã¾ããã ããã¦ã³ã³ãã¹ãã®æ¥ã«ãªã£ã¦ãå®éã«ãã£ã¦ã¿ããï¼åãã解ãããçµæã¯ã²ã©ããã®ã§ããã ãããåé¡èªä½ã¯ãç·´ç¿åé¡ããã§ããã¬ãã«ã§ããã°è§£ãããã®ã ã£ãã®ã§ãã ããã§åã¯ãå¿ç¨åã足ããªããã®ã ã¨ä»®èª¬ãç«ã¦ãå¿ç¨åã®ããã«ã¯è¦ããã¨ã®ãªãåé¡ã«åãçµãå¿ è¦ãããã¨èãã¾ããã ã¤ã³ã¿ã¼ãããä¸ã§æ¢ããã¨
絵師ãã¿ãããªæãã§ããè¯ããããã ãããããããã¾ãã
Microsoftã®ç 究ãã¼ã ãã¢ãã«ã®ã¦ã§ã¤ããã-1ãã0ãã1ãã®3ã¤ã®å¤ã®ã¿ã«ãããã¨ã§å¤§è¦æ¨¡è¨èªã¢ãã«ã®è¨ç®ã³ã¹ããæ¿æ¸ããããã¨ã«æåããã¨çºè¡¨ãã¾ããã [2402.17764] The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits https://arxiv.org/abs/2402.17764 å¾æ¥ã®ã¢ãã«ã§ããã°å ¥åã«å¯¾ãã¦ã0.2961ããªã©ã®ã¦ã§ã¤ããããç®ãã¦ãã足ãå¼ãããå¿ è¦ãããã¾ããããã-1ãã0ãã1ãã®3å¤ã®ã¿ã§ããã°ããç®ãä¸è¦ã«ãªããå ¨ã¦ã®è¨ç®ã足ãç®ã§è¡ããããã«ãªãã¾ãã ãã®ãããåãæ§è½ãåºãã®ã«å¿ è¦ãªã³ã¹ããé常ã®å¤§è¦æ¨¡è¨èªã¢ãã«ã«å¯¾ãã¦æ¿æ¸ããã¨ã®ãã¨ããªããããããã®ãã©ã¡ã¼ã¿ã¼ãã-1ãã0ãã1ãã¨ãã3ã¤ã®å¤ãåãããlog[2](3)ã®å¤ããã1.
1987å¹´æ±äº¬åºèº«ãä¼ç¤¾å¡ããã³ãã¼ã°ãã«ã¬ã¼ãããã³ã©ã¤ã¹ãªã©ã好ç©ãªã®ã§ãèãåä¾ãããããªããã¨å¿é ã«ãªãã¨ãããããã ãã³ã¼ãã¼ã¯ãã©ãã¯ã§ã飲ãããåç»ã¤ã³ã¿ãã¥ã¼ åã®è¨äºï¼å¯ãæ¸ããæ¡å¼µãã ãã©ãã¯ã¬ã¿ã¼ã¨ã¯ ãã©ãã¯ã¬ã¿ã¼ããããããã¤ã§ããã ãªã¼ã«ãã¤ã³ã°ãªãã·ã¥ã¨ãã´ã·ãã¯ãªã©ã¨ãå¼ã°ããæ¸ä½ã§ãæ°èãã¯ã¤ã³ã®ã©ãã«ã§ä½¿ããã¦ããããã¥ã¼ã¨ã¼ã¯ã»ã¿ã¤ã ãºã®ãã´ã¨ãã ãã®æ¸ä½ã§æ¸ãããã¨é»ãé¨åãå¤ããªãã®ã§ãã©ãã¯ã¬ã¿ã¼ã¨ãããããã èµ·æºã12ä¸ç´ã«ã¾ã§é¡ãæ´å²ã®ããæ¸ä½ã§ãä½ãç¥ããªãã¦ããããè¦ãã¨é°å²æ°ãããªãã¨ãããä¸çã®ç§å¯ãæ¸ãã¦ããããã ã å人çã«ã¯ãã³ã¬ãDEATH NOTEãã§ããã©ãã¯ã¬ã¿ã¼ã®ãLãã£ã¦ããæåãããè¦ã¦ãããã¹ãªãªã³ã°ãªé°å²æ°ãå¼ãç«ã¦ã¦ããã¨æãã
æååºã¯ãä»å¾ãAIã¨èä½æ¨©ã«é¢ããæ³è§£éã«ã¤ãã¦ãä»å¾ãåãããããå½¢ã§åçºãã¦ããã¨èª¬æããå¿ è¦ã«å¿ãã¦ãèä½è äººæ ¼æ¨©ãèä½é£æ¥æ¨©ã¨AIã¨ã®é¢ä¿ï¼ä¿³åªã»å£°åªãªã©ã®å£°ãå«ãã å®æ¼ã»ã¬ã³ã¼ããªã©ã®å©ç¨ã¨AIã¨ã®é¢ä¿ãå«ãï¼ã«ããã¦ãæ¤è¨ãã¹ãç¹ã®æç¡ãå 容ã®æ¤è¨ãå«ãããèãæ¹ãã®è¦ç´ããªã©å¿ è¦ãªæ¤è¨ãè¡ããï¼ååºï¼ é¢é£è¨äº ãçæAIã¯èä½æ¨©ä¿è·ã®æ¤è¨ãä¸ååãæ°èåä¼ãªã©å£°æããèä½æ¨©æ³30æ¡ã®4ã¯å¤§ããªèª²é¡ã æ°èåä¼ãªã©4å£ä½ã¯ãçæAIã«ã¾ã¤ããèä½æ¨©ä¿è·çã®åæ¤è¨ãæ±ããå ±å声æãçºè¡¨ãããæ¥æ¬ã®èä½æ¨©æ³ç¬¬30æ¡ã®4ãã諸å¤å½ã«æ¯ã¹ãAIå¦ç¿ã«æ¥µãã¦æå©ã«ä½ããã¦ãããã¨ã¯å¤§ããªèª²é¡ãã¨ææãã¦ããã ã¯ãªã¹ã¿ã«ç»åçæAIãæè¼ããäºå®ãªãââæä¾å ã»ã«ã·ã¹ãçºè¡¨ãããã¼ã¿ã»ãããã¯ãªã¼ã³ãªãã®ãã使ããªãã ã¯ãªã¹ã¿ã«ç»åçæAIãæè¼ããäºå®ã¯ãªãââã¤ã©ã¹ãä½æã½ãã
æ¨æ¥è©±é¡ã«ãªã£ããBitNetãã¨ãã1ãããã§æ¨è«ããLLMãã©ããã¦ã試ããããªã£ãã®ã§æ©é試ãã¦ã¿ãã BitNetã¨ããã®ã¯ã1ããã(-1,0,1ã®ä¸ç¶æ ãæã¤)ã¾ã§æ å ±ãåãè½ã¨ããã¨ã§é«éã«æ¨è«ããã¨ããã¢ã«ã´ãªãºã ãã ããæ£ç¢ºã«ã¯0ã1ãã§ã¯ãªãã-1ã0ã1ã¨ãããã¨ã ãã®ææ³ã®è¡ãçãã¨ããã¯ãGPUãä¸è¦ã«ãªãæ°ãããã¼ãã¦ã§ã¢ãåºç¾ããä¸çã§ããã¨äºè¨ããã¦ããããã¸ããã https://arxiv.org/pdf/2402.17764.pdf ã¨ãããã¨ã§æ©é試ãã¦ã¿ããã¨ã«ããã ãªãã£ã·ã£ã«ã®å®è£ ã¯å ¬éããã¦ããªããããããã1ããã(ã¨è¨ã£ã¦ããã®ãããããªã,-1,0,1ã®ä¸ç¶æ ãæã¤ããã ã è«æèè ã¯log2(3)ã§1.58ãããã¨ãã主張ããã¦ãã)éååã®ã¢ã«ã´ãªãºã èªä½ã®ç 究ã®æ´å²ã¯å¤ãã®ã§ãBitNetã«ããTransformerã®éè¯å®è£
ã©ã³ãã³ã°
ã©ã³ãã³ã°
ã©ã³ãã³ã°
ãªãªã¼ã¹ãé害æ å ±ãªã©ã®ãµã¼ãã¹ã®ãç¥ãã
ææ°ã®äººæ°ã¨ã³ããªã¼ã®é ä¿¡
j次ã®ããã¯ãã¼ã¯
kåã®ããã¯ãã¼ã¯
lãã¨ã§èªã
eã³ã¡ã³ãä¸è¦§ãéã
oãã¼ã¸ãéã
{{#tags}}- {{label}}
{{/tags}}