Sakana AI
This is the second in a series of blog posts presenting the results of Sakana AI’s research projects that were supported by the Japanese Ministry of Economy, Trade and Industry’s GENIAC supercomputing grant. Summary At Sakana AI, we draw inspiration from nature to extend the capabilities of foundation models. This was evidenced in our recent work on improving performance with evolutionary model me
Transformers (how LLMs work) explained visually | DL5
Breaking down how Large Language Models work Instead of sponsored ad reads, these lessons are funded directly by viewers: https://3b1b.co/support --- Here are a few other relevant resources Build a GPT from scratch, by Andrej Karpathy https://youtu.be/kCc8FmEb1nY If you want a conceptual understanding of language models from the ground up, @vcubingx just started a short series of videos on the
Conformer：Transformerを音声èªè˜ã«å¿œç”¨ï¼ï¼Ÿ Googleã«ã‚ˆã‚‹Transformer×CNNãŒå‡„ã™ãŽã‚‹ï¼ï¼
3ã¤ã®è¦ç‚¹ âœ”ï¸ Transformerã¨CNNを組ã¿åˆã‚ã›ãŸãƒ¢ãƒ‡ãƒ«ï¼ŒConformerを音声èªè˜ã«å¿œç”¨ âœ”ï¸ ç•³ã¿è¾¼ã¿ãƒ¢ã‚¸ãƒ¥ãƒ¼ãƒ«ãŒConformerã«ãŠã„ã¦æœ€ã‚‚é‡è¦ã§ã‚ã‚‹ã“ã¨ãŒã‚ã‹ã£ãŸ âœ”ï¸ æ—¢å˜ã®éŸ³å£°èªè˜ç ”究ã®ä¸ã§ã‚‚最高ã®ç²¾åº¦ã‚’ç¢ºèª Conformer: Convolution-augmented Transformer for Speech Recognition written by Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, Ruoming Pang (Submitted on 16 May 2020) Comments: Accepted at Interspeech20
作ã£ã¦ç†è§£ã™ã‚‹ Transformer / Attention - Qiita
ã“ã‚“ã«ã¡ã¯ã€‚ミクシィ AI ãƒãƒœãƒƒãƒˆäº‹æ¥éƒ¨ã§ã—ゃã¹ã‚‹ãƒãƒœãƒƒãƒˆã‚’作ã£ã¦ã„るインコã§ã™ã€‚ ã“ã®è¨˜äº‹ã¯ ミクシィグループ Advent Calendar 2018 ã®5日目ã®è¨˜äº‹ã§ã™ã€‚ ã“ã®è¨˜äº‹ã®ç›®çš„ ã“ã®è¨˜äº‹ã§ã¯2018å¹´ç¾åœ¨ DeepLearning ã«ãŠã‘る自然言語処ç†ã®ãƒ‡ãƒ•ã‚¡ã‚¯ãƒˆã‚¹ã‚¿ãƒ³ãƒ€ãƒ¼ãƒ‰ã¨ãªã‚Šã¤ã¤ã‚ã‚‹ Transformer を作るã“ã¨ã§ã€ Attention ベースã®ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã‚’ç†è§£ã™ã‚‹ã“ã¨ã‚’目的ã¨ã—ã¾ã™ã€‚ 機械翻訳ãªã©ã® Transformer, 自然言語ç†è§£ã® BERT ã‚„ãã®ä»–多ãã®ç¾åœ¨ SoTA ã¨ãªã£ã¦ã„る自然言語処ç†ã®ãƒ¢ãƒ‡ãƒ«ã¯ Attention ベースã®ãƒ¢ãƒ‡ãƒ«ã§ã™ã€‚ Attention ã‚’ç†è§£ã™ã‚‹ã“ã¨ãŒä»Šå¾Œã®è‡ªç„¶è¨€èªžå‡¦ç† x Deep Learning ã®å¿…é ˆã«ãªã£ã¦ãã‚‹ã®ã§ã¯ãªã„ã§ã—ょã†ã‹ã€‚ ã“ã®è¨˜äº‹ã«ã‹ã‹ã‚Œã¦ã„るコード (Github) æ´å² - RNN ã‹ã‚‰
1
ランã‚ング
ランã‚ング
障害
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
{{#tags}}- {{label}}
{{/tags}}