Seq2Seq+Attentionã®ãã®å…ˆã¸ - Qiita
自然言語処ç†ã«ãŠã„ã¦Sequence-to-Sequenceモデルã€ãã—ã¦Attentionã¯å¤§ããªå½±éŸ¿ã‚’与ãˆã¦ãã¾ã—ãŸã€‚ ã„ã¾ã‚„Sequence-to-Sequence + Attentionモデルã¯è‡ªç„¶è¨€èªžå‡¦ç†ã¨ãƒ‡ã‚£ãƒ¼ãƒ—ラーニングを語る上ã§ã¯æ¬ ã‹ã›ãªã„å˜åœ¨ã¨ãªã‚Šã¤ã¤ã‚ã‚Šã¾ã™ã€‚ è¿‘å¹´ã®è‡ªç„¶è¨€èªžå‡¦ç†ã§ã¯ã“ã®Sequence-to-Sequenceã¨Attentionをベースã«ã—ãŸãƒ¢ãƒ‡ãƒ«ãŒå¤šãæ案ã•ã‚Œã¦ã„ã¾ã™ã€‚ ã“ã®è¨˜äº‹ã§ã¯Sequence-to-Sequenceをベースã¨ã—ãŸãƒ¢ãƒ‡ãƒ«ãŒã©ã†ã„ã£ãŸé€²åŒ–ã‚’é‚ã’ã¦ã„ã‚‹ã‹ã‚’æ´å²ã‚’追ã„ãªãŒã‚‰ã¾ã¨ã‚ã¦ã„ã“ã†ã¨æ€ã„ã¾ã™ã€‚ Sequence-to-Sequenceモデル (2014) Sequence-to-Sequenceモデルã¯Sequence to Sequence Learning with Neural Networksã®è«–æ–‡ã§æ案ã•ã‚Œã€ã€ŒSeq2
Seq2Seqメモ - chainerã§è‡ªç„¶è¨€èªžå‡¦ç†ã§ãã‚‹ã‹ãƒžãƒ³
Sequence-to-Sequence(Seq2Seq)å¦ç¿’ã¯ã€ä»»æ„é•·ã®å…¥åŠ›åˆ—ã‹ã‚‰ä»»æ„é•·ã®å‡ºåŠ›åˆ—を出力ã™ã‚‹ã‚ˆã†ãªå¦ç¿’ã®ã“ã¨ã§ã€Neural Networkã®æž 組ã¿ã§æ‰±ã†æ–¹æ³•ãŒæ案ã•ã‚Œã¦ã€ã„ã„çµæžœãŒå ±å‘Šã•ã‚Œã¦ã„ã¾ã™ã€‚雑ãªãƒ¡ãƒ¢ã€‚ 入力・出力列ã®ä¾‹ (自然)言語処ç†ç³» 機械翻訳(翻訳元->翻訳先) [1409.3215] Sequence to Sequence Learning with Neural Networks [1406.1078] Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation http://www.slideshare.net/yutakikuchi927/learning-phrase-representations-using-rnn-
seq2seq-attnニューラル機械翻訳を試㙠- Qiita
@tkngueã¨å…±è‘— TL;DR NMTを試ã—ã¦ã¿ã¾ã—ãŸ. 翻訳çµæžœã‚’見るã¨ã€ã†ã¾ãã„ã£ã¦ã„る翻訳ã¯é©šãã®çµæžœã§ã—ãŸ. ãŸã ,内容語をã¨ã‚“ã§ã‚‚ãªã„é–“é•ã„ã‹ãŸã‚’ã—ã¦ã„ã‚‹ã‚‚ã®ã‚‚ã‚ã£ãŸã‚Šã—ã¦é¢ç™½ã‹ã£ãŸã§ã™. æ¦‚è¦ __"ã§ãƒã€œã·ã‚‰ãƒ¼ã«ã‚“ã"__ã®æ³¢ãŒæ©Ÿæ¢°ç¿»è¨³ã«ã‚‚æ¥ã¦ã„㦠大ããªãƒ‘ラダイムシフトã«ãªã‚Šã¤ã¤ã‚ã‚Šã¾ã™. ãã®åœ§å€’çš„ãªè¡¨ç¾èƒ½åŠ›ã®é«˜ã•ã€€ã‚„　高価ãªGPGPUãŒå¤§é‡ã«ãªã„ã¨è©±ã«ãªã‚‰ãªã„ãŠé‡‘æŒã¡ã‚²ãƒ¼ã§ã‚ã‚‹ã“㨠や 膨大ãªãƒã‚¤ãƒ‘パラメータ（å¦ç¿’ã®éš›ã®è¨å®šã®çµ„ã¿åˆã‚ã›ï¼‰ãŒ å˜åœ¨ã™ã‚‹ã“ã¨ã‹ã‚‰ 「ã•ã‚ã‚‹ã®å«Œã ãªãƒ¼ã€ã¨æ€ã„ã¤ã¤ã‚‚ã€ã•ã‚らã–ã‚‹ã‚’å¾—ãªã„状æ³ã«ãªã£ãŸã®ã§è§¦ã£ã¦ã¿ã¾ã—ãŸ. 実際ã«ã•ã‚ã£ã¦ã¿ã¦, ã‚„ã£ã±ã‚Šå‡„ã‹ã£ãŸã®ã§ 投稿ã™ã‚‹æ¬¡ç¬¬ã§ã™. 本稿ã§ã¯, å†é…布å¯èƒ½ãªå¯¾è¨³ã‚³ãƒ¼ãƒ‘スKFTTを対象ã«æ—¥è‹±ç¿»è¨³ã‚’ニューラルãƒãƒƒãƒˆç¿»è¨³(NMT)ã§è©¦ã—ã¦ã¿ã¾ã™. åŠ ãˆã¦, ã“ã‚Œã¾ã§ã®ãƒ•ãƒ¬ãƒ¼ã‚ºãƒ™ãƒ¼ã‚¹ã®çµ±è¨ˆçš„機械翻訳ã®å…¸
TensorFlowサイトã®ãƒãƒ¥ãƒ¼ãƒˆãƒªã‚¢ãƒ«ã€ŒSequence-to-Sequence Modelsã€ã‚’翻訳ã—ã¦ã¿ãŸ
上図ã®å››è§’ã¯RNNセルを表ã—ã¦ã„ã¾ã™ã€‚ã¾ãŸã“れらã®å››è§’ã¯ã€æœ€ã‚‚一般的ã«ã¯GPUセルもã—ãã¯LSTMセル(説明㯠RNNãƒãƒ¥ãƒ¼ãƒˆãƒªã‚¢ãƒ« ã‚’å‚ç…§ã®ã“ã¨)を表ã—ã¦ã„ã¾ã™ã€‚エンコーダãŠã‚ˆã³ãƒ‡ã‚³ãƒ¼ãƒ€ã¯é‡ã¿ã‚’共有ã™ã‚‹ã‹ã‚‚ã—ãã¯ã€ã‚ˆã‚Šä¸€èˆ¬çš„ã«ã¯ã€ç•°ãªã‚‹ãƒ‘ラメータセットを使用ã—ã¾ã™ã€‚多層セル㌠sequence-to-sequence モデルã«ãŠã„ã¦ã‚‚ã†ã¾ã使ã‚ã‚Œã¦ã„ã¾ã™ã€‚例： 翻訳å‘ã‘ Sutskeverã»ã‹ 2014å¹´ ( PDF ) ã“ã“ã§è¡¨ã•ã‚Œã¦ã„る基本的ãªãƒ¢ãƒ‡ãƒ«ã«ãŠã„ã¦ã€ã‚らゆるインプットã¯ã€ãƒ‡ã‚³ãƒ¼ãƒ€ã«å¼•ã継ãŒã‚Œã‚‹å”¯ä¸€ã®ã‚‚ã®ã§ã‚ã‚‹ã€å›ºå®šã‚µã‚¤ã‚ºã®ãƒ™ã‚¯ãƒˆãƒ«ã«ã‚¨ãƒ³ã‚³ãƒ¼ãƒ‰ ã•ã‚Œãªã‘ã‚Œã°ãªã‚Šã¾ã›ã‚“。インプットã¸ã®ç›´æŽ¥ã‚¢ã‚¯ã‚»ã‚¹ã‚’デコーダã«ä¸Žãˆã‚‹ãŸã‚ã«ã€æ³¨æ„メカニズム(attention mechanism)ã¯ã€ Bahdanauã»ã‹ã€2014 ( PDF )ã«ã¦å°Žå…¥ã•ã‚Œã¾ã—ãŸã€‚注æ„メカニズム(attenti
Chainerã§ã‚·ãƒ³ãƒ—ルãªç†Šæœ¬å¼ç¿»è¨³ - Qiita
背景 ã‚る日，ç§ã¯æ©Ÿæ¢°ç¿»è¨³ã«èˆˆå‘³ã‚’æŒã£ãŸï¼Ž （英語ã«è‹¦ã—む日本ã®ç ”究者ã®å£°ã‚’èžã„ãŸã‹ã‚‰ã‹ã‚‚ã—ã‚Œãªã„ã—，勉強会ã®æº–å‚™ã«è¿½ã‚ã‚Œã¦ã„ãŸã‹ã‚‰ã‹ã‚‚ã—ã‚Œãªã„．） ã—ã‹ã—ç§ã¯é–€å¤–æ¼¢ãªã®ã§ï¼Œencoder-decoderモデル㮠"ã‚ã®" 図をèªã‚€ã“ã¨ãŒã§ããªã‹ã£ãŸï¼Ž 先日気æŒã¡ã‚’掴むã“ã¨ãŒã§ããŸã®ã§ï¼Œè‡ªåˆ†ãªã‚Šã«å™›ã¿ç •ã„ãŸã‚‚ã®ã‚’記録・共有ã™ã‚‹ãŸã‚ã«ã“ã®è¨˜äº‹ã‚’書ã„ãŸï¼Ž 先人ã®è©³ç´°ãªè¨˜äº‹ http://qiita.com/odashi_t/items/a1be7c4964fbea6a116e RNNã®å›³ã‚’翻訳 翻訳ã¾ã§ã®æµã‚Œã‚’別視点ã‹ã‚‰æã„ãŸï¼Ž encoder-decoderモデルã«ã‚ˆã£ã¦ã€Œå½¼ 㯠走るã€ã‚’「He runsã€ã¨ç¿»è¨³ã™ã‚‹ã¾ã§ã®æµã‚Œã¯ï¼Œä»¥ä¸‹ã®ã‚ˆã†ã«ãªã£ã¦ã„る． encodeå´ã®å›³ã«ãŠã„ã¦ï¼Œæ—¥æœ¬èªžã®ç™»å ´é †åºãŒodashi_tã•ã‚“ã®å›³ã¨ã¯é€†ã«ãªã£ã¦ã„る． é€†é †ã«å…¥åŠ›ã™ã‚‹ã¨å®Ÿé¨“çš„ã«è‰¯ã„çµæžœãŒå¾—られã¦ã„ã‚‹
Sequence to Sequence Learning with Neural Networks
Sequence to Sequence Learning with Neural Networks Ilya Sutskever Google ilyasu@google.com Oriol Vinyals Google vinyals@google.com Quoc V. Le Google qvl@google.com Abstract Deep Neural Networks (DNNs) are powerful models that have achieved excel- lent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequ
TensorFlow Tutorial | Sequence-to-Sequence Models
This tutorial: An encoder/decoder connected by attention. While this architecture is somewhat outdated, it is still a very useful project to work through to get a deeper understanding of sequence-to-sequence models and attention mechanisms (before going on to Transformers). This example assumes some knowledge of TensorFlow fundamentals below the level of a Keras layer: Working with tensors directl
1
リリースã€éšœå®³æƒ…å ±ãªã©ã®ã‚µãƒ¼ãƒ“スã®ãŠçŸ¥ã‚‰ã›
最新ã®äººæ°—エントリーã®é…ä¿¡
j次ã®ãƒ–ックマーク
kå‰ã®ãƒ–ックマーク
lã‚ã¨ã§èªã‚€
eコメント一覧を開ã
oページを開ã
http://renom.jp/ja/notebooks/tutorial/time_series/jp-en_nmt_seq2seq/notebook.html
A simple Seq2Seq translator
{{#tags}}- {{label}}
{{/tags}}