[B! rnn] tokuryooã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

tokuryoo id:tokuryoo

rnnã«é–¢ã™ã‚‹tokuryooã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (65)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

LSTMã‚’è¶…ãˆã‚‹æœŸå¾…ã®æ–°æ˜Ÿã€QRNN - Qiita
RNNã€Œã“ã‚Œã£ã¦ã‚‚ã—ã‹ã—ã¦ã€ CNNã€Œã‚ãŸã—ãŸã¡ã®ãƒ¢ãƒ‡ãƒ«ãŒãƒ»ãƒ»ãƒ»ã€ ã€Œã€Œå…¥ã‚Œæ›¿ã‚ã£ã¦ã‚‹ï½žï½žï½žï¼ï¼Ÿã€ã€ ã¨ã„ã†ã‚ã‘ã§QRNNã€QUASI-RECURRENT NEURAL NETWORKSã¨ã¯ã€RNNã®æ©Ÿæ§‹ã‚’CNNã§ã€Œç–‘ä¼¼çš„(QUASI)ã«ã€å®Ÿè£…ã™ã‚‹ã¨ã„ã†ãƒ¢ãƒ‡ãƒ«ã§ã™ã€‚ã“ã‚Œã«ã‚ˆã‚Šã€æ—¢å˜ã®RNN(ã¨ã„ã†ã‹LSTM)ãŒæŠ±ãˆã¦ã„ãŸã„ãã¤ã‹ã®å•é¡Œã®è§£æ±ºã‚’è©¦ã¿ã¦ã„ã¾ã™ã€‚ å…ƒè«–æ–‡ã¯ä»¥ä¸‹ã¨ãªã‚Šã¾ã™ã€‚ QUASI-RECURRENT NEURAL NETWORKS ä½œè€…ã«ã‚ˆã‚‹ãƒ–ãƒã‚° ä½œè€…ã®æ–¹ã®ãƒ–ãƒã‚°ã«Chainerã®ã‚µãƒ³ãƒ—ãƒ«ã‚³ãƒ¼ãƒ‰ãŒã‚ã£ãŸã®ã§ã€ãã‚Œã‚’å…ƒã«TensorFlowã§å®Ÿè£…ã—ã¦ã¿ã¾ã—ãŸã€‚æ—©ãå‹•ã‹ã—ãŸã„ï¼ã¨ã„ã†æ–¹ã¯ã“ã¡ã‚‰ã‚’è¦‹ã¦ã¿ã¦ãã ã•ã„ã€‚ icoxfog417/tensorflow_qrnn (Starã‚’é ‚ã‘ã‚Œã°åŠ±ã¿ã«ãªã‚Šã¾ã™ m(_ _)m) æœ¬è¨˜äº‹ã§ã¯ã€ã“ã®ç ”ç©¶ã®ãƒ¢ãƒãƒ™ãƒ¼ã‚·ãƒ§ãƒ³ã¨ãã®ã‚¢ãƒ—ãƒãƒ¼ãƒã«ã¤ã„
tokuryoo 2017/10/31
lstm

rnn
ãƒªãƒ³ã‚¯
GitHub - guillaume-chevalier/LSTM-Human-Activity-Recognition: Human Activity Recognition example using TensorFlow on smartphone sensors dataset and an LSTM RNN. Classifying the type of movement amongst six activity categories - Guillaume Chevalier
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
tokuryoo 2017/08/02
lstm

rnn
ãƒªãƒ³ã‚¯
è‡ªç„¶è¨€èªžå‡¦ç†ã«ãŠã‘ã‚‹ã€Attentionã®è€ãˆã‚‰ã‚Œãªã„çŸã• - Qiita
ã€Œã‚ãªãŸã£ã¦ã€ç§ã®è¨€è‘‰ã®æœ€å¾Œã®æ–¹ã—ã‹èžã„ã¦ãªã„ã®ãã€ å®Ÿã¯äººé–“ã ã‘ã§ãªããƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã‚‚ãã†ã ã£ãŸã€ã¨ã„ã†çµæžœã‚’ç¤ºã—ã¦ã„ã‚‹ã®ãŒã“ã¡ã‚‰ã®è«–æ–‡ã§ã™ã€‚ Frustratingly Short Attention Spans in Neural Language Modeling è¨€ã„è¨³ã¨ã—ã¦ã¯ã€Œã ã£ã¦å›ã®æ¬¡ã®è¨€è‘‰ã‚’äºˆæ¸¬ã™ã‚‹ã ã‘ãªã‚‰ãã‚Œã§ååˆ†ã ã‹ã‚‰ã€ã¨ã„ã†ã“ã¨ã«ãªã‚‹ã‚“ã§ã™ãŒã€ãã†è¨€ã†ã¨è§’ãŒç«‹ã¤ã®ã¯äººé–“é–¢ä¿‚ã«ãŠã„ã¦ã‚‚ç ”ç©¶ã«ãŠã„ã¦ã‚‚åŒæ§˜ã®ã‚ˆã†ã§ã™ã€‚ æœ¬ç·¨ã¯ã€ä¸Šè¨˜ã®è«–æ–‡ã®ç´¹ä»‹ã¨å…±ã«ä»–ã®é–¢é€£è«–æ–‡ã‚‚äº¤ãˆãªãŒã‚‰ã€å®Ÿéš›ã®æ‰€æœ¬å½“ã«æœ€å¾Œã®æ–¹ã—ã‹å¿…è¦ãªã„ã®ã‹ã€ãã†ã§ã‚ã‚Œã°ãªãœãã‚“ãªã“ã¨ã«ãªã‚‹ã®ã‹ã€ã¨ã„ã†æ‰€ã«ã¤ã„ã¦è¦‹ã¦è¡Œã‘ã‚Œã°ã¨æ€ã„ã¾ã™ã€‚ ãªãŠã€å‚ç…§ã—ãŸè«–æ–‡ã¯ä»¥ä¸‹ã®GitHubã§ç®¡ç†ã—ã¦ã„ã¾ã™ã€‚æ—¥ã€…æ›´æ–°ã•ã‚Œã¦ã„ã‚‹ãŸã‚ã€ç ”ç©¶å‹•å‘ãŒæ°—ã«ãªã‚‹æ–¹ã¯æ˜¯éžStar&Watchã‚’ã—ã¦é ‚ã‘ã‚Œã°ã¨ï¼ã€‚ arXivTimes At
tokuryoo 2017/07/31
rnn

lstm
ãƒªãƒ³ã‚¯
RNNï¼šæ™‚ç³»åˆ—ãƒ‡ãƒ¼ã‚¿ã‚’æ‰±ã†Recurrent Neural Networksã¨ã¯
Recurrent Neural Networksã¨ã¯ä½•ã‹ RNNã®å¿œç”¨äº‹ä¾‹ æ©Ÿæ¢°ç¿»è¨³ éŸ³å£°èªè˜ ç”»åƒã®æ¦‚è¦ç”Ÿæˆ èª¬æ˜Žæ–‡ã‹ã‚‰ã®ç”»åƒç”Ÿæˆ çŸ¥ã£ã¦ãŠãã¨ä¾¿åˆ©ãªRNNã®ç¨®é¡žã¨é€²åŒ– Simple RNN LSTM GRU Bi-directional RNN Attention RNN Quasi-Recurrent Neural Network TensorFlowã«ã‚ˆã‚‹RNNã®å®Ÿè£… ã¾ã¨ã‚ å‚è€ƒæ–‡çŒ® äººé–“ã¯ã€ç›®ã®å‰ã§èµ·ããŸå‡ºæ¥äº‹ã‹ã‚‰ã€æ¬¡ã«èµ·ã“ã‚Šãã†ãªå‡ºæ¥äº‹ã‚’äºˆæ¸¬ã—ãªãŒã‚‰æ–‡è„ˆã‚’èªã‚“ã§åˆ¤æ–ã‚’ä¸‹ã™ã“ã¨ãŒã§ãã¾ã™ã€‚ä¾‹ãˆã°ã€è»Šã‚’é‹è»¢ã—ã¦ã„ã‚‹éš›ã«æ©è¡Œè€…ãŒé£›ã³å‡ºã—ãã†ã ã¨æ€ãˆã°ã€ååˆ†ãªé–“éš”ã‚’ç½®ã„ã¦èµ°è¡Œã™ã‚‹ã“ã¨ãŒå‡ºæ¥ã‚‹ã§ã—ã‚‡ã†ã€‚ ã¾ãŸã€ç¾å®Ÿä¸–ç•Œã¯æ™‚é–“ã®åˆ¶ç´„ã‚’å—ã‘ã‚‹äº‹è±¡ã¯ãŸãã•ã‚“ã‚ã‚Šã¾ã™ã€‚ã‚¢ãƒ‹ãƒ¡ãƒ¼ã‚·ãƒ§ãƒ³ãªã©ã®ã‚¹ãƒˆãƒ¼ãƒªãƒ¼ã§ã¯ã€å‰å›žã®æ–‡è„ˆã‚’å‰æã¨ã—ã¦æ¬¡ã®å±•é–‹ãŒé€²ã‚“ã§ã„ãã¾ã™ã€‚ Recurrent Neural Ne
tokuryoo 2017/07/07
rnn

deep learning
ãƒªãƒ³ã‚¯
ãƒ“ãƒ¼ãƒ ã‚µãƒ¼ãƒã®åŸºç¤ŽçŸ¥è˜ã¨æ©Ÿæ¢°å¦ç¿’ã¸ã®3ã¤ã®æ´»ç”¨äº‹ä¾‹
æ·±ã•å„ªå…ˆæŽ¢ç´¢ã¨å¹…å„ªå…ˆæŽ¢ç´¢ æ·±ã•å„ªå…ˆæŽ¢ç´¢ å¹…å„ªå…ˆæŽ¢ç´¢ ãƒ“ãƒ¼ãƒ ã‚µãƒ¼ãƒ æ©Ÿæ¢°å¦ç¿’ã¸ã®å¿œç”¨ Google Alloã®è¿”ç” å¦ç¿’æ™‚ã«ãƒ“ãƒ¼ãƒ ã‚µãƒ¼ãƒã®å¹…ã‚’æŒãŸã›ã¦å¦ç¿’ 3Då½¢çŠ¶ã®å¦ç¿’ã¸ã®å¿œç”¨ ã¾ã¨ã‚ å‚è€ƒæ–‡çŒ® ãƒ“ãƒ¼ãƒ ã‚µãƒ¼ãƒï¼ˆBeam Searchï¼‰ã¯ã€æŽ¢ç´¢ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®ä¸€ç¨®ã§ãƒ¡ãƒ¢ãƒªã‚’ãã‚Œã»ã©å¿…è¦ã¨ã—ãªã„æœ€è‰¯å„ªå…ˆæŽ¢ç´¢ã§ã™ã€‚ æ©Ÿæ¢°å¦ç¿’ã®åˆ†é‡Žã§ã‚‚ã€ç¿»è¨³ã‚„ãƒãƒ£ãƒƒãƒˆãƒœãƒƒãƒˆã®è¿”ç”ãªã©ã«å¿œç”¨ã•ã‚Œã¦ã„ã¾ã™ã€‚æœ¬è¨˜äº‹ã§ã¯ã€ãƒ“ãƒ¼ãƒ ã‚µãƒ¼ãƒã®ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã‚’ç†è§£ã—ã¦ã©ã®ã‚ˆã†ã«å¿œç”¨ã•ã‚Œã¦ã„ã‚‹ã®ã‹ã‚’è§£èª¬ã—ã¾ã™ã€‚ æ©Ÿæ¢°å¦ç¿’ã‚’æ´»ç”¨ã—ãŸã‚·ã‚¹ãƒ†ãƒ ã‚’æ§‹ç¯‰ã™ã‚‹éš›ã«ã‚‚ã€æŽ¢ç´¢ç©ºé–“ãŒåºƒã„å ´åˆãªã©ã«ã¯å¿œç”¨å¯èƒ½ãªã®ã§ã€ä½¿ã„ã“ãªã›ã‚‹ã‚ˆã†ã«ã—ã¦ãŠãã¨å½¹ã«ç«‹ã¡ã¾ã™ã€‚ æ·±ã•å„ªå…ˆæŽ¢ç´¢ã¨å¹…å„ªå…ˆæŽ¢ç´¢ ã„ããªã‚Šãƒ“ãƒ¼ãƒ ã‚µãƒ¼ãƒã®è§£èª¬ã«å…¥ã‚‹å‰ã«ã€ç†è§£ã—ã‚„ã™ã„ã‚ˆã†ã«ã‚°ãƒ©ãƒ•æŽ¢ç´¢ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã‚’ç´¹ä»‹ã—ã¾ã™ã€‚ æ·±ã•å„ªå…ˆæŽ¢ç´¢ æ·±ã•å„ªå…ˆæŽ¢ç´¢ã¯ã€ãã®åã®é€šã‚Šå¯èƒ½ãªé™ã‚Šçªãé€²ã‚“ã§ã€è¡Œã‘ãªããªã£ãŸ
tokuryoo 2017/07/07
rnn

deep learning
ãƒªãƒ³ã‚¯
æ•™å¸«ãªã—RNNã«ã‚ˆã‚‹é€£ç¶šã—ãŸä¸å®Œå…¨æƒ…å ±ï¼ˆä¸»ã«ãƒ†ã‚ã‚¹ãƒˆï¼‰ã®è£œå®Œ - ã«ã»ã‚“ã”ã®ã‚Œã‚“ã—ã‚…ã†
æ•™å¸«ãªã—RNNã«ã‚ˆã‚‹é€£ç¶šã—ãŸä¸å®Œå…¨æƒ…å ±ï¼ˆä¸»ã«ãƒ†ã‚ã‚¹ãƒˆï¼‰ã®è£œå®Œ èƒŒæ™¯ Googleï¼ˆã‚„ã€éŽåŽ»ã®ç§ã®ãƒ–ãƒã‚°ã®æŠ•ç¨¿ï¼‰ãªã©ã§ä½Žè§£åƒåº¦ã®æ˜ åƒã‚’é«˜è§£åƒåº¦åŒ–ã™ã‚‹æ–¹æ³•ã¯ã€GANã‚’ä½¿ã†ã“ã¨ã§ã‚ã‚Šã¨ã†ã¾ãã„ãã¾ã—ãŸã€‚ ãƒ†ã‚ã‚¹ãƒˆã«ã‚‚åŒã˜ã‚ˆã†ãªå•é¡ŒãŒé©å¿œã§ãã‚‹ã®ã‹ã£ã¦æ€ã†ã“ã¨ãŒã‚ã‚Šã¾ã™ã€‚RNNã¯ã™ã§ã«è¨˜è¿°ã•ã‚Œã¦ã„ã‚‹æ–‡å—ã‚’ãƒ’ãƒ³ãƒˆã¨ã—ã¦æ¬¡ã®æ–‡å—ã‚’äºˆæƒ³ã™ã‚‹ã“ã¨ãŒã§ãã¾ã™ã€‚ ãã®ç‰¹æ€§ã‚’åˆ©ç”¨ã—ã¦ã€æƒ…å ±ãŒå¤±ã‚ã‚ŒãŸãƒ†ã‚ã‚¹ãƒˆã‹ã‚‰å¾©å…ƒã‚’è©¦ã¿ã‚‹ä¾‹ã‚’ç¤ºã—ã¾ã™ã€‚ (ã¾ãŸã€æˆ¦æ™‚ä¸ã®æˆ¦è‰¦ã«ä¹—ã£ã¦ã„ãŸæä¹—å“¡ã®åŠ£åŒ–ã—ãŸæ‰‹è¨˜ã‹ã‚‰æƒ…å ±ã‚’å¾©å…ƒã—ãŸã„ã¨ã„ã†æ€ã„ãŒã‚ã‚Šã¾ã™ã€‚æ‰‹è¨˜ã®æµ·æ°´ã«æ²ˆã¿è…æ•—ãŒé€²ã‚“ã§ã„ãŸã‚Šã€ç†è¨˜ãŒä¹±ã‚Œã¦ã„ã¦è§£æžãŒå›°é›£ã§ã‚ã£ãŸã‚Šã¨ã„ã†ç‚¹ã‚’è£œå®Œã§ãã‚Œã°è‰¯ã„ã¨æ€ã„ã¾ã™ã€‚å½¼ã‚‰ã®æ€ã„ã€å¯èƒ½ãªé™ã‚Šç¾ä»£ã«è˜‡ã‚‰ã›ãŸã„ã¨æ€ã„ã¾ã™ã€‚) å…ˆè¡Œç ”ç©¶ ä»Šå›žã¯è¦‹å½“ãŸã‚‰ãªã„ï¼ˆã»ã‚“ã¨ã«å˜åœ¨ã—ãªã„ã‚“ã ã£ãŸã‚‰ã€è«–æ–‡å‡ºã—ãŸã„ï¼‰ æ‰‹æ³• RNNã§æ¬ è½ã—ãŸé€£ç¶šã™ã‚‹æƒ…å ±ã‚’å‘¨è¾ºã®å˜èªž
tokuryoo 2017/03/06
deep learning

rnn

nlp
ãƒªãƒ³ã‚¯
æ·±å±¤å¦ç¿’ã«ã‚ˆã‚‹è‡ªç„¶è¨€èªžå‡¦ç† - RNN, LSTM, ãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«æ©Ÿæ¢°ç¿»è¨³ã®ç†è«–
æœ¬ç¨¿ã§ã¯ãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ï¼Œèª¤å·®é€†ä¼æ’æ³•ï¼Œè¨€èªžãƒ¢ãƒ‡ãƒ«ï¼ŒRNNï¼ŒLSTMï¼Œãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«æ©Ÿæ¢°ç¿»è¨³ã®ä¸€é€£ã®æ‰‹æ³•ã«ã¤ã„ã¦æ•°ç†çš„ã«è§£èª¬ã™ã‚‹ï¼Ž å‰ç·¨ã®ç›®æ¬¡ ãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ é †ä¼æ’ (Forwardpropagation) é€†ä¼æ’ (Backpropagation) ãƒªã‚«ãƒ¬ãƒ³ãƒˆãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ (RNN) Recurrent Neural Network Language Model (RNNLM) Backpropagation Through Time (BPTT) Long Short-Term Memory (LSTM) Gated Recurrent Unit (GRU) RNN ã®ãƒ‰ãƒãƒƒãƒ—ã‚¢ã‚¦ãƒˆã¨ãƒãƒƒãƒæ£è¦åŒ– ãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«æ©Ÿæ¢°ç¿»è¨³ (NMT) Sequence to Sequence (seq2seq) æ³¨æ„ (Attention) åŒæ–¹å‘ã‚¨ãƒ³ã‚³ãƒ¼ãƒ€ãƒ¼ãƒ»å¤šå±¤LSTM è©•ä¾¡æ‰‹æ³•
tokuryoo 2017/02/17
deep learning

nlp

lstm

rnn
ãƒªãƒ³ã‚¯
ä»Šå¹´(2016)å‚è€ƒã«ã•ã›ã¦ã‚‚ã‚‰ã£ãŸChainerã®è«–æ–‡å®Ÿè£…ã€ã‚µãƒ³ãƒ—ãƒ«ã‚³ãƒ¼ãƒ‰é›† - studylog/åŒ—ã®é›²
åŽ»å¹´æ›¸ã„ãŸã‚µãƒ³ãƒ—ãƒ«ã‚³ãƒ¼ãƒ‰é›†ã®2016å¹´ç‰ˆã§ã™ã€‚ å€‹äººçš„ãªèˆˆå‘³ç¯„å›²ã®ã¿é›†ã‚ã¦ã„ã‚‹ã®ã§ç¶²ç¾…çš„ã§ã¯ç„¡ã„ã¨ã¯æ€ã„ã¾ã™ã€‚ åŸºæœ¬çš„ã«ä¸Šã®æ–¹ãŒæ–°ã—ã„ã‚³ãƒ¼ãƒ‰ã§ã™ã€‚ QRNN(Quasi-Recurrent Neural Networks) è«–æ–‡ã§ã¯chainerã‚’ä½¿ã£ã¦å®Ÿé¨“ã—ã¦ãŠã‚Šã€æ™®é€šã®LSTMã¯ã‚‚ã¡ã‚ã‚“cuDNNã‚’ä½¿ã£ãŸLSTMã‚ˆã‚Šã‚‚é«˜é€Ÿã‚‰ã—ã„ã€‚ ä¸€ç•ªä¸‹ã«chainerå®Ÿè£…ã‚³ãƒ¼ãƒ‰ãŒåŸ‹ã‚è¾¼ã¾ã‚Œã¦ã„ã‚‹ã€‚ New neural network building block allows faster and more accurate text understanding [1611.01576] Quasi-Recurrent Neural Networks Softmaxãªã©ã«è¦ã—ã¦ã„ã‚‹æ™‚é–“ã‚’é™¤ãç´”ç²‹ã«RNNéƒ¨ã ã‘ã§æ¯”è¼ƒã™ã‚‹ã¨ç›¸å½“é€Ÿããªã£ã¦ã„ã‚‹ã®ãŒã‚ã‹ã‚‹ã€‚ ã“ã®ã‚ˆã†ã«å®Ÿé¨“ã§chainerãŒä½¿ã‚ã‚Œã¦ã„ã‚‹ç ”
tokuryoo 2016/12/13
chainer

paper

deep learning

rnn
ãƒªãƒ³ã‚¯
JUMAN++ - LANGUAGE MEDIA PROCESSING LAB
æ—¥æœ¬èªžå½¢æ…‹ç´ è§£æžã‚·ã‚¹ãƒ†ãƒ Juman++ â€ Juman++ã¯è¨€èªžãƒ¢ãƒ‡ãƒ«ã‚’åˆ©ç”¨ã—ãŸé«˜æ€§èƒ½ãªå½¢æ…‹ç´ è§£æžã‚·ã‚¹ãƒ†ãƒ ã§ã™ï¼Žè¨€èªžãƒ¢ãƒ‡ãƒ«ã¨ã—ã¦ Recurrent Neural Network Language Model (RNNLM) ã‚’ç”¨ã„ã‚‹ã“ã¨ã«ã‚ˆã‚Šï¼Œå˜èªžã®ä¸¦ã³ã®æ„å‘³çš„ãªè‡ªç„¶ã•ã‚’è€ƒæ…®ã—ãŸè§£æžã‚’è¡Œã„ã¾ã™ï¼Žãã‚Œã«ã‚ˆã‚Š JUMANï¼ŒMeCab ã«æ¯”ã¹å¤§ããæ€§èƒ½ãŒå‘ä¸Šã—ã¦ã„ã¾ã™ï¼Žæ–‡æ³•ãƒ»è¾žæ›¸ãƒ»å‡ºåŠ›ãƒ•ã‚©ãƒ¼ãƒžãƒƒãƒˆç‰ã¯ JUMAN ã‹ã‚‰å¼•ãç¶™ã„ã ã‚‚ã®ã‚’åˆ©ç”¨ã—ã¦ã„ã¾ã™ï¼Žæœ¬ã‚·ã‚¹ãƒ†ãƒ ã¯ CRESTã€ŒçŸ¥è˜ã«åŸºã¥ãæ§‹é€ çš„è¨€èªžå‡¦ç†ã®ç¢ºç«‹ã¨çŸ¥è˜ã‚¤ãƒ³ãƒ•ãƒ©ã®æ§‹ç¯‰ã€ã®æ”¯æ´ã«ã‚ˆã‚Šé–‹ç™ºã•ã‚Œã¾ã—ãŸï¼Ž â†‘ ä½¿ç”¨æ–¹æ³• â€ ãŸã¨ãˆã°ï¼Œæ¬¡ã®ã‚ˆã†ãªãƒ†ã‚ã‚¹ãƒˆã‚’å…¥åŠ›ã™ã‚‹ã¨ï¼Œ % cat sample.txt å¤–å›½äººå‚æ”¿æ¨© ç§ã‚‚ã‚ã•ã£ã¦æ—¥æ›œæœ€çµ‚æ—¥ ä»¥ä¸‹ã®è§£æžçµæžœãŒå¾—ã‚‰ã‚Œã¾ã™ï¼Ž % cat sample.txt | jumanpp å¤–å›½ ãŒã„ã“ã å¤–å›½
tokuryoo 2016/09/28
rnn

nlp

deep learning
ãƒªãƒ³ã‚¯
A Neural Network for Machine Translation, at Production Scale
Posted by Quoc V. Le & Mike Schuster, Research Scientists, Google Brain Team Ten years ago, we announced the launch of Google Translate, together with the use of Phrase-Based Machine Translation as the key algorithm behind this service. Since then, rapid advances in machine intelligence have improved our speech recognition and image recognition capabilities, but improving machine translation rem ai
tokuryoo 2016/09/28
rnn

deep learning

nlp
ãƒªãƒ³ã‚¯
Supervised Sequence Labelling with Recurrent Neural Networks - reading
LTI @ Carnegie Mellon University Supervised Sequence Labelling with Recurrent Neural Networks Kazuya Kawakami Alex Graves, Ph.D Thesis (2008) Chapter 4 to Chapter 7 LTI @ Carnegie Mellon University Recurrent Neural Networks â€¢ Benefit of Recurrent Networksâ€¨ - Learning representations of contextual information of sequence. â€¢ Limitations of RNNsâ€¨ - The range of accessible context is limited. (Vanish
tokuryoo 2016/09/27
ctc

rnn

lstm

deep learning
ãƒªãƒ³ã‚¯
ãƒªã‚«ãƒ¬ãƒ³ãƒˆãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã®ç†ä¸å°½ãªåŠ¹åŠ›ï¼ˆç¿»è¨³ï¼‰ - Qiita
ä¸Šè¨˜ã¯ã€æ™®é€šã® RNN ã®å‰é€²ãƒ‘ã‚¹ã®ä»•æ§˜ã‚’æ±ºã‚ã¾ã™ã€‚ãã®ãŸã‚ã® RNN ã®ãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã¯ã€ï¼“ã¤ã®è¡Œåˆ— $W_{hh}, W_{xh}, W_{hy}$ ã§ã™ã€‚éš ã‚ŒçŠ¶æ…‹ $self.h$ ã¯ã€ã‚¼ãƒãƒ™ã‚¯ãƒˆãƒ«ã§åˆæœŸåŒ–ã•ã‚Œã¾ã™ã€‚$np.tanh$ é–¢æ•°ã¯ã‚¢ã‚¯ãƒ†ã‚£ãƒ™ãƒ¼ã‚·ãƒ§ãƒ³ã‚’ $[-1, 1]$ ã®ç¯„å›²ã«æŠ¼ã—ã¤ã¶ã™ã€éžç·šå½¢æ€§ã‚’å®Ÿç¾ã—ã¾ã™ã€‚ã©ã®ã‚ˆã†ã«ä½œç”¨ã™ã‚‹ã®ã‹ç°¡å˜ã«è¿°ã¹ã‚‹ã¨ï¼štanh å†…ã«ï¼’ã¤ã®é …ãŒã‚ã‚Šã¾ã™ï¼šä¸€æ–¹ã¯å‰ã®éš ã‚ŒçŠ¶æ…‹ã«åŸºã¥ãã€ä»–æ–¹ã¯ç¾åœ¨ã®å…¥åŠ›ã«åŸºã¥ãã¾ã™ã€‚numpy ã® $np.dot$ ã¯è¡Œåˆ—ã®æŽ›ã‘ç®—ã§ã™ã€‚ï¼’ã¤ã®ä¸é–“é …ã¯åŠ ç®—ã•ã‚Œã€tanh ã«ã‚ˆã‚Šæ–°ãŸãªçŠ¶æ…‹ãƒ™ã‚¯ãƒˆãƒ«ã«æŠ¼ã—ã¤ã¶ã•ã‚Œã¾ã™ã€‚æ•°å¦ã®è¡¨è¨˜ã«æ…£ã‚Œã¦ã„ã‚‹ãªã‚‰ã€éš ã‚ŒçŠ¶æ…‹ã®æ›´æ–°ã¯ä»¥ä¸‹ã®ã‚ˆã†ã«è¨˜è¿°å¯èƒ½ã§ã™ã€‚$h_t = \tanh ( W_{hh} h_{t-1} + W_{xh} x_t )$ ã€ã“ã“ã§ tanh ã¯è¦ç´ ã”ã¨ã«é©ç”¨ã•ã‚Œã¾ã™ã€‚
tokuryoo 2016/09/27
deep learning

rnn
ãƒªãƒ³ã‚¯
å†å¸°åž‹ãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆ in æ©Ÿæ¢°å¦ç¿’ãƒ—ãƒãƒ•ã‚§ãƒƒã‚·ãƒ§ãƒŠãƒ«ã‚·ãƒªãƒ¼ã‚ºè¼ªèªä¼š
3. æœ¬ç« ã§æ‰±ã†å•é¡Œ ï½ ã‚µãƒ³ãƒ—ãƒ«æ¯Žã«ç³»åˆ—é•·ãŒé•ã†ãƒ‡ãƒ¼ã‚¿ã‚’ã©ã†æ‰±ã†ã‹ï¼Ÿ ãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆ ïƒ· ïƒ· ïƒ· ïƒ¸ ïƒ¶ ïƒ§ ïƒ§ ïƒ§ ïƒ¨ ïƒ¦ 1.0 9.0 3.0 å…¥åŠ›ãƒ™ã‚¯ãƒˆãƒ«ç³»åˆ— å‡ºåŠ›ãƒ™ã‚¯ãƒˆãƒ«ç³»åˆ— ïƒ· ïƒ· ïƒ· ïƒ¸ ïƒ¶ ïƒ§ ïƒ§ ïƒ§ ïƒ¨ ïƒ¦ 6.0 5.0 2.0 ïƒ· ïƒ· ïƒ· ïƒ¸ ïƒ¶ ïƒ§ ïƒ§ ïƒ§ ïƒ¨ ïƒ¦ 2.0 4.0 6.0 ïŒ ïƒ·ïƒ· ïƒ¸ ïƒ¶ ïƒ§ïƒ§ ïƒ¨ ïƒ¦ 1.0 9.0 ïƒ·ïƒ· ïƒ¸ ïƒ¶ ïƒ§ïƒ§ ïƒ¨ ïƒ¦ 6.0 3.0 ïƒ·ïƒ· ïƒ¸ ïƒ¶ ïƒ§ïƒ§ ïƒ¨ ïƒ¦ 7.0 3.0 ïŒ å›ºå®šæ¬¡å…ƒ å›ºå®šæ¬¡å…ƒ å¯å¤‰ å…¥åŠ›ã¨åŒã˜ç³»åˆ—é•· 4. å¿œç”¨ ï½ è‡ªç„¶è¨€èªžå‡¦ç† ï½ é€”ä¸ã¾ã§ã®æ–‡ç« ã‹ã‚‰æ¬¡ã®å˜èªžã‚’é †æ¬¡äºˆæ¸¬ ï½ éŸ³å£°èªè˜ ï½ çŸæ™‚é–“ãƒ•ãƒ¬ãƒ¼ãƒ æ¯Žã®éŸ³ç´ èªè˜ We can get an idea of the quality of the leaned feature 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9
tokuryoo 2016/09/06
rnn

lstm

ctc
ãƒªãƒ³ã‚¯
Pythonã‚³ãƒ¼ãƒ‰ã‚’ä½¿ç”¨ã—ã¦ã€Pythonã‚³ãƒ¼ãƒ‰ã‚’æ›¸ãæ–¹æ³•ã‚’äººå·¥çŸ¥èƒ½ã«å¦ç¿’ã•ã›ã‚‹ | POSTD
ã“ã“ã§ã¯å°‘ã—ã®é–“ã€è‡ªå¾‹èµ°è¡Œè»Šã®ã“ã¨ã¯å¿˜ã‚Œã¦ãã ã•ã„ã€‚ç‰©äº‹ã¯æ·±åˆ»ã«ãªã£ã¦ãã¦ã„ã¾ã™ã€‚ã“ã®è¨˜äº‹ã§ã¯ã€ç‹¬è‡ªã®ã‚³ãƒ¼ãƒ‰ã‚’æ›¸ããƒžã‚·ãƒ³ã‚’ä½œã‚‹ã“ã¨ã«çš„ã‚’çµžã£ã¦è©±ã‚’é€²ã‚ã¦ã„ããŸã„ã¨æ€ã„ã¾ã™ã€‚ GlaDoS Skynet Spynetã‚’ä½¿ç”¨ã—ã¾ã™ã€‚ å…·ä½“çš„ã«è¨€ã†ã¨ã€Pythonã®ã‚½ãƒ¼ã‚¹ã‚³ãƒ¼ãƒ‰ã‚’å…¥åŠ›ã™ã‚‹ã“ã¨ã§ã€è‡ªåˆ†ã§ã‚³ãƒ¼ãƒ‰ã‚’æ›¸ãã‚ˆã†ã«ã€æ–‡å—ãƒ¬ãƒ™ãƒ«ã§ã®Long Short Term Memoryãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯ã‚’è¨“ç·´ã—ã¦ã„ãã¾ã™ã€‚ã“ã®å¦ç¿’ã¯ã€Theanoã¨Lasagneã‚’ä½¿ã£ã¦ã€EC2ã®GPUã‚¤ãƒ³ã‚¹ã‚¿ãƒ³ã‚¹ä¸Šã§èµ·å‹•ã•ã›ã¾ã™ã€‚èª¬æ˜ŽãŒæ›–æ˜§ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ãŒã€åˆ†ã‹ã‚Šã‚„ã™ãèª¬æ˜Žã§ãã‚‹ã‚ˆã†ã«é ‘å¼µã£ã¦ã¿ã¾ã™ã€‚ ã“ã®è©¦ã¿ã¯ã€ ã“ã¡ã‚‰ã®ç´ æ™´ã‚‰ã—ã„ãƒ–ãƒã‚°è¨˜äº‹ ã«è§¦ç™ºã•ã‚Œè¡Œã†ã«è‡³ã‚Šã¾ã—ãŸã€‚çš†ã•ã‚“ã‚‚ãœã²èªã‚“ã§ã¿ã¦ãã ã•ã„ã€‚ ç§ã¯ãƒ‡ã‚£ãƒ¼ãƒ—ãƒ©ãƒ¼ãƒ‹ãƒ³ã‚°ã®ã‚¨ã‚ã‚¹ãƒ‘ãƒ¼ãƒˆã§ã¯ã‚ã‚Šã¾ã›ã‚“ã—ã€Theanoã‚„GPUã‚³ãƒ³ãƒ”ãƒ¥ãƒ¼ãƒ†ã‚£ãƒ³ã‚°ã‚’æ‰±ã†ã®ã‚‚åˆã‚ã¦ã§
tokuryoo 2016/08/24
deep learning

lstm

rnn
ãƒªãƒ³ã‚¯
Supervised Sequence Labelling with Recurrent Neural Networks - Alex Graves
Technische UniversitÂ¨at MÂ¨unchen FakultÂ¨at fÂ¨ur Informatik Lehrstuhl VI: Echtzeitsysteme und Robotik Supervised Sequence Labelling with Recurrent Neural Networks Alex Graves VollstÂ¨andiger Abdruck der von der FakultÂ¨at fÂ¨ur Informatik der Technischen UniversitÂ¨at MÂ¨unchen zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften (Dr. rer. nat.) genehmigten Dissertation. Vorsitze
tokuryoo 2016/07/25
æ•™ç§‘æ›¸ï¼ŸConnectionist Temporal Classification ã‚‚è¼‰ã£ã¦ã‚‹ã€‚

deep learning

rnn

cnn
ãƒªãƒ³ã‚¯
è«–æ–‡ç´¹ä»‹ Pixel Recurrent Neural Networks
1. Pixel Recurrent Neural Networks ICML2016 èªã¿ä¼š @ ãƒ‰ãƒ¯ãƒ³ã‚´ å¾—å±… èª ä¹Ÿ, Preferred Networks & æ±äº¬å¤§å¦ è«–æ–‡ç´¹ä»‹ 2. æ¦‚è¦ Pixel Recurrent Neural Networks. AÃ¤ron van den Oord, Nal Kalchbrenner, Koray Kavukcuoglu (all from Google DeepMind) ICML2016 ã®ãƒ™ã‚¹ãƒˆãƒšãƒ¼ãƒ‘ãƒ¼ï¼“æœ¬ã®ã†ã¡ã®ã²ã¨ã¤ -è¦ç´„- ãƒ”ã‚¯ã‚»ãƒ«ãƒ»ãƒãƒ£ãƒ³ãƒãƒ«å˜ä½ã®è‡ªå·±å›žå¸°åž‹ãƒ¢ãƒ‡ãƒ«ï¼ˆRNN è¨€èªžãƒ¢ãƒ‡ ãƒ«ã«ä¼¼ãŸç¢ºçŽ‡åˆ†å¸ƒï¼‰ã§ã‚ã£ã¦ä¸¦åˆ—åŒ–ã«ã‚ˆã£ã¦é«˜é€Ÿã«å‹¾é…è¨ˆç®—ãŒ ã§ãã‚‹ã‚‚ã®ã‚’ææ¡ˆã—ã€MNIST, CIFAR-10, ImageNet (32x32, 64x64) ã§éžå¸¸ã«é«˜ã„å¯¾æ•°å°¤åº¦ã‚’é”æˆã—ãŸ 2 3. ç”»åƒã®ç”Ÿæˆãƒ¢ãƒ‡ãƒ« â€¢ è¨“ç·´ãƒ‡ãƒ¼ã‚¿ï¼šå¤§é‡ã®ç”»
tokuryoo 2016/07/22
rnn

deep learning
ãƒªãƒ³ã‚¯
IBM Video Streaming
By signing up, you agree to receive em ail updates from IBM as well as the Terms of Service and Privacy Policy.
tokuryoo 2016/07/11
deep learning

ctc

rnn
ãƒªãƒ³ã‚¯
ãƒ‡ã‚£ãƒ¼ãƒ—ãƒ©ãƒ¼ãƒ‹ãƒ³ã‚°ã«ã‚ˆã‚‹ è‡ªç„¶è¨€èªžå‡¦ç† (æŠ€è¡“ç·¨)
åå¤å±‹å¤§å¦ç‰¹åˆ¥è¬›ç¾© 2016å¹´6æœˆ29æ—¥(æ°´) ãƒ‡ã‚£ãƒ¼ãƒ—ãƒ©ãƒ¼ãƒ‹ãƒ³ã‚°ã«ã‚ˆã‚‹ è‡ªç„¶è¨€èªžå‡¦ç† (æŠ€è¡“ç·¨) æ—¥æœ¬ã‚¢ã‚¤ãƒ»ãƒ“ãƒ¼ãƒ»ã‚¨ãƒ æ ªå¼ä¼šç¤¾ æ±äº¬åŸºç¤Žç ”ç©¶æ‰€ åªäº• ç¥å¤ª [email protected] 1 ãƒ‹ãƒ¥ãƒ¼ãƒ©ãƒ«ãƒãƒƒãƒˆãƒ¯ãƒ¼ã‚¯æŠ€è¡“è©³ç´° â€¢ ç›®çš„é–¢æ•° â€¢ èª¤å·®é–¢æ•° â€¢ ç›®çš„é–¢æ•°ã®æœ€å°åŒ– â€¢ å‹¾é…æ³• â€¢ ç›®çš„é–¢æ•°ã®å¾®åˆ†è¨ˆç®— â€¢ èª¤å·®é€†ä¼æ¬æ³• â€¢ èª¤å·®ã®åˆ†è§£ã¨å¯¾å‡¦æ‰‹æ³• â€¢ æŽ¨å®šèª¤å·®ã«åŠ¹ãæ‰‹æ³• â€¢ æœ€é©åŒ–èª¤å·®ã«åŠ¹ãæ‰‹æ³• â€¢ RNNã®è©±é¡Œ 2 ç›®çš„é–¢æ•°: èª¤å·® â€¢ æ•™å¸«ã‚ã‚Šå¦ç¿’ã®ç›®çš„é–¢æ•° â€¢ ð‘¥ âˆˆ ð‘‹: å…¥åŠ›, ð‘¦ âˆˆ ð‘Œ: å‡ºåŠ› â€¢ å…¥åŠ›xã‹ã‚‰yã‚’äºˆæ¸¬ã—ãŸã„å•é¡Œè¨å®š â€¢ çœŸã®ç›®çš„é–¢æ•°: ð¿âˆ— ðœƒ = âˆ«ð‘ ð‘¥, ð‘¦ â„“ðœƒ ð‘¥, ð‘¦ ð‘‘ð‘¥ð‘‘ð‘¦ â€¢ â„“ðœƒã¯äº‹ä¾‹ã”ã¨ã®æå¤±é–¢æ•°ï¼ˆå¾Œè¿°ï¼‰ â€¢ è¨“ç·´ãƒ‡ãƒ¼ã‚¿ã§ã®èª¤å·® â€¢ ãƒ‡ãƒ¼ã‚¿åˆ†å¸ƒp(x,y)ã¯æ™®é€šã‚ã‹ã‚‰ãªã„ã®ã§è¨“ç·´ãƒ‡ãƒ¼ã‚¿Nå€‹: D=
tokuryoo 2016/07/05
LSTM - "forget gate fã®ãƒã‚¤ã‚¢ã‚¹bã¯1ã«åˆæœŸåŒ–ã™ã‚‹"

deep learning

nlp

lstm

rnn
ãƒªãƒ³ã‚¯
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Deep Recurrent Neural Network architectures, though remarkably capable at modeling sequences, lack an intuitive high-level spatio-temporal structure. That is while many probl ems in computer vision inherently have an underlying high-level structure and can benefit from it. Spatio-temporal graphs are a popular tool for imposing such high-level intuitions in the formulation of real world probl ems. In
tokuryoo 2016/06/29
computer vision

deep learning

rnn
ãƒªãƒ³ã‚¯
On Multiplicative Integration with Recurrent Neural Networks
We introduce a general and simple structural design called Multiplicative Integration (MI) to improve recurrent neural networks (RNNs). MI changes the way in which information from difference sources flows and is integrated in the computational building block of an RNN, while introducing almost no extra parameters. The new structure can be easily embedded into many popular RNN models, including LS
tokuryoo 2016/06/28
paper

deep learning

rnn
ãƒªãƒ³ã‚¯
1 2 3 4 æ¬¡ã®ãƒšãƒ¼ã‚¸