[B! online learning] chezouã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯

chezou id:chezou

online learningã«é–¢ã™ã‚‹chezouã®ãƒ–ãƒƒã‚¯ãƒžãƒ¼ã‚¯ (7)

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

${{author_name}}$
{{author_name}}{{created}}
{{ #comment }}{{ comment }}{{ /comment }}
- {{ label }}

${{author_name}}$

{{{comment_expanded}}}

{{label}}

{{#is_bookmark}}ãƒªã‚¹ãƒˆ{{/is_bookmark}}{{^is_bookmark}}ãƒªãƒ³ã‚¯{{/is_bookmark}}

Optimizer : æ·±å±¤å¦ç¿’ã«ãŠã‘ã‚‹å‹¾é…æ³•ã«ã¤ã„ã¦ - Qiita
ã¯ã˜ã‚ã« æ·±å±¤å¦ç¿’ã®å‹¾é…æ³•ã«ã¯æ§˜ã€…ãªæ‰‹æ³•ãŒæå”±ã•ã‚Œã¦ã„ã¾ã™ã€‚ãã®ä¸ã§ã€ã©ã®æ‰‹æ³•ã‚’ç”¨ã„ã‚‹ã®ãŒé©åˆ‡ã§ã‚ã‚‹ã‹è‡ªåˆ†ã§ã‚‚ã‚ã¾ã‚Šç†è§£ã§ãã¦ã„ãªã„éƒ¨åˆ†ãŒã‚ã‚Šã€ä»Šå›žã¯å‹¾é…æ³•ã®ä¸ã§ã‚‚å®Ÿéš›ã«æ·±å±¤å¦ç¿’ã§ä¸»ã«ç”¨ã„ã‚‰ã‚Œã¦ã„ã‚‹æ‰‹æ³•(SGD, Momentum SGD, AdaGrad, RMSprop, AdaDelta, Adam)ã«ã¤ã„ã¦ã€å®Ÿè£…ã™ã‚‹ã“ã¨ã‚’å‰æã«èª¿ã¹ã¦ã¾ã¨ã‚ã¦ã¿ã¾ã—ãŸã€‚å®Ÿè£…ãƒ•ãƒ¬ãƒ¼ãƒ ãƒ¯ãƒ¼ã‚¯ã¯Chainerã‚’æƒ³å®šã—ã¦ã„ã¾ã™ã€‚ SGD SGD(Stochastic Gradient Descent : ç¢ºçŽ‡çš„å‹¾é…é™ä¸‹æ³•)ã¯ã€Optimizerã®ä¸ã§ã‚‚åˆæœŸã«æå”±ã•ã‚ŒãŸæœ€ã‚‚åŸºæœ¬çš„ãªã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã§ã™ã€‚é‡ã¿$\mathbf{w}$ã®æ›´æ–°ã¯ä»¥ä¸‹ã®ã‚ˆã†ã«è¡Œã£ã¦ãŠã‚Šã¾ã™ã€‚ã“ã®ã¨ãã€$E$ã¯èª¤å·®é–¢æ•°ã€$\eta$ã¯å¦ç¿’ä¿‚æ•°ã‚’è¡¨ã—ã¦ã„ã¾ã™ã€‚ \mathbf{w}^{t + 1} \gets \mathbf{w}^{t} -
chezou 2017/02/06
machine learning

online learning
ãƒªãƒ³ã‚¯
ã‚ªãƒ³ãƒ©ã‚¤ãƒ³å¦ç¿’ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ AROWã«ã¤ã„ã¦ - kivantiumæ´»å‹•æ—¥è¨˜
ä»¥å‰AROWã‚’ä½¿ã£ãŸè¨˜äº‹ã‚’æ›¸ãã¾ã—ãŸ(Caffeã«ã‚ˆã‚‹ç‰¹å¾´æŠ½å‡º+AROWã«ã‚ˆã‚‹åˆ†é¡žã‚’è©¦ã—ãŸ - kivantiumæ´»å‹•æ—¥è¨˜)ãŒã€AROWã®å†…éƒ¨ã‚’å…¨ç„¶çŸ¥ã‚‰ãšã«ä½¿ã£ã¦ã„ãŸã®ã§å‹‰å¼·ã—ã¾ã—ãŸã€‚ãã®è¨˜éŒ²ã§ã™ã€‚ ã“ã“ã®è¨˜è¿°ãŒæ§˜ã€…ãªã‚ªãƒ³ãƒ©ã‚¤ãƒ³å¦ç¿’ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®ç‰¹å¾´ã‚’ã†ã¾ãèª¬æ˜Žã—ã¦ã„ãŸã®ã§å‚è€ƒã«ã™ã‚‹ã¨ã„ã„ã‹ã‚‚ã—ã‚Œã¾ã›ã‚“ã€‚kazoo04.hatena blog.com ã‚ªãƒ³ãƒ©ã‚¤ãƒ³æ©Ÿæ¢°å¦ç¿’ã«ã¤ã„ã¦ã¯ã“ã†ã„ã†æœ¬ãŒå‡ºã¦ã„ã¦è©•åˆ¤ã‚‚è‰¯ã„ã§ã™ã€‚ï¼ˆã“ã“ã§å”çªã«è²¼ã‚‰ã‚Œã‚‹Amazonã¸ã®ãƒªãƒ³ã‚¯ï¼‰ ã‚ªãƒ³ãƒ©ã‚¤ãƒ³æ©Ÿæ¢°å¦ç¿’ (æ©Ÿæ¢°å¦ç¿’ãƒ—ãƒãƒ•ã‚§ãƒƒã‚·ãƒ§ãƒŠãƒ«ã‚·ãƒªãƒ¼ã‚º) ä½œè€…: æµ·é‡Žè£•ä¹Ÿ,å²¡é‡ŽåŽŸå¤§è¼”,å¾—å±…èª ä¹Ÿ,å¾³æ°¸æ‹“ä¹‹å‡ºç‰ˆç¤¾/ãƒ¡ãƒ¼ã‚«ãƒ¼: è¬›è«‡ç¤¾ç™ºå£²æ—¥: 2015/04/08ãƒ¡ãƒ‡ã‚£ã‚¢: å˜è¡Œæœ¬ï¼ˆã‚½ãƒ•ãƒˆã‚«ãƒãƒ¼ï¼‰ã“ã®å•†å“ã‚’å«ã‚€ãƒ–ãƒã‚° (3ä»¶) ã‚’è¦‹ã‚‹ ã‚ªãƒ³ãƒ©ã‚¤ãƒ³å¦ç¿’ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã¨ã¯ SVMãªã©ã®æ©Ÿæ¢°å¦ç¿’ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã§ã¯åˆ†é¡žã‚’è¡Œã†å‰ã«å…¨ã¦ã®å¦ç¿’ãƒ‡ãƒ¼ã‚¿
chezou 2017/02/06
machine learning

online learning
ãƒªãƒ³ã‚¯
AdaGradãŒ12å€é€Ÿããªã‚‹é”æ³•
AdaGradã¯å¦ç¿’çŽ‡ã‚’è‡ªå‹•èª¿æ•´ã—ã¦ãã‚Œã‚‹å‹¾é…æ³•ã®äºœç¨®ã§ã€ã„ã‚ã‚“ãªäººãŒçµ¶è³›ã—ã¦ã„ã¾ã™ã€‚ å‹¾é…ã‚’è¶³ã—è¾¼ã‚€æ™‚ã«ã€å„æ¬¡å…ƒã”ã¨ã«ä»Šã¾ã§ã®å‹¾é…ã®2ä¹—å’Œã‚’ã¨ã£ã¦ãŠã„ã¦ã€ãã®å¹³æ–¹æ ¹ã§å‰²ã£ã¦ã‚ã’ã‚‹ã ã‘ã¨ã€æã‚ã—ãã‚·ãƒ³ãƒ—ãƒ«ã§ã™ã€‚ Adaptive Subgradient Methods for Online Learning and Stochastic Optimization John Duchi, Elad Hazan, Yoram Singer. JMLR 2011. ä¸åº¦ã€ @echizen_tm ã•ã‚“ãŒãƒ–ãƒã‚°ã‚’æ›¸ã„ã¦ã¾ã—ãŸã€‚ AdaGrad+RDAã‚’å®Ÿè£…ã—ã¾ã—ãŸã€‚ é€šå¸¸ã®SGDãªã©ã¯å¦ç¿’çŽ‡ã‚’ã ã‚“ã ã‚“æ¸›è¡°ã•ã›ãªãŒã‚‰å‹¾é…ã‚’è¶³ã—ã¦ã„ãã‚ã‘ã§ã™ãŒã€ã©ã®æ§˜ã«æ¸›è¡°ã•ã›ã‚‹ã‹ã¨ã„ã†å•é¡Œã«ã„ã¤ã‚‚é ã‚’æ‚©ã¾ã›ã¾ã™ã€‚ AdaGradã§ã¯æœ€åˆã®å¦ç¿’çŽ‡ã“ãå¤–ã‹ã‚‰ä¸Žãˆã¾ã™ãŒã€æ¸›è¡°ã®ã•ã›æ–¹ã‚„æ¸›è¡°çŽ‡ã¨ã„ã£ãŸãƒã‚¤ãƒ‘ãƒ¼ãƒ‘ãƒ©ãƒ¡ãƒ¼ã‚¿ã‹ã‚‰
chezou 2014/08/07
online learning

machinelearning
ãƒªãƒ³ã‚¯
Exact Soft Confidence-Weighted Learning (ICML2012) èªã‚“ã - kisa12012ã®æ—¥è¨˜
æ¦‚è¦ ã‚ªãƒ³ãƒ©ã‚¤ãƒ³ã§ã®åˆ†é¡žå¦ç¿’ã®ä¸–ç•Œã§ã¯ï¼ŒCWãŒéžå¸¸ã«å¼·åŠ›ãªã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã¨ã—ã¦æ³¨ç›®ã•ã‚Œã¦ã„ã¾ã™ï¼Žç‰¹ã«ï¼Œãã®åœ§å€’çš„ãªåˆ†é¡žç²¾åº¦åŠã³åŽæŸé€Ÿåº¦ã¯åœ§å·»ã®ä¸€è¨€ã§ã‚ã‚Šï¼Œè‡ªç„¶è¨€èªžå‡¦ç†ã‚’ä¸å¿ƒã«æ§˜ã€…ãªåˆ†é‡Žã§å¿œç”¨ä¾‹ã‚„æ´¾ç”Ÿã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ãŒææ¡ˆã•ã‚Œã¦ã„ã¾ã™*1ï¼Ž ä¸€æ–¹ã§ï¼ŒãƒŽã‚¤ã‚ºãƒ‡ãƒ¼ã‚¿ã®ãŒæ··å…¥ã—ã¦ã„ãŸå ´åˆã«ç²¾åº¦ãŒã‚¬ã‚¿è½ã¡ã™ã‚‹æ€§è³ªãŒCWã®é‡å¤§ãªæ¬ ç‚¹ã¨ã—ã¦å¤šãã®äººã‹ã‚‰æŒ‡æ‘˜ã•ã‚Œã¦ã„ã¾ã—ãŸï¼ŽãƒŽã‚¤ã‚ºãŒäºˆã‚å–ã‚Šé™¤ã‹ã‚Œã¦ã„ã‚‹å®Ÿé¨“è¨å®šãªã‚‰ã°è‰¯ã„ã®ã§ã™ãŒï¼ŒãƒŽã‚¤ã‚ºãŒå«ã¾ã‚Œã¦ã„ã‚‹å¯èƒ½æ€§ã®é«˜ã„å®Ÿãƒ‡ãƒ¼ã‚¿ã«ã¯CWã¯ä¸ã€…ä¸ä¾¿ï¼Žã“ã®å•é¡Œã‚’è§£æ±ºã™ã‚‹ãŸã‚ï¼ŒãƒŽã‚¤ã‚ºè€æ€§ã®å¼·ã„CWç³»ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ã®æ±ºå®šç‰ˆï¼ˆï¼Ÿï¼‰ã¨ã—ã¦ï¼ŒSCW (Soft Confidence-Weighted)ã‚¢ãƒ«ã‚´ãƒªã‚ºãƒ ãŒICML2012ã¨ã„ã†ä¼šè°ã§ææ¡ˆã•ã‚Œã¾ã—ãŸï¼Žæœ¬ã‚¨ãƒ³ãƒˆãƒªã§ã¯ï¼ŒSCWã®ç´¹ä»‹ã‚’è¡Œã„ã¾ã™ï¼Ž Exact Soft Confidence-Weighted Learning, Wang
chezou 2013/11/27
machine learning

online learning
ãƒªãƒ³ã‚¯
GitHub - echizentm/ConfidenceWeighted: confidence weighted classifier
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert
chezou 2013/11/26
machinelearning

online learning
ãƒªãƒ³ã‚¯
ãƒžãƒ«ãƒã‚¯ãƒ©ã‚¹SCWè©•ä¾¡ãƒ¡ãƒ¢ - y_tagã®æ—¥è¨˜
æ˜¨å¹´ã®ICML2012ã§ã€ã‚ªãƒ³ãƒ©ã‚¤ãƒ³åˆ†é¡žå™¨ã§ã‚ã‚‹Soft Confidence-Weighted LearningãŒææ¡ˆã•ã‚ŒãŸï¼ˆ"Exact Soft Confidence-Weighted Learning"ï¼‰ã€‚ã“ã®éš›ã«ææ¡ˆã•ã‚ŒãŸã®ã¯åŸºæœ¬çš„ãªäºŒå€¤åˆ†é¡žã ã£ãŸãŒã€ä»Šã¾ã§ã®ã‚ªãƒ³ãƒ©ã‚¤ãƒ³åˆ†é¡žå™¨ã¨åŒã˜ã‚ˆã†ã«ã€ãƒžãƒ«ãƒã‚¯ãƒ©ã‚¹ã¸ã®æ‹¡å¼µã‚‚å¯èƒ½ã§ã‚ã‚‹ã€‚è©³ã—ãã¯ä»¥ä¸‹ã®è¨˜äº‹ã‚’å‚ç…§ã—ã¦ã„ãŸã ããŸã„ã€‚ Exact Soft Confidence-Weighted Learning (ICML2012) èªã‚“ã - kisa12012ã®æ—¥è¨˜ ã‚ªãƒ³ãƒ©ã‚¤ãƒ³ç·šå½¢åˆ†é¡žå™¨ã¨SCW - Sideswipe ãã‚“ãªãƒžãƒ«ãƒã‚¯ãƒ©ã‚¹SCWã‚’Jubatusã«å®Ÿè£…ã—ã¦ã¿ãŸã€‚ã‚³ãƒ¼ãƒ‰ã¯"y-tag/jubatus"ã®scwãƒ–ãƒ©ãƒ³ãƒã«ã‚ã‚‹ã€‚ã“ã®ãƒ–ãƒ©ãƒ³ãƒã¯0.4.2ã®ãƒªãƒªãƒ¼ã‚¹ç›´å¾Œã‚’ãƒ™ãƒ¼ã‚¹ã¨ã—ã¦ã„ã‚‹ã€‚ ãŸã å˜ç´”ãªåˆ†é¡žå™¨ã®è¿½åŠ ã ã‘ãªã‚‰è©±ã¯ç°¡å˜ãªã®ã ãŒã€åˆ†
chezou 2013/11/06
machine learning

online learning
ãƒªãƒ³ã‚¯
AROW ã¯ CW ã‚ˆã‚Šå¹¾åˆ†ãƒžã‚·ã‹ - ny23ã®æ—¥è¨˜
ä»Šè©±é¡Œï¼Ÿã® AROW (Adaptive Regularization Of Weight Vectors) ã® oll å‘ã‘ã®å®Ÿè£…ã‚’è¦‹ã‹ã‘ãŸã®ã§ï¼ŒConfidence-Weighted (CW) ãŒ Passive-Aggressive (PA)/LIBLINEAR ã«æ¯”ã¹ã¦å…¨ç„¶ãƒ€ãƒ¡ (PA/LIBLINEAR: 91%, CW: 88%; Perceptron: 89%) ã ã£ãŸä¾‹ã®ã‚¿ã‚¹ã‚¯ï¼ˆå¦ç¿’ï¼ãƒ†ã‚¹ãƒˆãƒ‡ãƒ¼ã‚¿ã¯å…¬é–‹ä¸ã®ã‚³ãƒ¼ãƒ‰ã«åŒæ¢±ã—ã¦ã‚ã‚‹ï¼‰ã§ï¼Œè©¦ã—ã¦ã¿ãŸï¼Žãƒ‘ãƒ©ãƒ¡ã‚¿ã‚’èª¿æ•´ã—ãŸãŒ (c=100, 50, 10, 5, 1, 0.5, 0.1, 0.05, 0.01, 0.005, 0.001, 0.0005, I=1, 5, 10, 20, 50, 100)ï¼ŒPA/LIBLINEAR -0.5% ç¨‹åº¦ã®ç²¾åº¦ (90.4%) ã—ã‹å‡ºãªã‹ã£ãŸï¼Žç¹°ã‚Šè¿”ã—å›žæ•°ã‚’10å›žä»¥ä¸‹ã¨ã‹ã§å›ºå®šã™ã‚Œã°ï¼ŒAROW
chezou 2013/11/06
machine learning

online learning
ãƒªãƒ³ã‚¯
1