SlideShare a Scribd company logo

PGに簡単なゲームのやり方を学習させる Vol.1 - まずはQ学習を理解する

•
•
敬 松風敬
敬 松風敬

The document discusses the Q-learning algorithm for reinforcement learning. It explains that Q-learning uses a Q-table to store Q-values that represent the expected reward for taking an action in a given state. The Q-value for a state-action pair is updated each iteration based on the immediate reward and maximum future reward estimated by previous Q-values. The document provides pseudocode for the Q-learning update rule and gives an example of applying it to navigate between rooms to learn the best path.

1 of 41
Download to read offline
PG - Q @shohu33
ATARI
PG PG
(policy)
PG PG
PG Q ( ) ( or ) ( ) Q
"Q-Learning Tutorial". Mnemosyne Studio. http://mnemstudio.org/path-finding-q-learning-tutorial.htm
6 5
1. (Gamma) 2. Q 0 3. : 3.1 3.2 5 : 3.2.1 3.2.2 3.2.3 Q Q(state, action) = R(state, action) + Gamma * Max[Q(next state, all actions)] * Q 3.2.4 3.2.5 5 3.3
PGに簡単なゲームのやり方を学習させる Vol.1 - まずはQ学習を理解する
1. (Gamma)
(Gamma) 0 1 0 ( ) 0.8
[ ]
100
PGに簡単なゲームのやり方を学習させる Vol.1 - まずはQ学習を理解する
2. Q( ) 0
0 1 5 100 Q( ) 0( )
PGに簡単なゲームのやり方を学習させる Vol.1 - まずはQ学習を理解する
3.1
RANDOM
3.2 5
3.2.1
3 5
3.2.2
5
3.2.3 Q Q(state, action) = R(state, action) + Gamma * Max[Q(next state, all actions)]
今回、部屋1 から 部屋5 に移動する⾏動を選んだので Q state=1, action=5, Gamma=0.8, next state=5, all actions = 1,4,5 Q
Q Q 5 1,4,5 Q
3.2.4
5
3.2.6 5
5
3.3
3.1
3 1 Q(state, action) = R(state, action) + Gamma * Max[Q(next state, all actions)] Q(3, 1) = R(3, 1) + 0.8 * Max[Q(1, 3), Q(1, 5)] = 0 + 0.8 * Max(0, 100) = 80 1
PGに簡単なゲームのやり方を学習させる Vol.1 - まずはQ学習を理解する
1 5 5 5 Q(state, action) = R(state, action) + Gamma * Max[Q(next state, all actions)] Q(1, 5) = R(1, 5) + 0.8 * Max[Q(5, 1), Q(5, 4), Q(5, 5)] = 100 + 0.8 * Max(0, 0, 0) = 100 Q
PGに簡単なゲームのやり方を学習させる Vol.1 - まずはQ学習を理解する
Q Q 2 Q 2 → 3 → 1 → 5 OR 2 → 3 → 4 → 5 5
Q
Q

More Related Content

PGに簡単なゲームのやり方を学習させる Vol.1 - まずはQ学習を理解する