Skip to content

Fix MDP class and add POMDP subclass and notebook #781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
norvig merged 14 commits into from
Mar 2, 2018
Merged

Fix MDP class and add POMDP subclass and notebook #781

Changes from 1 commit
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
bd23fb7
Fixed typos and added inline LaTeX
bakerwho Feb 26, 2018
9d5ec3c
Fixed backslash for inline LaTeX
bakerwho Feb 27, 2018
d8af22b
Fixed more backslashes
bakerwho Feb 27, 2018
a09490d
Merge branch 'master' of https://github.com/aimacode/aima-python into…
bakerwho Feb 27, 2018
548c1cb
generalised MDP class and created POMDP notebook
bakerwho Feb 27, 2018
d04e14d
Fixed consistency issues with base MDP class
bakerwho Feb 28, 2018
bee037e
Small fix on CustomMDP
bakerwho Feb 28, 2018
97e51cd
Set default args to pass tests
bakerwho Feb 28, 2018
7e763e6
Added TableDrivenAgentProgram tests (#777)
ayushjain19 Feb 28, 2018
3fed661
Fixing tests
bakerwho Feb 28, 2018
bab969e
Fixed conflicts
bakerwho Feb 28, 2018
d778ec0
fixed conflicts
bakerwho Feb 28, 2018
35be708
fixed test_rl
bakerwho Feb 28, 2018
1a1a049
removed redundant code, fixed a comment
bakerwho Feb 28, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
removed redundant code, fixed a comment
  • Loading branch information
@bakerwho
bakerwho committed Feb 28, 2018
commit 1a1a0495073d463f2c5bae0e45587d0151ea9dee
5 changes: 2 additions & 3 deletions rl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,15 @@ def __init__(self, pi, mdp):
self.Ns1_sa = defaultdict(int)
self.s = None
self.a = None
self.visited = set()
self.visited = set() # keeping track of visited states

def __call__(self, percept):
s1, r1 = percept
self.mdp.states.add(s1) # Model keeps track of visited states.
mdp = self.mdp
R, P, terminals, pi = mdp.reward, mdp.P, mdp.terminals, self.pi
s, a, Nsa, Ns1_sa, U = self.s, self.a, self.Nsa, self.Ns1_sa, self.U

if s1 not in self.visited: # Reward is only available for visited state.
if s1 not in self.visited: # Reward is only known for visited state.
U[s1] = R[s1] = r1
self.visited.add(s1)
if s is not None:
Expand Down