Skip to content

Added POMDP-value-iteration #929

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
norvig merged 20 commits into from
Jul 11, 2018
Merged

Added POMDP-value-iteration #929

Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
3a9b2fd
Added POMDP value iteration
ad71 Jun 3, 2018
f1606c4
Added plot_pomdp_utility function
ad71 Jun 3, 2018
5963be2
Added tests for pomdp-value-iteration
ad71 Jun 3, 2018
e981747
Updated README.md
ad71 Jun 3, 2018
2fac992
Fixed notebook import
ad71 Jun 3, 2018
6a6615a
Changed colors
ad71 Jun 4, 2018
f4f5b50
Added notebook sections for POMDP and pomdp_value_iteration
ad71 Jun 4, 2018
d9011f3
Fixed notebook parsing error
ad71 Jun 4, 2018
e22e7dd
Replace pomdp.ipynb
ad71 Jun 4, 2018
2d5e074
Updated README.md
ad71 Jun 4, 2018
47d6638
Fixed line endings
ad71 Jun 4, 2018
d401af8
Fixed line endings
ad71 Jun 4, 2018
2f25b7f
Fixed line endings
ad71 Jun 4, 2018
da199dc
Fixed line endings
ad71 Jun 4, 2018
089da5e
Removed numpy dependency
ad71 Jun 6, 2018
5a79deb
Added docstrings
ad71 Jun 6, 2018
8653daf
Fix tests
ad71 Jun 6, 2018
a3c78c9
Added a test for pomdp_value_iteration
ad71 Jun 22, 2018
7a80c3e
Remove numpy dependencies from mdp.ipynb
ad71 Jun 22, 2018
1c7a8cc
Added POMDP to mdp_apps.ipynb
ad71 Jun 22, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Added docstrings
  • Loading branch information
@ad71
ad71 committed Jun 6, 2018
commit 5a79debe9c453f9ebdcc62f2ac3fdffb698f3e87
12 changes: 12 additions & 0 deletions mdp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,8 +374,12 @@ def max_difference(self, U1, U2):


class Matrix:

@norvig norvig Jul 11, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure ... maybe we should use numpy.matrix ?

@antmarakis antmarakis Jul 11, 2018

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are to use numpy here, we should consider using numpy on the rest of the repository as well. It would be an interesting undertaking and I am all for it, but adding another dependency is something that needs discussion.

Maybe an Issue can be opened to discuss this when this is more concrete?

@ad71 ad71 Jul 11, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we aren't supposed to use external libraries, but using numpy will make room for a lot of improvements in many algorithms. I think this can be discussed in a new issue as @MrDupin suggested.

"""Matrix operations class"""

@staticmethod
def add(A, B):
"""Add two matrices A and B"""

res = []
for i in range(len(A)):
row = []
Expand All @@ -386,13 +390,17 @@ def add(A, B):

@staticmethod
def scalar_multiply(a, B):
"""Multiply scalar a to matrix B"""

for i in range(len(B)):
for j in range(len(B[0])):
B[i][j] = a * B[i][j]
return B

@staticmethod
def multiply(A, B):
"""Multiply two matrices A and B element-wise"""

matrix = []
for i in range(len(B)):
row = []
Expand All @@ -404,10 +412,14 @@ def multiply(A, B):

@staticmethod
def matmul(A, B):
"""Inner-product of two matrices"""

return [[sum(ele_a*ele_b for ele_a, ele_b in zip(row_a, col_b)) for col_b in list(zip(*B))] for row_a in A]

@staticmethod
def transpose(A):
"""Transpose a matrix"""

return [list(i) for i in zip(*A)]


Expand Down