Minutes 06-03-2012

D. Silver's paper

• Discussed belief state approximation with particles
• Discussed pseudo code in detail
• Worked out tree construction

Possible Extensions

• Continuous observations
• Store observation and corresponding reward for all roll-outs
• Continuous actions
• How to transform a large number of discrete actions to one (or a few) continuous action(s)?
• Combinations of both

Possible Modifications

• Particle filter could be replaced by (extended) Kalman filter
• Expected reward computation (R_Diff)

Evaluation

• Compare approximate results to exact results (e.g. from J. Hoey [1])

[1] Jesse Hoey and Pascal Poupart. Solving POMDPs with continuous or large discrete observation spaces. In Proceedings on the International Joint Conference on Artificial Intelligence. pages 1332 - 1338. 2005.