Results: Runtime

Experimental Set-up

• Mean value is taken over 1,000 episodes
• Each data point is indicated by 'o'

Results

First 1,000 milliseconds

First 1,000 milliseconds (logarithmic scale)

Results: Horizon 100 Tiger Problem

Experimental Set-up

• Mean value is taken over 1,000 episodes
• Error bars represent standard error
  

Results

• With random baseline



First 5,000 samples

 











 

• Without random baseline



First 5,000 samples

Results: Horizon 10 Tiger Problem

Experimental Set-up

• Mean value is taken over 10,000 episodes
• Error bars represent standard error
  

Results

• With random baseline

First 10,000 samples

First 1,000 samples

• Without random baseline

First 10,000 samples

First 1,000 samples

Feedback 23-06-12

Progress

• Computed maximum average reward that the random agent could achieve
• Updated performance graphs (see below)
• Implemented leaf visualization (see below)

Results

First 1000 Samples

First 100 Samples

Visualization of Leafs

Feedback 17-04-12

Progress

• Implemented flat Monte Carlo (uniform sampling for first action choice)
• Experiments can now be run in two ways:
• With graphical output (using RL-Viz, see below)
• Or with plain text output (using .properties files for the settings)
  
• Added RL-Viz to support:
• Simple resetting of parameters
• Simple selecting of agent and environment
• Visualization of agent and environment

  

Feedback 13-04-12

Progress

• Removed unnecessary information and functions from observation tree (most of them weren't used anyway)
• The computation of the belief state update is correct (I just read a number wrong)
• Changed from separated runs to just doing more episodes (10,000) for the experiments
• Belief and action trajectories can now be shown for each episode

Thesis Structure

Preliminary Results

Horizon 2 Tiger Problem

Horizon 10 Tiger Problem

Thesis Structure