1. TRPO performs better than the other baselines used on **GPL**.
2. TRPO sometimes outperforms the other baselines on **EFC** & **HLR** and vise-versa.
3. TRPO performs worse that threshold policy (this was described as unsurprising as threshold has an access to the latent parameters).

The authors claimed that the fact that TRPO sometimes performs worse that **EFL** and **HLR** is interesting. Thy state that additional hyperparameters  and policies can increase TRPO performance.

San Diego State University

1. Research Question
2. Environment
3. Baselines
4. Implementation details
5. Analysis

Experiments
(Accelerating Human Learning With Deep Reinforcement Learning)

**EFC** (Exponential Forgetting Curve), **HLR** (Half-Life Regression) , **GPL** (Generalized Power Law)  models are implemented as OpenAI Gym. environments.

Environments
(Accelerating Human Learning With Deep Reinforcement Learning)

These are the parameters that were used by the authors:
1. n = 30 (number of items)
2. T = 200 (number of steps)
3. D = 5 (delay between steps (seconds))
4. For **EFL** (exponential forgetting curve)  sample item difficulty ($\theta$) is from the distribution:
$$log\theta \sim  N log(0.077, 1)$$

5. For **HLR** (Half-Life Regression) :
$$ \overrightarrow {\rm \theta} = (1, 1, 0, \theta_3 \sim N (0, 1)) $$
$ \overrightarrow {\rm x_i} = $  (num attempts, num correct, num incorrect, one-hot encoding of item i out of n items).

6. For **GPL** (Generalized Power-Law) 
$$a = \overrightarrow{\rm \alpha} = 0; d \sim N(1, 1);$$
$$ \log \overrightarrow {\rm d} \sim N(0,1 ), \log r \sim N (0, 0.01); W = 5;$$
$$  \theta_{2w} = \theta_{2w - 1} = \frac {1}{\sqrt W - w + 1} $$

7. For TRPO batch size is 4000, $\gamma = 0.99$; step_size = 0.01
8. For Recurrent Network Policy number of hidden layers is 32

Implementation Details
(Accelerating Human Learning With Deep Reinforcement Learning)

**Can described DRL scheduling algorithm help students achieve their educational objectives better than baseline methods, under various assumptions about student learning? **

Research Question
(Accelerating Human Learning With Deep Reinforcement Learning)

Analysis
(Accelerating Human Learning With Deep Reinforcement Learning)

The authors compare TPRO to other four baseline schedulers:
1. Random Policy
2. Leitner System
3.  Variant SuperMemo
4.  Threshold-based Policy

As it was described in the research paper a threshold-based policy has direct access the student simulator's parameters when it calculates recall likelihoods, and it was used as an upper bound in the authors experiments.

Learn Before

Related