Evaluation (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
In order to evaluate above-mentioned experiments the authors used different statistical methods:
- Statistical analysis when number of items is varied
The distributions of the means of the differences in the rewards were created between all student models. After creating distributions Shapiko-Wilk Normality test was applied in order to find out whether the distribution was normal or not. As the assumption about homogenity of the variances turned out to be wrong, Welch two-sample test was used to measure the means of differences in rewards. Finally Games-Howel test was used.
- Statistical analysis of comparison between TRPO and TNPG
In order to measure difference between TRPO and TNPG rewards from all episods and runs were utilized. To determine the difference between these two Kruskal-Wallis test was used.
- Statistical analysis of comparison between likelihood and average of sum of outcomes
Kruskal-Wallis test followed by posthoc Dunn’s test with Bonferroni correction was used to compare DRL agents with likelihood and average of sum of outcomes reward functions.
- Statistical analysis of the performance of TRPO with LSTM
Kruskal-Wallis test was used to evaluate the difference various tutors were LSTM was used for reward shaping. After this Dunn's test with Bonferron correction was applied to measure stochastic variance. Apart from that, LSTM was compared to other methods used in DRL agents.
0
1
Tags
Data Science
Related
Experimental Setup (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
Reward functions and performance metrics (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
Training the LSTM (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
Relation between rewards and thresholds (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
Performance of RL agent when the number of items are varied (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
Performance of TRPO vs. TNPG algorithms (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
Performance of TRPO with reward shaping (research objective) (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
Comparison between likelihood and average of sum of outcomes based reward functions (research objective) (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)
Evaluation (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)