In the 21st century MOOC (Massive Open Online Courses) have proliferated like Coursera, Udemy, Edx and several applications like Duolingo, Quizzlet became quite popular.  The researchers have a keen interest to acquire this systems with more intelligence, so that learning process could be overseen. 

As it was described in the paper Computer-Assisted Instruction can increase students performance by 0.3 std while human tutors by 2. 

Intelligent Tutoring Systems combine AI, cognitive science and educational theory which makes researches in this field arduous.  

In the earlier studies 3 main modules have been defined in ITS:

1. **The expert Knowledge module** - this module can be thought as a source of the knowledge. The purpose of this module is to generate questions, answers and the solutions for the particular problem.

2. **The student model module**  - the student skills and knowledge which varies constantly. It is very important for the **tutor module** to know about students current level and skills.

3. **The tutor module** - the teaching strategy. This module has to decide which lesson to present to the student.

Recently, fourth module was added **The user interface module**:



Intelligent Tutoring Systems (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)

In reinforcement learning, the machine goes through trial and error processes where it is rewarded/penalized for the actions it performs. The machine's goal is to maximize the total reward by leveraging previous attempts to make the next decision. The model starts from completely random trials and eventually leads to something with very sophisticated tactics and skills. Compared to supervised/unsupervised learning, reinforcement learning is much more advanced. A reinforcement model will continuously learn, unlike supervised/unsupervised models which all have an endpoint after the training and test data phases.

Reinforcement Learning

1. Reinforcement Learning
2. Intelligent Tutoring Systems

San Diego State University

1. History
2. Related Theory
3. Related Work

Background and Related Work (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)

History (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)

1. Models of Human Memory
2. Spaced Repetition
3. Leitner System
4. Supermemo System
5. Intelligent Tutoring System
6. Relation between Tutoring Systems and Student Learning
7. Reinforcement Learning
8. Trust Region Policy Optimization
9. Truncated Natural Policy Gradient
10. Recurrent Neural Networks

Related Theory (Using deep reinforcement learning for personalizing review sessions on e-learning platforms with spaced repetition)

The attempts to make mathematical functions that would model  human memory have a long history. In the 18th century, Ebbinghaus studied **Exponential Forgetting Curve**.

Then spaced repetition and **Half Life Regressions** have been introduced that were for language learning. 

Currently, RL seems more promising in this task. Several works combined both RL and DL. RNNs have been used for the reward prediction. Additionally, it was found that TRPO and TNPG have the best performance, TRPO having a little bit better results than the TNPG. 
  

Learn Before

Related

Learn After