https://ieeexplore.ieee.org/document/8948025

San Diego State University

1. Introduction
2. Related Work
3. Methodology
4. Results
5. Discussion
6. Conclusion

Analysis of the Factors Influencing Learners’
Performance Prediction With Learning Analytics

MOOC datasets have been used to predict academic performance or student dropouts. Academic failure rate is especially high for MOOC as only 25% of highly committed users finish the course.  For this reason, several researches focused to provide emails or feedback for those students who are at risk. The issue with these models is that they are highly context-dependent - they are only applicable for the particular environment. Knowing the features that influence academic performance the most can solve extrapolation problem. As Moreno-Marcos suggests sometimes data selection can be more important than the algorithm itself. Forum data, clickstream, course duration and setup, assignments, final exams are among the factors that can be beneficial for this purpose. In this paper the authors have 4 research questions:

RQ1) Which factors, among previous grades, forum variables, course duration, type of assignments and data collection procedures, influence grades predictions and to what extent?

RQ2) How does the presence/absence of clickstream data and interactions with exercises affect prediction results?

RQ3) How does the question format of the final exam (close-ended and open-ended questions) affect prediction results?

RQ4) Is the predictive power greater for predicting the final exam grade rather than the final grade of the course?

Introduction (Analysis of the Factors Influencing Learners’ Performance Prediction With Learning Analytics)

Based on previous works, in dropout prediction task CNN combined with DNN have outperformed other models like Random Forest, SVM. Even though dropouts are important, even more important are the students who do not drop out but fail the course. In order to forecast at-risk students models will typically predict exam or assignment grades. Actually there are few studies that have focused on predicting final exam grade. Features are one of the most important factors for developing model with reasonable accuracy. While student demographics can be good features to add to the model, they might have low predictive power compared to student interactions. It was found that one of the strongest predictors are exercises in the MOOC. Apart from that, from where the data comes from can influence the model performance. Additionally one of the previous studies achieved good performance on predicting certificate earners and assignment grades solely based on demographic data. 

Related Work (Analysis of the Factors Influencing Learners’ Performance Prediction With Learning Analytics)

In this paper data from two MOOC courses hosted in edX are used: Introduction to Programming with Java - Part 1: Starting to Program in Java. and Introduction to Computing with Java. The characteristics of these courses differed like duration for the first one was 5 weeks and for the second one 10 weeks. The features that are available in the dataset can be grouped into course  features, forum features and clickstream. The authors wanted to predict pass/fail rates and exam grades (on 0-1 scale).  They used different models for this purpose:
1. Regression 

2. Support Vector Machines (SVM)

3. Decision Trees 

4. Random Forest 

Methodology Analysis of the Factors Influencing Learners’ Performance Prediction With Learning Analytics)

In order to answer their first research question(influence of previous grades, course duration, type of assignment ,forum variables and data collection procedure), the authors have developed several models with different features and found that previous  grades were the strongest predictors and predictive power improves over time. Additionally, forum variables have relatively low predictive power. Regarding the second research question (effect of clickstream data and variables related to exercises on prediction) it was found that the clickstream data doesn't give the models better predictive power alongside with courseware data and only analyzing courseware data can be enough. In case if exercise data isn't available, clickstraem data can be useful. The next question was related with how the type of question can affect the final exam performance, like whether the questions are close ended or open-ended.  It was found that predictive power of open ended questions were lower compared to multiple choice questions. Final exam and final grade are highly correlated (0.95) and it was found that average knowledge in the long term is more difficult to predict than actual knowledge at the specific moment.

Results (Analysis of the Factors Influencing Learners’ Performance Prediction With Learning Analytics)

It was found that the strongest predictors were features related with exercises and previous assignment grades, this is consistent with the finding of the previous papers. Forum data had low predictive power which can be explained by the fact that only small portion of students actually make posts. This means that forum data can be useful if and if only certain conditions are met. Additionally, it was found that predictive power for the first weeks were worse. 

Learn Before

Related