A Neuro-Symbolic Feedback Engine for Evidence-Grounded Feedback to At-Risk Students, with an LLM-as-a-Judge

Abstract

Early-warning models can flag students at risk, but they do not say why a student is at risk or what feedback should follow. This study examines a neuro-symbolic pipeline in which a trained model makes the prediction, a symbolic rule layer explains it, and the evidence behind the prediction is checked against the student’s data before feedback is issued. Seven models such as Logistic Regression (LR), Random Forest (RF), Extreme Gradient Boosting (XGBoost), Multilayer Perceptron (MLP), and Long Short-Term Memory (LSTM), Bidirectional Long Short-Term Memory (BiLSTM) and Gated Recurrent Unit (GRU) were trained on 2013 students from the Open University Learning Analytics Dataset and tested on 2014 students, to predict four outcomes (Distinction, Pass, Fail and Withdrawn) for 10,740 students from records up to the midpoint of each module. All models beat the majority-class baseline (Macro-F1 0.455 to 0.508 against 0.170). Random Forest scored the highest, while the MLP was more sensitive to students needing support, identifying 35% of Withdrawn students against 15%. Shallow decision trees fitted to each model’s predictions gave readable if-then rules that matched the random forest for 82% of students and the MLP for 72%, compared with 49% and 36% for the majority class. The rules held steady under resampling and carried over from 2013 to 2014 students’ data, and its features influenced the models about three times as much as random features. A language model then generated feedback for 100 students per model, and each claim was checked against the student’s data. Feedback generated from the students’ values alone contained incorrect claims in 10% (MLP) and 7% (random forest) of messages and supplying the model’s most influential features together with the claim check brought this to 0% for both. The rule alone did not help (28% and 25% incorrect claims), so the gain came from the model’s evidence and the check, with the rules making the model’s behaviour readable and testable. The neuro-symbolic combination therefore produced feedback with fewer false claims. Thus, the trained model paired with rules and claim check provides reliable early warning feedback for students.