StudentSim is a research system that trains personalized AI student simulators from publicly available, de‑identified learner records. Each simulator models how a specific student answers tasks and how those answers change after receiving tutor guidance. The system is intended to provide early, scalable feedback during AI tutor development; it is not a classroom product and is not designed to replace teachers.
Motivation
Evaluating an AI tutor requires knowing what happens after guidance reaches a learner: did the learner fix a mistake, did the hint help reasoning, or did it reveal too much? Collecting such feedback from real learners is slow, expensive and hard to scale, which limits how quickly researchers can iterate on tutor designs. StudentSim aims to partially address this by generating personalized, data‑driven simulations of student behavior that provide additional feedback signals during research‑stage training and evaluation.
Two education‑grounded evaluation objectives
StudentSim defines two complementary evaluation goals informed by educational research:
- Behavioral fidelity: whether the simulator matches a specific student’s recorded responses on held‑out problems.
- Guidance responsiveness: whether the simulator updates its response after reading tutor guidance.
Both signals are needed for tutor training: behavioral fidelity captures the learner’s starting point, and guidance responsiveness captures how instruction affects performance.
Data and training pipeline
StudentSimEval is constructed entirely from public, research‑use datasets with de‑identified student records; no personally identifying information is used. Because individual students often contribute only sparse records (few essays, answers, or interactions), StudentSim uses a two‑stage training pipeline:
- Train on pooled records from many students in the same domain to learn common response formats, typical errors, and how students tend to revise after guidance.
- Fine‑tune the model on a single student’s own records to produce a personalized simulator.
This approach reduces overfitting and avoids relearning common domain patterns for each individual model.
Domains and evaluation setup
The benchmark covers three domains with different response types:
- Chess: predict a player’s next move and how that player may revise after coaching.
- Second‑language English writing: generate essays with error patterns matching a learner and predict corrections after teacher feedback.
- Mathematics: predict a student’s multiple‑choice answer and whether they update after an explanation.
All methods are compared using the same per‑student training records and the same held‑out evaluation records.
Performance against baselines
StudentSim outperforms leading baselines across the three domains. For example, in chess StudentSim achieves behavioral fidelity of 0.5150 and guidance responsiveness of 0.9067. By comparison, GPT‑5.4 reaches 0.2316 (behavioral fidelity) and 0.7186 (guidance responsiveness), while Maia2 (a chess‑specific human move prediction model) reaches 0.4535 and 0.2721. These results indicate that GPT‑5.4 can follow guidance but less reliably matches a specific player’s behavior, whereas Maia2 predicts human moves more closely but lacks a natural‑language guidance pathway.
Using simulated feedback to improve tutors
The research also evaluates whether a trained student simulator can serve as a reward source for tutor reinforcement learning. In a chess tutor experiment, the tutor receives a position and a real player’s previous wrong move, then generates guidance. The student simulator (with fixed parameters) reads the guidance and produces a revised move. Reward is based on improvement in move quality from the prior move to the revised move, measured against Stockfish.
Three training conditions were compared: a supervised fine‑tuned tutor without reinforcement learning (no‑RL), a tutor trained with GPT‑5.4 as the simulated student reward source, and a tutor trained with StudentSim as the reward source. All conditions used the same tutor policy, supervised starting point, and reinforcement learning setup; human chess experts performed blind evaluations of tutor responses.
The tutor trained with StudentSim rewards achieved the highest ratings on reported metrics: 90.5% accuracy versus 75.7% for the no‑RL baseline and 71.6% for the GPT‑5.4 reward condition. It also received higher scores for guidance quality and personalization. These results provide a proof of concept that student simulators trained on real learner data can provide useful feedback for improving AI tutors.
Limitations and future work
StudentSim is not a substitute for human‑subject studies: real learner experiments remain essential for assessing learning outcomes, safety, usability, and classroom fit. The intended role of student simulators is earlier in development: to test tutor strategies, compare models, and refine training signals before deploying costly human studies.
Future extensions could target longer learning trajectories, additional subjects, richer student profiles, and studies that directly connect simulated feedback with real learning outcomes.
Availability
The paper and the StudentSim training and evaluation code, including the StudentSimEval protocol, are publicly available as released by the authors.
(Figures and implementation details are available in the authors’ published materials.)
