But there was one key difference. Half of the students were randomly assigned to a fixed sequence of practice tasks ranging from easy to difficult. The other half received a personalized sequence with the AI teacher continuously adjusting the difficulty of each problem based on how the student performed and interacted with the chatbot.
The idea is based on what educators call the “zone of proximal development.” When problems are too easy, students get bored. When they are too difficult, students get frustrated. The goal is to keep students in a good position: challenged but not overwhelmed.
The researchers found that students in the personalized group did better on a final exam than students in the fixed-problem group. The difference is characterized as the equivalent of 6 to 9 months of additional training, an eye-catching claim for an online after-school course that lasts only five months. The inventor of the AI tutor, Angel Chung, a PhD student at the Wharton School, admitted that her statistical unit conversion was “not a perfect estimate.” (A paper draft of the experiment was published online in March 2026, but has not yet been published in a peer-reviewed journal.)
Still, it’s early evidence that small tweaks—in this case, calibrating the difficulty of practice tasks for the student—can make a difference.
Chung said ChatGPT’s responses can already feel very personal because they directly answer a student’s unique questions. But this level of customization is not enough. “Students usually don’t know what they don’t know,” Chung said. “The student doesn’t have the ability to ask the right questions to get the best learning.”
To address this, Chung’s team combined a large language model with a separate machine learning algorithm that analyzes how students interact with the online course platform—how they answer practice questions, how many times they revise or edit their coding, and the quality of their conversations with the chatbot—and uses that information to decide which problem to serve next.
How different students interact with the chatbot teacher
In other words, personalization is not just about tailoring explanations. It is about adapting the learning path itself.
This idea is not new.
Long before generative AI tools like ChatGPT were invented, educational researchers developed “intelligent learning systems” that tried to do something similar: assess what a student knew and suggest the right next problem. These earlier systems couldn’t generate natural conversations, but they could provide advice and instant feedback. Careful studies have found that well-designed versions help students learn significantly more.
Their Achilles heel was the engagement. Many students simply did not want to use them.
Today’s AI tools can help address this problem. Students may feel more engaged with a chatbot that talks to them in an almost human way.
In the University of Pennsylvania study, students in the customized group spent more time on exercises, about three extra minutes per problem, adding up to about an hour per module in the Python course, compared to half as much time (half an hour or less) for the comparison students. The researchers believe that these students did better because they were more engaged in their practical work.
Students’ prior knowledge of a subject affected how well the personalized sequence worked. Students who were new to Python scored more than those who already had experience with Python, who did just as well on the fixed sequence of practice problems. Students from less elite high schools also appeared to benefit more.
How student experience affects outcomes
All Taiwanese students in this study volunteered to take an optional computer programming course that could strengthen their college applications. Many were highly motivated, with highly educated parents, and many already had prior coding experience.
It’s not clear whether the chatbot will work well with less motivated students who fall behind in school and need extra help the most.
One possible solution: merging new and old.
Ken Koedinger, a professor at Carnegie Mellon University and a pioneer in intelligent learning systems, is experimenting with using new AI models to alert remote educators that can motivate struggling students who deviate. “We’re having more success,” Koedinger said.
People are not old yet.
This story about I have teachers is produced by The Hechinger Reportan independent, nonprofit news organization that covers education. Sign up for Evidence points and others Hechinger Bulletins.
