Researchers develop an AI-powered ‘Learning Map’ to comprehensively assess a student’s academic performance

Nowadays, personalised experiences are expected in almost every aspect of our lives, from streaming recommendations to fitness plans. Yet the same importance is not given to one of the most important aspects of life—education. In most educational systems, a student's progress has largely been distilled into a single letter grade or a numerical average. While these metrics offer a quick and easy way to assess a student’s performance, they notoriously fail to capture the rich diversity in how each individual learns and understands. This limitation becomes even more pronounced in the vast landscape of online learning, where engagement and tailored support are critical yet often elusive. 

"To a man with a hammer, everything looks like a nail", said Mark Twain about solving problems with limited tools.

The core problem, as researchers of a new study highlight, is that traditional grades are a blunt instrument used to assess multiple parameters of learning. They don't tell us whether a student truly assimilated the material — meaning they've connected new information with their existing knowledge in a meaningful way — or if they simply memorised facts for a test. This distinction is crucial, especially when considering the diverse ways students engage with complex topics. What one student grasps deeply, another might skim, and a third might connect to an entirely different concept.  Educators need to see not just what a student knows, but how they know it, and where their unique interests lie.

To address this,  researchers from the International Institute of Information Technology Bangalore (IIITB) and Gooru Learning Inc., USA,  have developed the Learning Map, a novel AI-powered system designed to improve our understanding of student assimilation. The Learning Map aims to make the individual assimilation patterns among students visible and actionable.

At the heart of this innovation is a new data structure called a Polyline, which provides a multidimensional fingerprint of a student's understanding across all the topics in a course. Instead of a single grade, a Polyline shows a score between 0 and 1 for each topic, reflecting how well a student has assimilated that specific concept. This creates what the researcher calls a Jagged Profile for every learner, acknowledging that no two students learn in exactly the same way or achieve the same level of understanding across all dimensions. For instance, a student might excel in Network Centrality but score lower in Graph Theory, a nuance completely lost in a single overall grade.

Creating Polylines starts with a constructivist learning approach. Instead of just multiple-choice tests, students are encouraged to write summaries and essays, and to present seminars. These activities require students to articulate their understanding in their own words, providing rich textual data that reflects their genuine assimilation. This textual content, from student submissions to course resources and topics, is then fed into an AI model.

The AI process involves several steps. First, for each course topic, a detailed description is generated, often with the help of large language models like ChatGPT. Then, for all learning entities, like topics, resources, and student submissions, the textual content is converted into numerical embeddings. These embeddings are like unique digital codes that capture the meaning and relationships of words and sentences. Once these numerical representations are ready, the system calculates a similarity score.  A high similarity score indicates that the student's understanding aligns well with the topic. These similarity scores, ranging from 0 to 1, form the individual Polyline for each student. For students with multiple submissions, a Highline is computed to represent their highest achievement across all topics. The Learning Map tool was then developed to visualise the multi-dimensional Polylines.

The researchers evaluated their Learning Map by comparing its performance with other models, using metrics such as trustworthiness, continuity, and rank correlation. While the Learning Map showed moderate scores in preserving overall local structure compared to these methods, its unique strength lies in explicitly preserving the progression property, which is paramount for understanding learning trajectories. They also validated the Polylines themselves through expert evaluation. Subject matter experts rated student essays for relevance to the topics, and the model's agreement with their ratings was assessed. The model achieved a fair level of agreement, even outperforming some large language models in this specific task, demonstrating its reliability in capturing assimilation.

We have long known that traditional grading systems are too simplistic. Learning analytics dashboards often show activity data but lack insights into deep assimilation. Intelligent Tutoring Systems (ITS) can be highly personalised but are often domain-specific and computationally intensive, making them hard to scale. Concept Maps, another visualisation tool, offers interpretability but lacks scalability and automation. The Learning Map, by contrast, offers a scalable, automated, and interpretable solution that focuses directly on visualising individual assimilation patterns.

However, the reliance on student essays as proxies for assimilation, while providing rich data, can introduce biases related to language proficiency or cultural background. The expert evaluation, though rigorous, was limited in scope due to the specialised nature of the course. Furthermore, the system has not yet been deployed at scale in real-time classroom settings, and some human intervention is still needed for fine-tuning hyperparameters. The team intends to explore integrating even more advanced language models and to conduct long-term deployment studies to validate their impact further.

For students, Learning Map offers unprecedented transparency into their own learning journey, empowering them to take ownership of their education, identify their strengths and weaknesses, and pursue their interests more effectively. For educators, it provides actionable insights, enabling them to tailor teaching strategies, offer personalised feedback, and design targeted interventions for students who are struggling. By shifting the focus from a single average score to a rich, multi-dimensional understanding of assimilation, this research paves the way for a truly personalised and effective educational future.

This article was written with the help of generative AI and edited by an editor at Research Matters.