Score Predictor
The Challenge
We wanted our students to have a good idea of whether they are going to pass their final exam, where they stand performance-wise in the course, and in comparison to their peers.
The Context
Since the ultimate purpose of purchasing our courses, is to pass an exam and obtain a professional certification, we believed our students would greatly benefit from knowing how ready they are for that final exam with an accurate score prediction.
We also need to stay competitive, up-to-date and innovative with the latest educational technology to surpass our competitors and their offers. Our customers are individuals who want to either change their careers or advance in their field.
The Solution
We designed a Score Predictor modal, which later became the first and most relevant part of the Dashboard/Home Page. It offers students complex and accurate predictive technology that gives them insight into their and what they need to work on to pass the exam based on performance data.
The idea was to calculate a statistically accurate score prediction and visualize it in a way the student can easily understand their actual versus expected performance along with their predicted score. In order to provide an accurate prediction, we utilize a significant amount of performance data, we collect form 50% to 75% of total lessons and assessments and actual exam data first before delivering score predictions, we also render these predictions in the form of a range instead of a finite number to maintain statistical accuracy.
The Process
This was a great data visualization challenge, because I had to display three different variables on the same graph, sometimes overlapping each other, and a range. When I tried to find graphs that were a good example for this particular case, I couldn't find any, so I created it myself.
Tutorial Page
Since this is a complex and predictive tool, it can create uncertainty among some students, because a prediction is appealing but not always reliable, so we decided to give students a short tutorial with information about how the technology behind it works, how their predicted scores are calculated and why, and introduce them to each variable in detail, so they understand the graphs before seeing them. We also made this tutorial available at all times and easily accessible from the predicted score widget.
It takes completing many lessons and assessments in the course for predictions to activate, since enough data needs to be analyzed in order to form an accurate prediction. In the meantime, we want users to be able to look forward to this feature while encouraging them to complete more of their courseware to activate it, which is why this feature is present during the entire course.
Final Exam Predictor
The first score prediction is for the final exam, which tells the student how likely they are to pass their certification exam. For this prediction we decided to use a normal distribution graph because exam scores are normally distributed, with most of the students getting a median score, and few scoring either on the high end or low end of the spectrum.
I then had to find a way to represent their prediction over the bell graph while avoiding complexity and confusion for the user. It took me a couple of tries but I arrived at a clear, simple and aesthetically pleasing version. We also added the average score so the user knows what grade most students achieved in the final exam and how their performance compares. This is a data point that changes every year and varies by certification due to recurrent additions and updates to the study materials.
The team decided that the only way to make this prediction accurate and not a faulty guess, was to wait until the student has completed 75% of the course to analyze enough data to render a reliable prediction. We don't want students to expect a score and get a very different result when they take the actual exam. Initially we wanted the icon for this predictor to be a crystal ball, but then decided against it because we don't want this tool to be associated with "magic", we want it to be associated with accuracy and data.
Topic Score Predictor
Another feature we decided to deliver is Topic Score predictions, predicting how the student is going to perform in every topic. During research, students have expressed several times they want to know what their weak areas are so they can improve them, this feature satisfies that need. Topic Score predictions are made when >50% of the topic has been completed for optimal accuracy.
We also offer students the ability to see their actual topic score here, which is the average score the student obtained from aggregating all the lessons and assessments that belong to one particular topic. So we decided to give them the option to see one data point at a time, so they can see each score type on their own or both for comparison.
We also made it possible for the syllabus to be accessed from this widget so that students can go to their coursework directly from this widget after seeing their performance metrics.
Early Versions
Below are some early versions of prediction graphs, they were difficult and fun to create because multiple variables have to be displayed simultaneously on the same graph and at times they can overlap each other, so I needed to find a design that was not busy, too colorful and hard to quickly understand.
Topic Score Predictor Versions
Exam Score Predictor Versions
