Dashboard
The Challenge
Creating a dashboard that helps students easily understand their progress and performance, probability of passing, and immediate tasks to complete.
The Context
Our platform is an educational product that contains different types of learning materials that include lessons, videos, flashcards, slides, quizzes and practice exams. It also provides features like scheduling and time management, progress metrics, study modes, forums, adaptive assessments and performance predictors. Our customers purchase our product to learn about a subject matter within a specific time frame, and pass a professional certification exam, to become a certified public accountant, a certified financial analyst, or a certified investment manager, among other professions.
For these reasons, the key goals our users have are understanding the course material, managing their study schedule, knowing how they're performing in the course and how likely they are to pass their certification exam, and what next steps they need to take.
The Solution
I believed this dashboard had 3 fundamental missions: help students understand where they are in the course, where they need to be, and what to do next. So I created a metrics-based dashboard, with performance data visualizations and actionable widgets.
As the main designer of this platform, who designs 80% of the three integrated products (student web, student mobile, professor web), I know how much data we collect and how much we know about the student’s journey, and that’s why I believe I went this route.
Toggle
The first action the student is able to take is to toggle between the old and new dashboard, we decided to do this because in the dashboard survey, 59% of users said they wanted to be able to access only the new version, while 41% said they wanted to be able to switch between the old and new version. We decided to include this feature in the short term, to minimize frustration and confusion for 41% of our users.
1st Widget: Performance
For the first widget, I used three key data points to establish current and future performance of the student in the course. The previous dashboard didn't have the predicted performance data we are now able to provide, it also didn't give users actionable components, only informational ones, if the user wanted to continue studying, they needed to navigate away from the dashboard.
The student is given a widget that has three different states. The first one is the predicted exam score, for the final exam, based on 75% of course completion and quiz scores, the reason we require 75% completion is because we want to have enough data to be able to give students a close prediction and not a guess, it's also why it's a range, because it's not definite.
The second state contains predicted topic scores, which are determined by quiz performance by topic. We created this feature because students requested during research, a way to identify lagging study areas they need to focus on. Since these scores are also predictions, they are ranges, and they populate as students complete each study topic.
The third and last state of this widget is the course grade, which is the current average course score. Initially this is the only active graph on this widget, while the student completes enough of the course to activate the prediction graphs, in the meantime they are in their empty states with information about when they will become active.
2nd Widget: Next Assignments
For the second dashboard widget I defined the most urgent actions the user needs to take, which in this platform are lessons & assessments, based on due date and completion status (overdue & upcoming). Time is a factor that determines success in our platform, because courses need to be completed in a specific timeframe, since these certifications are updated with new content & regulations every year, and every user has a set exam date, each lesson & quiz has a due date, and we use this data to show students the completion status of their assignments.
3rd Widget: Course Progress Widget
Then we have a course progress widget, that displays the percentage of the course that has been completed, measured against the expected percentage of completion by today (green numerical indicator), and a categorical indicator describing whether the student is on track or behind. I also display the amount of time left on the course, and an option to edit the study plan right on the widget, which reorganizes study materials and deadline expectations based on a new exam date and/or study schedule.
Last Widget: Syllabus
The last element of the dashboard is a syllabus that contains all the lessons in the course organized by topic/sub-topics, and latest activity, as the default expanded topic contains the last lesson the student worked on. Topics and lessons cannot be arbitrarily re-organized because in this case, the user has to complete each action (lesson/quiz) in a consecutive order, to be able to understand and complete the following lesson.
However, lessons can be filtered based on performance, and students can see all the ones they need to focus on with one click, whether because they are overdue, or because they didn't reach a passing grade. This was something students brought up during research, they wanted to be able to easily see their weak areas so they can dedicate more time to them, that’s why we added this feature to the syllabus.
Each lesson on the syllabus has an indicator (icon) that lets the user know whether the lesson is complete, in progress, overdue, current, or not started, along with critical information about each lesson like the estimated completion time, its due date and score. We previously had more data points for each lesson on the syllabus, but during research users told us that they felt overwhelmed by several columns of data, that didn't help them understand their performance better, and added confusion to the syllabus experience, so we kept only the critical data points.
The default way the syllabus displays lessons is on a list view, but there are two other options the user has to see their lessons, both are calendar views, one weekly and one monthly. On the weekly view, students are able to see all the lessons that are due that week at once, these lessons also have an indicator that displays completion status. Because we still have enough space, most of the lesson title is displayed with a character limit.
For the monthly calendar view, all the progress of the day’s lessons are aggregated and averaged, to assign each day a completed, in progress, overdue or not started status, through icons identical to the ones on the weekly view, list view, and the previous ‘next assignments’ widget, to maintain a consistent visual language.
The Process
We considered changing the dashboard in parts rather than all at once, but we ran a survey that told us that 62% of our users prefer to see changes all at once rather than gradually, because if we have too many frequent updates, it would cause confusion and users would have to relearn how to use the dashboard multiple times. The survey also told us to clearly communicate with students when we make changes, to provide more guidance and to avoid interrupting their study flow.
Performance Widget Iterations
For the ‘Predicted Exam Score’ section of the performance widget, I considered using a simpler graph to display the predicted final score, but decided against it because this data point is somewhat complex, it compares the user’s performance to other students’ performance in the final exam and to the passing grade. Final exam scores are distributed in a normal distribution (a bell-shaped curve), which means that most students get a score in the middle of the range, and getting an extremely low or high score is highly unlikely. We wanted to visually represent this for students to understand it, so we kept the bell curve graph I had initially designed.
For the 'Predicted Topic Scores' section, we decided to go with bar graphs because we needed to display multiple score predictions in one widget. I considered adding more ‘passing’ and ‘failing’ colors to each graph, to both the bars and the numerical values, but decided against it because the widget was becoming too busy with color.
For the 'Course Grade' section, given that course grade is such a straightforward and simple data point, I decided to use a completion donut with the course grade inside the donut. This also looked more balanced than a bar graph, because a single bar would only take up the middle section of the widget.
I also thought of making the donut graph smaller and adding a short caption describing what the graph means, but decided instead to leave the caption out and keep the widget simple, because the donut and percentage were enough to understand this simple data point.
Next Assignments Widget Iterations
Initially I knew I wanted to have around 5 lessons/quizzes in the ‘Next Assignments’ widget, so students could see their current assignment, overdue assignments if any, and upcoming assignments, to provide some context of the material they’re studying through the surrounding lessons. I then started thinking of indicators that could represent the progress and urgency of each assignment in this widget.
I played around with multiple colors and icons, I realized that for example, if many assignments are overdue, and we used a light red background for them, the widget would look overwhelmingly red, so it’s better to have the red color be on the icons only, and not in the background. Instead I used the background color for the current assignment, a light blue, which is the first assignment due today, that is also not complete or overdue yet, and the one users are more likely to want to go to. However overdue assignments are always above the current assignment.
The colors and icons indicating progress in this widget also had to be the same as the syllabus ones so we wouldn't have two visual systems for progress. I decided to add deadline day information through an icon and caption column on the right of the assignments, to emphasize urgency, since these lessons need to be completed as soon as possible, generally within the current week. If students fall behind and don’t complete the course, they’ll fail their certification exam.
Course Progress Widget Iterations
The course progress widget occupies the full width of the dashboard for now, we will shorten it to add community and gamification widgets in the future. In our case, progress is determined by reading study materials, taking quizzes, grades and time, since the purpose of completing the course is to learn enough about the subject matter within a specific timeframe, to pass the final exam required by the professional institution. The goal of this widget is to let students know whether they are on track, or behind in their studies.
I wanted performance data visualizations to be prevalent throughout this dashboard, so I added a bar graph, numerical and categorical values, as well as color to this widget, so the user fully and quickly understands where they stand in the course. I also added an expected percentage of completion indicator, so they can easily see if they are meeting progress expectations.
How much time is left in the course was a data point from the old dashboard I included in this widget, on the lower right, since it’s an important piece of information because it's a countdown to the final exam date, and it's closely related to course completion and progress. On the lower left, we have an option to edit the study plan that was set in the beginning of the course, in case the user wants to reevaluate their study schedule based on their course progress. Previously, the student had to navigate away from the dashboard to a separate page to do that.
Syllabus Widget Iterations
Creating this syllabus was a long process of determining which data points were meaningful and which ones didn't help the student clearly understand their performance. We collect a lot of data, everything the user does in the platform is collected, so we know how much time was spent on each lesson, how many tries each quiz has, the different scores obtained on each try, how much time is left in each assignment, what the average grade is, and more. There was a time when we used to display a lot of this data and students told us that it was very overwhelming (they said it looked like a bus schedule), and they didn't know what to do with all that information, so we made it a mission to pick only critical data points and show the user only the information they need, instead of most of the information available, sometimes less is more.
One approach I took on the syllabus, was to have only two icon colors instead of three, while maintaining three icons for progress. The reasoning behind this was that it was a simpler color system and it was more aesthetically pleasing. All the designers from other teams and my PM preferred this version, but we soon found out it wasn't the one preferred by users. When we ran the two-color version versus the initial three-color version, we found out that students preferred the three-color version, which represents progress in green, yellow, and red, instead of blue and red.
The reason users preferred the three-color version was because they are used to seeing complete or correct represented in green, yellow representing in-progress, and red representing something negative. And though we didn't expect this result, it made perfect sense to me, because when we look at our environment, both on and offline, for example in signaling like traffic lights, green, yellow and red are used, and this is a very prevalent color system in our lives. Using a different color for each icon/status also makes it easier to differentiate them, without overwhelming the user since there are only three progress statuses.
The Competing Dashboards
We started out with three designs, one from an in-house designer from another team, one from our research agency, and one from me. My version was strongly preferred by users during research.
Our Research Agency’s Version
We have a research agency that specializes in educational products and has experience with a variety of our learning platforms. Our agency went with a topic/timeline approach, in which they displayed overall course data at the top of the page, then a timeline/topic rollup with lessons, completion data, filters, and icons linking out to related content, in a syllabus-style.
My opinion of this approach was that it would have worked great for a syllabus page. Before this new dashboard, we had a stand-alone syllabus page, so I thought this would have been a comprehensive version of that, with general course data at the top, and detailed course data at the bottom. But I didn't think it was a good design for a dashboard, because it looked more like a long to-do list, which I believed would not motivate students, but instead overwhelm them, which is an issue specially keeping in mind this is the first page the user sees. When I envision a dashboard, I see data visualizations, representing meaningful information that can guide users through their journey in a way they can easily understand and take action.
Our In-House Designer’s Version
The in-house designer who created this version currently works in a different business unit but used to work on this product before I did, so she has a good amount of knowledge about it. Her approach was based on actual course score and completion data, visualized through simple graphs, followed by study materials in both video and written forms, plus tips that help students manage their coursework, and links to adaptive (custom) assessments. The score predictor was placed at the bottom of the page, giving low emphasis to the new predicted score tools.
My opinion of this approach was that its data points where good as informational data but not as actionable data. I thought it was somewhat overwhelming because of the various content types and data points displayed at once. I also thought our new predictive tools should be more prevalent on the dashboard, since I believed the likelihood of passing the final exam was a very important piece of information our students want to know, along with their struggling study areas, and this design only had a link to the score predictor at the bottom of the page. This version scored second best during research.
My Version
For my approach, I put together a metrics-based dashboard, with easy to understand performance graphs and top-priority tasks and information. Our platform is very complex and extensive, so I wanted to make this dashboard a summary of the experience that was easy to digest.
This was my first wireframe for the new dashboard and the design that won during research. Initially I thought of displaying only the predicted exam score, which is why this version only has that prediction, but as I refined this version, I realized since students have to complete 75% of the course in order to be able to see their predicted score, we need other data points we can display in the meantime, that can inform students about their progress and performance in the course.
On the top right we have the first version of the ’Next Assignments’ widget. I wanted to have a widget containing the immediately upcoming lessons, separate from the syllabus, at the top of the dashboard, so the student knows what their next actions need to be, right after seeing their status in the course in the first widget. In my mind, the mission of this dashboard, is to tell students where they are and where they need to be in the course, the first two widgets do that.
The next two widgets were informational data points that had been relevant in our previous dashboard, ‘Exam Date’ and ‘Course Progress’, but presented differently now with icons and graphs, previously they were only a number and a caption. The next widget represents a future effort, ‘Achievements’ is badging, which is a gamifying angle we want to add to motivate students, lastly the ‘Community’ widget, would link users to an online forum we currently have, where students and professors discuss academic topics.
For the bottom part of the dashboard I added a simplified version of our current syllabus, because we had a syllabus page with multiple data columns that were shown in research to confuse and overwhelm students, so from the start I knew I wanted the syllabus to have a minimal amount of numerical columns, so students had enough information about each lesson and exam, but not too much so that they would get confused or overwhelmed.
We tested these three designs along with our current dashboard design and users resoundingly preferred my design, from there we began to define and redefine each widget.