Learning Analytics: Report Types

    

We’ve looked at categories. We’ve looked at complexities. We’ve looked experience types. There’s only one direction left to slice and dice findings from our Learning Analytics Research Study—which looked exclusively at how clients use Watershed, including report types. In this post, we’ll explore which report types organizations use most often and then compare how these organizations use different report types.

What report types are used most often?

In terms of report views, the Program report took first place. In fact, it was more than twice as popular as the Bar report, which was the next most popular Watershed report type.

The Bar, Leaderboard (which shows simple tabular data), and Heatmap report came next; followed by Pie, Line, Interactions, and Activity reports. The Interactions Report is simply an activity stream of events, while the Activity Report is a specially designed collection of visualizations that provides in-depth analysis of an activity—such as a course, video, or assessment—and performs advanced question analysis to help ensure organizations ask the right assessment questions. Spider, Scatter, Range, and Correlation reports saw very few views.

Learning Analytics Report Types


What's a Program Report?

The Program Report is a type of report unique to Watershed that presents a collection of visualizations designed to give an overview of a learning program’s progress and completions.

Watershed Learning Program Report

Users can drill down into organizational hierarchies to track and compare milestone completions with overall completion percentages across teams, departments, groups, and individual learners.

(NOTE: These report views are relatively concentrated, however, with a single organization representing about 75% of all Program report views. If we removed that one organization from the data, the Program report drops to third place.)


How do people view report types?

When you look at the number of each report type (i.e. Report Count) with their respective views, you can easily see which report types had significantly more or less views compared to others. This shows us that just because there’s a lot of one particular report type, it doesn't necessarily mean that report type also has a high view rate.

Program, Pie, and Spider reports all show significantly more views than you might expect when compared to their respective report counts. These three report types share a common thread among—Watershed clients aim these reports primarily at people managers, rather than the L&D team. As we’ve seen again and again in this series, reports aimed at people managers always get more report views due to the high number of people managers compared to other stakeholders.

On the other side, leaderboards have fewer report views than we might expect given their report count. There could be a few reasons for this, but more research is needed to determine which factor is the most significant. For example:

  • Leaderboards are simple reports showing the underlying numbers and are, therefore, more likely to appeal to the smaller number of people, such as the L&D team, who are especially interested in the data, than to the larger group of people who prefer at-a-glance, visual reports.

  • Leaderboards tend to be the report types used for API data connections (e.g. for use with Watershed's Tableau Web Data Connector). Reports created for this purpose are unlikely to have many views because they aren’t designed for humans.

  • Leaderboards aren’t as visually engaging as other report types, so people are less likely to click on them.

We’re all special snowflakes.

There are some patterns with popular and unpopular report types, but looking client by client, the picture is especially heterogeneous. Each client uses a very different mix of report types, as you can see from the following report type DNA visualization.

Watershed Learning Report Views

You can see from this visualization that each organization generally uses one, two, or—at most—three report types on a regular basis, while the other types are viewed much less often. Report types vary in popularity from organization to organization, but each has its favorites. We noticed this anecdotally while doing the research as well, so it’s great to see the data backs this up.

We found that organizations tend to stick with a preferred report type and use the same report configuration structure again and again. This remained true even when another report type was more appropriate for their data.

Actionable Insights

Don’t get stuck in a rut with a particular report type. Get to know the range of report types that are available to you and use the most appropriate one for each situation.

Next Up: Series Recap

We reach the end of this series in our next post with a summary of what we’ve covered. We hope you’ve found the series helpful in thinking about your next (or first) steps in learning analytics!


Getting started with learning analytics is easy.

Download this checklist to start tracking the analytics you already have in place and where you’d like to go next. Continue your journey by downloading the following eBook, which walks you through the five steps to start using learning analytics.

eBook: How to Start Using Learning Analytics

Andrew Downes

About The Author

As one of the authors of xAPI, Andrew Downes has years of expertise in data-driven learning design. And, with a background in instructional design and development, he’s well versed in creating learning experiences and platforms in both corporate and academic environments.