When you’re looking to buy a new learning technologies product (whether that’s an authoring tool, LMS, or something else), gaining a real understanding of its functionality and quality can be difficult. This is especially true of xAPI implementations. The product may say it implements xAPI, but what does that mean?
For instance, when considering a learning technology product ask yourself:
Does it generate detailed transcripts of learner interactions, or just the bare minimum record of completion?
Is the data accurate, and has the tracking been well tested for bugs?
Can you easily get your data out of the product and into a Learning Record Store (LRS) such as Watershed? Or, is the data locked in and irretrievable—defeating the main point of using xAPI in the first place?
We want to make sure anyone who uses Watershed is equipped to answer these questions so they can more easily report on their learning data in Watershed. That’s why we created a Certified Data Source program, where we test products against strict criteria, which we’ve explained below.
You also can find the full list of products that have met our criteria on our xAPI Data Source Directory. Almost all the products we’ve tested have failed at least one of these criteria during the first round of testing, but they’ve made fixes and improvements to become certified—which underlines the importance of this process.
Certified Data Source Requirements
1) Is the data detailed?
Certified data sources are required to send a good level of detail about learner interactions. What this means in practice varies depending on the type of tool. For example:
For authoring tools, we expect data about not just completions and scores, but also full details of question interaction—including the question text, available options, and learner response.
For a platform such as an LMS, we expect data about learner interactions within the platform, not just the ability to pass on data from hosted xAPI courses.
For a video platform, we’d expect to see data about which parts of the video were watched, not just the fact that it was watched.
These criteria doesn’t guarantee that the product will meet the data requirements of your specific use case, but it does ensure a good foundation that will cover many scenarios. We recommend discussing your specific data requirements with both your LRS vendor and the product vendor.
2) Is the data accurate?
As part of the certification process, we test the product with Watershed and look for bugs and errors in the data. For example, if I score 100% on the quiz, but the data says I failed, then something needs to be fixed.
We also check the xAPI statements to make sure the right properties are used for the right data, and test the data with relevant Watershed reports. The review checklist we use is outlined in our How to Implement xAPI Guide.
Again, we can’t guarantee that a product’s xAPI tracking is bug free, but at least you know it has been tested and any issues found have been addressed.
3) Is the data portable?
Too often, we hear that an LMS that claims xAPI support does not offer any way to get the xAPI data out of the LMS and into an LRS. This can be especially frustrating, given the whole point of xAPI is interoperability and freedom to migrate the data.
All our Certified Data Sources provide an easy way to integrate with Watershed so the data flows directly to your reports in real time. Sometimes this means a simple configuration page you can access, other times it’s a quick email exchange between us and your product vendor. Either way, integration should be quick, easy, and pain free. (But this isn’t always free. Some Certified Data Source vendors do have costs associated with enabling their xAPI implementation.)
Getting Started with Data Sources
To get started, head over to our Data Sources page to see the products we’ve already certified. If you don’t see a product you’re interested in, we may not have had a chance to test it yet. So, let us know and introduce us to your vendor contacts to start the process.