Create adaptive, dynamic learning content.
Join Frank Pietrantoni—the director of the office of health professions education at Nebraska Medicine/UNMC—as he shares his recipe for adaptive learning that combines a little bit of xAPI and a dash of SCORM, and then mixes it together in an LRS and LMS. In this webinar, you’ll learn how to enhance learning personalization by adding xAPI statements to the SCORM-compliant e-learning your team is already building. This process allows you to create content that adapts to user inputs related to their respective job roles, geographic locations, and/or levels of knowledge.
During the webinar, Frank covers how:
- xAPI can be used in Storyline,
- SCORM modules published to an LMS can send xAPI statements to an LRS and why this is important,
- e-learning can be more adaptive to people’s input, and
- mobile technology can be used to deliver and/or assess training.
Webinar Recording
Q&A
During the webinar, attendees were encouraged to ask questions. Below are the questions and answers.
- Some people don't like, or feel like they have the time, to complete eLearning, so will rush through it as soon as possible. When using the adaptive/dynamic questions, did you find that people looked to answer the questions so they were completing the quickest or easiest route through the module? Did you have to perform some follow-up monitoring to ensure people were choosing the correct answers and therefore completing the appropriate paths through the content?
In long modules that are related to Compliance (e.g. “check the box” driven, such as the one used in the presentation, we realize that many learners go in with the mindset to simply complete them as quickly as possible. That said, we do track time spent and use that to adjust our expectations and/or design accordingly. We do indeed follow up with learners who either didn’t complete the training and/or didn’t complete the appropriate track.
We are a teaching/learning institution, so learner behavior depends on the topic, context, and culture. For instance, the Annual Compliance topic is fairly “check the box.” For more in-depth topics that relate to learners’ work, however, learners take the time in the interest of learning how to better care for our patients. - Did you face any resistance to the amount of personal data you were now collecting about each individual? (This could be resistance from either the learners themselves, Compliance, or upper management.)
Most of the data already comes from the LMS (e.g. name, job role, etc.) either from the learners themselves if they are self-registration users, or from our HR system if they are employees. And we’re already required to report these fields to our accrediting bodies. - Considering that SCORM can do branching, what was the benefit of using xAPI?
We’re not aware of SCORM being able to track and report slide-level activity within an eLearning module. For example, we are tracking/reporting various user interactions within the module at the slide level (i.e. timestamping when a learner clicked a button, started a video, hovered over a hotspot, etc.). We’d be interested to hear how/if SCORM can do this.
Tim Dickinson: While SCORM can support a branching scenario, the data points generated do not typically provide the level of detail necessary in this use case. Using xAPI in this example, we're able to identify for users who selected a certain path (i.e. job role)and how did they perform against users who selected another path. - The recent SL 360 update has incorporated CMI5 xAPI profile. Would that change, or do we still have to incorporate our own custom xAPI statements in SL?
This is something that we’ll need to explore. Thanks for asking and bringing it to our attention.
Tim Dickinson: In the content of this use case, creating and delivering the course as a SCORM package, the customer javascript is still necessary to ensure the file generates the xAPI statements and sends them to your LRS. If you’re in a situation where your LMS supports the CMI5 xAPI profile, then you're in luck and won’t need the custom scripting! - How did you implement your tracking strategy, because you can track almost everything?
For the Annual Compliance Training Module, we determined what was needed to create separate learning paths (i.e. content). We then determined what we would need to report on (i.e. that a learner completed the content on Fluoroscopy). We also wanted to begin exploring how learners handled elective content to help us understand our culture as well as help us to inform our subject matter experts’ expectations. This is the example we discussed where we tracked how many learners clicked one or more of the seven elective buttons on a Code of Conduct Slide. - Beyond lesson completion, what other compliance criteria was determined?
We needed to track the unique learning paths/content learners completed (i.e. role, required content for specific scope of practice) - Ah, your strategy was to use both LMS with SCORM and xAPI data. Why couldn’t you just use xAPI data itself?
Our LMS doesn’t support publishing in xAPI, and we needed (and will continue to need to) publish to our LMS. Additionally, our users are very comfortable running various high-level completion reports from our LMS. Currently, the more granular reports from the LRS were/are run from our office and/or provided to leaders on an ad hoc basis. - What "wizard" that generates your xAPI JavaScript did you use? Was it hard to learn?
We use xapi.ly and worked with Torrence Learning on our statements. They have great resources on their website and are quick to respond to specific questions. The learning curve was pretty minimal to start utilizing it. - What data are you sending for specialized slides?
Along with the learner’s demographics, we are sending data related to what specific buttons are clicked on specific slides (e.g. the verb “interacted” for each button). On some modules we have begun to track the video “play” button in relation to the “next” button of that slide to calculate approximate time spent on that video/slide. - You also got feedback from the LRS to provide personalized learning?
We personalized learning from the start based on our learners’ roles, locations, and specialties. We didn’t use reports/predictive analytics from the LRS about specific questions (we didn’t have questions in our modules), but we use that feature for other workflows. We mapped the learners’ pathways (button interactions) with their identifiers in the LMS. - Holy cow! How can you have a Var with that length! (e.g. "response" : GetPlayer().GetVar("Whe.....");) I don't recall that Storyline can retrieve Var of slide titles? Is that so?
The xapi.ly wizard is helpful to generate this code. The response field is to be used to work with a variable that has been added to Storyline files. We included the slide title in the BUTTON name. We have also revised our code and stop using that field since we initially started. - Did this application use bookmarking?
The LMS bookmarks the user place. - How does this differ from branching activities within the modules? Do you also use your LMS to identify audiences and identify who is eligible for each module?
We are beginning to explore using pathways based on learner knowledge/proficiency, experience, and other unique attributes beyond role and geographic location. Currently, we are also looking toward ways we can use our LMS for more dynamic assignment of training (again based on unique and/or group identifiers). - Good examples of sending info to an LRS from a course. Are you pulling data from the LRS back into a course if someone exits and comes back? Are you pulling LRS user data into a course to know what their role is without them having to click a button each time?
We are not pulling data from our LRS into other systems yet. We do, however, send some specific data (related to microlearning completed) from the LRS to another third-party vendor for the administration of CMEs to physicians. - Is there any protection to ensure that the xAPI statements going into Watershed are only coming from a valid source
Watershed only accepts statements from our LMS based on secret and key codes. - Could you explain how the LRS works with an LMS? Does your LMS have to have this capability or can they be separate?
Currently, our LMS doesn’t allow for modules to be published in xAPI. Thus, we publish our modules in SCORM and include the xAPI programming so that xAPI statements are sent from the SCORM Module on the LMS to our LRS (which is a separate vendor) concurrently. - Are you planning on going back to "beef up" xAPI statements using optional components of statements such as context, grouping, etc., to capture more relationships between the data/objects in statements, and hierarchical aspects of the data and objects?
This is certainly something we’d explore as part of our vision to continually improve/optimize our educational products/services to the organization. - What is the difference between this use of xAPI and adaptive platforms such as Ottolearn, Cerego, or Area9? From a design standpoint, I would be interested in the capabilities of xAPI outside of adaptive platforms. Any intel?
We are not currently using these platforms but look forward to learning more about them.
Tim Dickinson: While acknowledging that we’re not experts in functionality of every platform, most adaptive learning platforms are focused on adapting to knowledge and comprehension gaps to increase the efficiency of learning. In this example, we’re able to use xAPI to adapt based on additional traits, such as job role and location. Additionally, xAPI can be used to adapt experiences based on data from other platforms and sources outside of the current course. - Does the data you're collecting become more relevant in light of COVID-19?
It certainly can and most likely will. We’ll look for opportunities to improve the education, training, and assessments of that education/training associated with COVID-19. In future organizational training, we anticipate including content specific to colleagues that are involved in the care of our COVID patients and/or working in those units. - Learning Paths - Can you go to a different Storyline course and maintain what you are doing, or do the pathways have to be within one Storyline file?
We don’t have this capability in our LMS at this time, unless it’s a video within the same module. But there isn’t a capability for us to trigger a button to open a different course/module. - Are you using xAPI to track any events that are happening during ILT or in offline activity versus online training? Could you talk more about that?
We are exploring this with orientation activities and competencies (observational assessment). We are also piloting this with “in the moment” coaching. An initial discussion could trigger an employee learning plan, with employee self-assessment afterward. - How much are you paying for an LRS?
Pricing is going to depend on the number of licenses, contract terms, and conditions, etc. We recommend that you contact the vendor(s) you are interested in to begin that discovery…even if it’s at a budgetary level to start with. - Do I understand correctly that question responses and/or text entry answers can be sent to Watershed?
This is correct. In various workflows, we collect both quantitative and qualitative data and send it to Watershed. - Just to clarify in terms of how the data is being extracted in the way you presented. Course --> LMS --> LRS --> visualization tool?
This looks to be correct. For the examples that we showed, we created the modules using Articulate Storyline 360. We published those as SCORM modules and loaded them to our LMS (Cornerstone). SCORM data stayed in the LMS while the xAPI data went to our LRS (Watershed). We then run reports out of both the LMS and LRS as needed. Both systems have the capability to create the visualizations (e.g. charts, graphs, etc.) of the data that’s within their systems.
About the Presenter
Frank Pietrantoni is the director of the office of health professions education at Nebraska Medicine/UNMC, and serves as an adjunct professor at Bellevue University. Frank has a BA in leadership and an MS in organizational performance. Frank has worked for Nebraska Medicine for 28 years in several managerial/leadership positions, as well as an organizational development consultant, an eLearning lead systems analyst, and a Six Sigma Black Belt. He's taught numerous professional development workshops and was a certified Achieve Global Instructor.