Introduction
There’s a good deal of buzz about using games and gamification to make elearning more fun and engaging. But how do you design an elearning game that goes beyond fact-based recall to have a deeper impact on your organization? Let’s take a look at an elearning game that Plum eLearning developed for our partners at Google, which recently won two Learning Technologies Awards and three Brandon Hall Group Excellence Awards .
Google’s challenge
Google’s Security Education team wanted to reduce potential software vulnerabilities through use of a process called Threat Modeling. Threat Modeling is an industry practice where software developers and project managers analyze the flow of data through products and services, creatively thinking about what could go wrong and coming up with mitigation strategies.
The “traditional” approach didn’t work
Like many of us would do, the Google team started their mission by recruiting facilitators to conduct instructor-led training (ILT) sessions. They quickly realized that this approach was limited, for several reasons.
- Scalability. Coordinating dozens of training sessions for hundreds of engineers across the country proved to be a daunting task.
- Facilitation skills. This would probably not surprise experienced trainers, but it turned out that subject-matter experts (SMEs) recruited as facilitators weren’t necessarily capable of producing the kind of robust, collaborative discussion that a productive Threat Modeling session requires.
- Participation. Participants weren’t always taking ownership of the brainstorming process, but rather waited to be told the answers. Or, one or two people might dominate the discussion while others stayed silent.
Pivoting to virtual training (VILT), as so many organizations did during the pandemic, could help with scalability, but it wouldn’t improve the quality of the threat models produced during these sessions.
Gamification to the rescue!
The Security Education team came to Plum eLearning with a proposed solution: they had designed an online game and wanted our help in developing it. In the game, teams would have to develop their project while also fortifying their defenses against common threats and vulnerabilities. Just like in the real world, only more fun!
How the game works
Players
Teams consist of two to six (or more) players, with each player selecting their role, such as “Project Owner”, “UX Expert” or “Software Engineer.” Each player is given an avatar.
Teams
Players choose a team name and bring their own, real-world project to the game (usually a software product or feature). The game tailors itself to the project based on a quick questionnaire. For example, larger projects must attain a higher score threshold to succeed.
Mission
The goal of the game is to build a product on time, represented by the colony base (shown on the left), while defending the base from enemies.
Enemies
There are four types of real-life threats against which the base must be defended.
Key task #1
A turn represents a week until the colony’s launch. On each turn, the team must decide how to allocate limited resources, represented by bots. Bots can be spent building the base (i.e., the project), or strengthening one of the four defenses against threats.
Key task #2
For each bot deployed, players answer questions about possible threats and brainstorm possible defenses or solutions to the threat. The game uses the player registration information to call on appropriate players in order to ensure participation from all team members.
Victory or failure
When the turns are up, the game uses an algorithm to evaluate the resource allocation decisions and action items entered to determine if the defenses hold up against the threats, or if they are breached. One key feature of the game is that teams can choose to delay launch (i.e., add more turns), if they feel they will be unprepared for attacks. This simulates the real-world decisions project teams must make against pressure to launch software with potential known or unknown vulnerabilities!
Leaderboard
Team names and data are logged in a leaderboard to provide additional motivation through healthy competition.
Real world impact
Before exiting the game, teams are encouraged to export notes and action items for follow up.
Why the game’s design is effective
Gamification theory
In his excellent (and free) Coursera course, Professor Kevin Werbach of the Wharton School defines gamification as “the use of game elements and game design techniques in non-game contexts.”
- Game elements, as you can easily guess, include things like points, resources that players can collect, missions, rewards, even avatars to represent players.
- Game design techniques involve thinking about the visual experience, the player journey, and the game balance (not too hard, not too easy).
- And of course, non-game contexts means that the game exists so that players can practice some sort of real world skill, as opposed to just playing games at work.
Applying gamification theory
Based on this framework, how did Google and Plum eLearning gamify Threat Modeling?
We’ve already learned about the non-game context, which of course was teaching software engineers to do threat modeling. But they don’t just do a simulation or use a fictional software project in the game; instead, teams bring their own real project into the game, and at the end can export the notes and action items from their discussion in order to fix real-world bugs and vulnerabilities. This feature was so crucial that an enhancement was added to export these action items not just into a Doc, where they might never get addressed, but directly into Google’s bug-tracking software, where they are very, very difficult to delete.
The Threat Model game includes common game elements. Teams get a team name and players get avatars, and there is a leaderboard displayed at the end so that they can see how they fared compared to others. These elements were carefully designed to increase the fun vibes of playing the game, but it is the mechanics of the game that make it not only engaging, but effective at simulating the competing priorities and challenges facing a real-world software development team. During each turn, the team must choose how they allocate their limited resources. They must build the project itself, but they also have to invest in defenses against security threats such as user abuse, or hackers.
Finally, we can examine the game design techniques reflected in the final product. The concept is based on a loose metaphor: players are Mars settlers who need to establish a secure base. Visual graphics and animations enhance this theme yet can easily be re-skinned to keep things fresh for teams that play multiple times. The scoring algorithms are carefully balanced; in order to thwart enemies, players must allocate their bots wisely. If they don’t invest enough in each category, their defenses will fail. And if they don’t invest enough in the colony itself, the project will fail to launch at all. The game also supports the player journey by onboarding teams with some basic instructions and a tutorial.
Addressing business challenges
Lastly, the game is effective because it addressed each of the issues that were presented with the ILT.
- Scalability. Teams can play the game any time they can schedule a couple of hours together, and there will no longer be any travel costs.
- Facilitation skills. Facilitators are no longer needed; the game walks players through the threat model process step by step.
- Participation. Players register so that the game can prompt each player, balancing participation with questions relevant to their role.
Results
The game was an immediate success. Within the first 90 days of launch:
- 4,200 players from 570 project teams at Google played the game
- 40% of teams extended gameplay in order to continue their threat model sessions
- Almost 3,000 action items were generated to decrease or mitigate real-world software vulnerabilities
After one year:
- 8,720 players from 1,744 project teams at Google played the game
- 40% of teams extended gameplay in order to continue their threat model sessions
- 14,424 actions items or bugs were generated to decrease or mitigate real-world software vulnerabilities
These metrics suggest that players aren’t just learning about threat modeling as a concept, they’re truly putting it into action and making their products, services, teams, departments, and the entire company more secure in the process.
Is gamification right for you?
If you want to consider gamification for your next employee learning experience, remember: just playing games at work isn’t gamification. Gamification as “the use of game elements and game design techniques in non-game contexts.” Ensure that you are incorporating solid game design principles, including linkage to real world tasks, balanced challenge, visual elements, and managing the player journey.
For a free consultation to see if gamification is right for your project, contact Plum eLearning.