Guy Wallace, one of my ISPI friends, has been interviewing some of our leaders in human performance technology (HPT). Last year, he sat down with Dick Clark, who is an outstanding researcher in our field. They talked about HPT, cognitive task analysis, constructivism, instructional games, simulations, and ways to view evidence.
Highlights.
I’m sharing my fairly rough notes, with approximate time stamps. These notes should make it easy to see what topics are on the video and where to find them. Thanks to Dick for reading these notes ahead of time and adding some additional thoughts.
2:56 | “Human Performance Technology”
The term itself—Dick is fine with it. Acknowledges that some don’t like the term “technology,” because they think of it as having to do with machines. But “technology” is really about the application of science and experience to solving problems.
4:02 | Gap Analysis
Here we are/here’s where we want to be/here’s the gap.
What’s the cause of the gap?
How do we close it?
We should emphasize gap analysis more and continue to develop our approaches to it over time.
8:20 | HPT in 30 Seconds (Elevator Pitches)
If talking with venture capitalists…
HPT folks have an extraordinary approach to going into an organization to analyze problems in a succinct & evidence-based way—figuring out how to achieve goals, what kinds of strategies or interventions would achieve those goals, and what kind of measures they need to show that the strategies worked.
10: 06 | If talking with students new to the field…
If you’re working—no matter where—you’ll always have clients or other people to whom you’ll be able to give advice or support. Do you have a system, or do you know of one that would allow you to go into any situation where people work, analyze what their goals are, get a sense of what they want to accomplish and where they are, and then reliably find effective, evidence-driven ways to help them achieve their goals?
In other words, you’ll learn how to diagnose problems, benefits, and opportunities. You’ll become highly successful in doing that. And if this is what you want, then an ISPI committed to evidence-based practice is the place to find it, and HPT is the solution that you need.
11:00 | Cognitive Task Analysis
Our cognitive architecture includes limits to what we can think about consciously at one time. Because of this, we make our decisions and solve problems using a part of our brain that stores automated, non-conscious knowledge.
Experts are about 70% unaware of how they do the analysis that they perform and how they make the decisions that help them to solve complex problems.
Story here to illustrate the difference—about chess masters & novices & automated expertise.
Really amazing thing: Chess masters cannot describe how they play the game (because they do it using automated expertise).
14:34 | Implications for Doing Analysis and for Interviewing SMEs
Our use of automated, non-conscious knowledge has implications for everything we do that requires thinking & problem solving.
This is especially important to keep in mind when interviewing SMEs, especially when we consider that most training & education is designed and delivered by experts. Because of this, it is often missing key points that experts can’t access. Thus people who are trained often lack the knowledge they need to succeed when they try to apply what they learned in their work. They have to adjust what they learned using “trial and error”. We need a better way to capture all of the expertise necessary for top performance, and Cognitive Task Analysis will help close the gap between what we need to know and what experts can teach us.
15:40 | Cognitive Task Analysis—Capturing Info from Experts
Dick briefly outlines one CTA method that works well:
- Interview experts one-on-one. Ask them to review how they do something step-by-step. Look for the actions & decisions they use to accomplish goals.
- Each expert is aware of different things. If interview three or four, then usually capture 80 to 90% of the mental analysis and decisions people have to make.
- At this point, do a trial and revise session with a novice.
- After that, go back to the experts for additional info. When prompted with info from the trial cycle, they can usually remember more about what they do.
When using CTA, they found 40% more learning from the CTA group in about 25% less time—even with surgical residents (who often learn well and quickly anyway).
20:10 | Constructivism & Alternatives
Alarmed at how often people use discovery as a way to get people to learn things or solve problems. This usually involves giving people a challenge & putting them in a group & having them solve problems.
There are many names for this, for example, problem-based learning, which is often used in medicine—although in most of its applications it doesn’t work very well.
We have 50 years of evidence that using discovery learning doesn’t work—and almost none that it does. Still, it persists—most likely because we are consciously aware when we have a meaningful insight and so feel that most knowledge is learned by “discovery.”
Alternative: Guided instruction (also called direct instruction). There is solid evidence to support this approach. Guided instruction requires that we demonstrate how to solve problems and accomplish complex tasks with authentic, work-based scenarios and then give trainees a problem to solve using what they saw demonstrated. The problems they solve must be engaging and perceived as authentic. While solving the problems, trainees must also receive immediate feedback about their successes and mistakes so that they do not “learn it wrong.”
22:32 | Authentic Context, Info, Demo, Application, Practice
Cognitive load theory: Because we can consciously think about so little at one time, we have to be careful about the amount and type of information we give our learners. The risk is that we’ll overload them with irrelevant, distracting but entertaining visuals and sound—but not teach them anything. We have to stay focused on the task being learned in all of our training presentations.
23:55 | Far Transfer
To enhance, use varied practice. Provide scenarios that are increasingly different, complex, and unusual.
Multi-media and simulation can help. Create different types of applications, even outside the work environment, perhaps including an unexpected environment (but only after mastering in normal a environment).
26:16 | Instructional Games & Simulations
Looked at games for the military. Looked at research studies and also evidence from the military’s use of games.
Came to a negative conclusion: Could not find any instance where games taught anybody anything better or quicker or less expensively than some alternative that would be less expensive or quicker. Yet there is some positive evidence about the benefits for some people (not all) of “game like features” in training such as competition (with yourself or others).
28:10 | Simulations
Games are different than simulations.
Simulations are a realistic depiction of the essential features of some system or process, principle, or whatever. Effective at certain points in people’s learning. Helps with hands on practice. Helpful with transfer.
29:05 | Games & Motivation
Games, it turns out (surprisingly), are not more motivating than other forms of instruction.
In fact, they may lead people to invest less mental effort (and therefore learn less).
Gabby Solomon’s research may shed some light: If you think something will be easy or enjoyable for learning, then you will invest less mental effort. (Great story here about TV vs. books.)
Overall: The return on investment for games is negative. However, the passion for games is so great, that most ignore the evidence.
36:00 | What Is Evidence?
Definition in medicine: Evidence is an empirically verified statement of fact about the impact of some intervention in some context.
How do we use the evidence that has been found via research?
Active ingredient model: Review results of the research that’s been done. What was it about the intervention/treatment/solution that actually made a difference?
Caution: Often there is confusion about the vehicle that carried the intervention vs. the message/information/structure. So we must ask:
- What’s the evidence?
- What caused the success?
- How do I decide whether I’m going to implement the active ingredient that came out of this research?
- If I implement, how do I adjust it to fit the culture and expectations of the organization I’m supporting?
39:23 Richard Mayer: Multimedia Learning
Mayer provides a good example of how to use research. He describes studies and relates them to a series of principles.
Caution: Many people doing studies have a vested interest in the solution that they are trying to validate, which can skew their results.
Temptation: To start with solutions in search of problems to solve.
Instead: Consultants should begin by being clear about the problem that needs to be solved and they must be open to any solution (rather than automatically recommending their favorite solution) that is appropriate for our clients: one that has the best evidence. The evidence must point to the active ingredients, so we know how to implement them so that they meet our client’s goals.
41:14 | Disagreements about What’s True
Future: Get people together who disagree. Have them design their solutions to problems and agree about how they can test those solutions in experiments.
42:10 | David Merrill
Great example: David could change his mind in an instant if the evidence constantly went against his thinking. This is hard for people to do. But, if we can’t do that, we aren’t going to grow.
You can find articles on the topics discussed in this interview on Dick’s research site, each of them including additional useful references.
Resources
Video: Richard E. (Dick) Clark, Ed.D.
Book: Turning Research into Results, by Richard E. Clark & Fred Estes
Website: Center for Cognitive Technology
Book: Multimedia Learning, by Richard E. Mayer
Video List: HPT Legacy Series
