Practices that support development
Key to maximizing the effects of retrieval practice are frequency (regular testing), spacing and variability (see section below on ‘Spaced and interleaved practice’).
Instructors:
- Administer low-stake quizzes frequently (ex: at least once a week).
- Use reading checks online to facilitate retrieval of pre-session readings.
- At the beginning of class, rather than re-reviewing what was covered during the last class, ask students to help identify key concepts and associated examples without looking at their notes.
- Close class by asking students to write down the most importance concepts covered.
- Provide weekly prompts that allow students to make connections between current and previously learned concepts. For example: a) provide examples of how today's material connects to real-world examples, b) describe how concepts covered in today's class relate to concepts covered last week, and c) explain how today's class connects to something you have learned in another course.
Students:
- Rather than re-reviewing previously seen information and old problems, test your own knowledge periodically by quizzing yourself and trying out new problems.
- Write summary of concepts covered after each class period without looking at your notes.
- Make and use flashcards for testing information that needs to be memorized or that would make learning new concepts more efficient if memorized.
- Take practice tests by using old exams or problem sets.
Spaced and interleaved practice
For spacing and interleaving to be effective, content should be practiced and reviewed at least several weeks and several months after it is introduced.
Instructors:
- Space review and recall of content/concepts across weeks and months. For example, on each quiz and/or problem set, you can dedicate a certain number of questions to previously covered material.
- Administer cumulative problem sets and exams throughout the semester.
- Review concepts covered in previous class periods/modules at the beginning of each class (see Retrieval section).
- Rearrange the order of practice problems in problem sets and exams as opposed to ordering problems by type.
Students:
- Space how often you test your own knowledge throughout the semester.
- When attempting practice problems, vary problem types in your practice.
Research shows that students may have a hard time generating self-explanations spontaneously (M. Chi, Bassok, Lewis, Reimann, & Glaser, 1989). Furthermore, the quality of their self-explanations may vary. It is important to deliberately construct conditions that facilitate the type of self-questioning that leads students to extract principles, connect concepts, and monitor understanding.
Instructors:
To encourage self-explanations, instructors can:
- Provide training on how to self-explain. Renkl et al. (1998) found that providing the instruction on how to self-explain facilitated students’ self-explanations, particularly for students with low-prior knowledge. Training consisted of: 1) providing information on the importance of self-explanations, 2) modeling self-explanations, 3) demonstrating articulation of self-explanations with an activity, and 4) coaching students as they attempted to self-explain (Renkl, Stark, Gruber, & Mandl, 1998).
- Use self-explanation prompts. Self-explanation prompts can take a number of forms from open-ended to more direct types (focused and menu-based forms):
a) Open-ended self-explanations: intersperse readings, videos and worked examples with a general prompt asking students to self-explain or justify an answer (Scheiter, Schüler, Gerjets, Huk, & Hesse, 2014).
b) Focused self-explanation: provide more explicit instruction as to what you would like students to self-explain. For example giving students some guiding questions that they can answer while reading/studying preparatory materials or worked examples: a) How are X and Y similar?, b) What would happen if …?, and c) How does X tie in with Y, which I learned before? (King, 1994). It is important that the questions relate to the integration of the content being presented and are not just prompting student to retain facts (R. E. Mayer, Dow, & Mayer, 2003).
c) Menu-based: ask students to choose explanations (principles, theorems, laws, etc.) from a menu which can be provided through online systems or in-class with or without the use of technology. For example, Hsu and Tsai (2011) asked students who made an error in an online educational game to select a cause for their mistake using a multiple-choice menu (Hsu & Tsai, 2011). Within the worked example literature, this type of self-explanation prompt is typically embedded throughout a multi-step problem to get students to relate individual steps or portions of a solution to abstract principles. Atkinson et al (2003), asked students to choose a probability principle after each step of a worked example from a drop menu (Atkinson, Renkl, & Merrill, 2003) while students in a study by Aleven & Koedinger (2002) justified each problem-solving step within an online geometry tutor by selecting one from a large glossary (Aleven & Koedinger, 2002).
Students:
- When reading or studying a set of worked examples, write a set of guiding questions that will help you interrogate the studying materials and make connections to previously learned information. Base your self-explanations on the following principles: 1) use your own words to self-explain, 2) focus on how and why over what, when and where and 3) actively attempt to make connections between what you are learning and what you have previously learned (King, 1994).
Worked and faded examples
Key elements for achieving the worked example effect are:
- variability of worked examples (using a set of worked examples that get at the same concept but have different cover stories)
- usage of self-explanation prompts (see Self-explanation section)
- fading (only showing parts of the solution and asking student to solve the rest)
The first two elements help students extract underlying principles and rules while the last one prepares students to make the transition from studying worked examples to problem solving by facilitating automatization of said principles and rules.
Instructors: worked and faded examples can be implemented either in class or outside class.
- Sequencing worked & faded examples: Given that variability of examples is key to facilitate deep understanding of principles, it is important to provide multiple isomorphic worked and faded examples for each task you would like students to learn how to solve. For instance, a sequence of worked examples might look like this:
- For a given task, start by demonstrating a fully worked example in class asking students questions throughout (ex: ‘Why was this strategy used?’, ‘What principle is being applied and why?’) to get them to self-explain. In-class demonstration of a worked example can be facilitated using active learning strategies. For certain self-explanation prompts, it might make sense to have students think on their own first and then share with a partner.
- Assign a set of worked examples and gradually fade support in subsequent worked examples by asking them to solve more and more steps within a problem. These sets of problems (fully worked and faded examples) can be split between in-class and outside class. How many additional worked and faded examples should be provided will depend on the complexity of the task you are asking students to master and your students’ prior knowledge.
- Additional approaches to encourage students to self-explain that are specific to worked examples and have not been previously mentioned in the Self-explanation section:
- Provide a fully-worked example to students and ask them to annotate the solution to justify or explain it, either in class (singly or in small groups), or after class through a collaborative annotator.
- Provide incorrect solutions and ask students to correct the error and justify the correction. The errors can be targeted at common misconceptions to help avoid these errors during problem solving (Durkin & Rittle-Johnson, 2012). Asking student to find the errors in incorrect solutions is not only a clever way to get students to self-explain, but is usually superior than asking students to self-explain correct solutions only (Booth, Lange, Koedinger, & Newton, 2013; Siegler & Chen, 2008).
- Provide students with two different solutions and ask them to compare and contrast them.
Students:
- To benefit from the worked example effect, students should 1) study solutions to different problems targeting the same concept (problems that look different but use the same set concepts/principles) and 2) ask themselves why a particular step in the solution is important and why the solution is correct. After practicing from a set of worked examples, students can create a faded example from a fully worked example by blocking the last solution step(s) and solving it without looking at the answer first.
- 14.73 The Challenge of World Poverty| MIT: Pop-quizzes are randomly administered throughout the semester. In addition to providing opportunities for retrieval, pop-quizzes encouraged students to come prepared to class. In 14.73 pop-quizzes count towards 8% of the overall course grade.
- Biology Professors Mary Pat Wenderoth & Scott Freeman | University of Washington: At the very beginning of the course, Wenderoth and Freeman teach students about the importance of retrieval for learning. In their courses, they implement several strategies to support retrieval:
- Testing groups: instead of ‘study groups’, Wenderoth divides the course into ‘testing groups’. Students are asked about which ideas they are not clear on. One student uses the board to try to explain the concept while the rest of the group is encouraged to test the person’s knowledge by asking questions (Brown, Roediger, & McDaniel, 2014).
- Free recall: at the end of class, Wenderoth provides students with a short period of time (10 minutes) to write down everything they remember from that class session. Students must wait until the allotted time is over to check their notes and find out what they missed and got correctly (Brown et al., 2014).
- Learning paragraphs: on Fridays, Wenderoth provides students with a question that gets students to retrieve and integrate multiple concepts from the week (Brown et al., 2014).
- Practice exams: Freeman gives students 30 minutes each week to complete an on-line practice exam created from previous years’ exam questions. After answers are submitted, students are randomly and anonymously assigned two other exams to grade using a provided rubric. Each student’s exam is scored by two other students. Practice exams represent 8% of the overall course grade (Freeman, Haak, & Wenderoth, 2011).
Spaced and interleaved practice:
- 18.03 Differential Equations | MIT: Each problem set in this class contains two parts. Part A contains problems directly related to the current unit. Part B, however, typically requires the application of all methods covered to date, as well as newer methods, allowing students to apply previously learned concepts periodically. In addition, the final for this class is cumulative.
- 7.03 Genetics | MIT: In 2012 and 2013, halfway through the semester, students were assigned a "Lab Practical Assignment" which combined more than half of the concepts covered in the semester, presented as a multi-step simulated laboratory assignment. In this assignment, students were provided with a set of mutant strains isolated after a genetic screen. Using a genetics experiment simulator, StarGenetics, students performed a series of genetic analyses covered throughout the semester to uncover the genetic basis for these mutations.
- Psychology Professor Kathleen McDermott | Washington University at S. Louis: uses low stakes-quizzes administered at the end of every class. The questions might be related to the pre-assigned reading, the lecture, or both. However, the quiz might also contain questions on previously-covered material that students did not understand well. To lower the anxiety that daily quizzing might provoke, McDermott explains within the first class the research on retrieval and allows students four free absences (Brown et al., 2014).
- Psychology Professor Natalie Lawrence | James Madison University: tested the effect of cumulative exams in her course. In one section of her class, students had cumulative exams throughout the semester (80% new material, 20% previously covered concepts) plus a cumulative final. In the other section, students had non-cumulative exams (100% all new material) with a cumulative final. Students in the ‘cumulative’ condition did better in both the final and on a subsequent test that took place two months after the completion of the course. The effect was most pronounced for low-scoring students (Lawrence, 2003).
Worked and faded examples:
- AlgebrabyExample | Collaboration between SERP (Strategic Education Research Partnership) and Minority Student Achievement Network (MSAN): A large, year-long randomized control trial with 300 participating classrooms and 6000 Algebra 1 students. Students were placed in either: a worked example condition (42 assignments that consisted of interleaved incorrect and correct worked examples and problems to solve; self-explanations where implemented with the worked examples and in the case of the erroneous worked examples, students had to correct the solution) vs. a control condition (assignments cover the same concepts and had the same number of items, except they only contain problems to solve; no worked examples or prompts for self-explanations). The worked example condition resulted in a statistically significant increase in conceptual understanding (10%) and performance in state assessment questions (7%) in comparison to the control condition. Increases in conceptual understanding were surprising given that students in the worked example condition had 50% less problems to solve than those in the control condition (Booth et al., 2015).