Research on teaching mathematics

I recently ran across the International Society for Design and Development in Education. Their stated goal is to develop standards for creating education materials, like airplane designers follow a standard pattern or method to create new airplanes. While I cannot tell much about what the I.S.D.D.E. actually does from its website, it is an interesting idea, and it got me thinking.

Education ideas come and go. Rather than chase fads, I would like to know what the research says. Does the evidence show unequivocally that an idea helps, or is the evidence mixed or non-existent? When we design new curricula and new materials, what do we know works? Bear in mind, I am not an education researcher, although I did spend considerable time thinking about these issues when I was working my two masters degrees. Since then, I've had seven years of math teaching experience -- as well as a lot of debate teaching experience to compare it to. But please take my opinions with a grain of salt. Of course, it's also incredibly difficult to design good education experiments. Instruction is so multifaceted and so much of teaching is spontaneous that is hard to design good controlled experiments. Still, there is good evidence out there; it is worth knowing.

Flipping the classroom

The new idea today is "flipping the classroom." Defining a term when its use is exploding is hard because so many people abuse the term. Roughly speaking, I would define flipping the classroom to mean students learn new material for homework and the teacher uses classroom time for practice problems -- the opposite of what typically happens in a math or science class. My definition would imply that humanities classes are, as usually taught, already "flipped." Students read new material at home (a textbook, poem, novel, or other source material), and in class, they discuss, debate, write, and in other ways practice with the ideas, themes, literary techniques, etc. As I read it, the idea of flipping is mainly an idea to improve math and science instruction.

Flipping the classroom makes a lot of sense on face. As a teacher, your most precious resource is your face-to-face class time. Personally, I ask students to do some basic practice problems at home, but I know that I can push them hardest when they are in my class, because I am there to monitor and help them. This aspect of the flipped classroom has immediate, clear appeal.

The first question is, Can the students learn the material on their own? I have written before about the problems with math textbooks before: students have trouble reading them, or more often, do not even try. On the other hand, I am not a big fan of the students simply watching a video lecture. It is better than not reading the textbook; students will actually watch the video. But watching the lecture is too passive. If you are flipping the classroom, I would advocate for the use of a discovery-based homework set. I provided an interactive example, but a paper version could work, too. With a few leading and thoughtful questions, students could work through the basic ideas. The questions could require looking at an interactive applet online or looking at a few printed examples. They would "read" mostly by doing some calculation or measurement problems, followed by questions that ask them to summarize the new idea.

My model here is the Discovering series. The Discovering Geometry book gives students a construction or two to do before making an observation of a geometric property. Most students can do the constructions and can make the right observation; I think this is manageable for them to do at home. If this discovery was paired with an online quiz -- just a few questions -- student completion would probably be over 90 percent, meaning they would have taught themselves the concept. This kind of model would work for any mathematical topic, but what the students can "discover" on their own depends on their strength as mathematicians and on the difficulty of the topic. The key requirement is that the students must do something right away. Rather than read for 20 minutes (yeah right) and then do examples -- or even watch a video for 20 minutes -- the at-home work must give a quick burst of text or explanation followed immediately by asking the students to do something. Even if the first question is trivial, break up the exposition!

So, good in theory. Is there evidence? So far, just a little, but it is promising. One of the best examples so far is Dr. Mumper at U.N.C.'s School of Pharmacy. He flipped his classes and kept good data on control subjects. Based on quality data, it seems outcomes on the final exam improved by about 5% (or is it 5 percentage points? I cannot tell). Another example is Dr. Mazur at Harvard; he saw real gains for his physics class. It sounds like his method, based on this description, really involved a lot of collaborative student work in class. On the other hand, some professors at Harvey Mudd were not impressed. Right now, the proper attitude is cautiously optimistic; perhaps more evidence will amass for flipping the classroom.

However, the discussion so far about flipping the classroom has focused on "Does it work?" -- rather than the more important question, "Is it worth the effort?" After all, flipping the classroom is a lot of work. Are there other ways to improve math or science teaching that are more effective or require less effort? In other words, a teacher should consider flipping the classroom only if there are no low-hanging fruit left to pick.

To the extent that these experiments were successful, some was probably due to non-integral improvements that happened to come along for the ride. The best experiment of all is not to compare a traditional lecture class to a flipped class. The best experiment of all would be to compare a lecture class that uses every research-supported improvement possible to the flipped class. For example, most flipped classes use short 15-minute video lecture because evidence shows students have trouble absorbing more than that. So, the lecture class' in-class lectures ought to be limited to 15 minutes or fewer as well. And so on. Make every possible improvement to the lecture class that is possible without flipping. My point is merely this: If you want to improve your instruction, but flipping the class is too daunting, do not let that be an excuse for inaction. There are plenty of ways to improve your class.

On the one hand, the idea of flipping is so radical that it forces instructors to rethink everything. I am sure plenty of instructors do need to rethink every part of their practice because all of it needs improvement! On the other hand, making a whole lot of improvements, even if they are less radical than flipping the class, might have an even greater combined effect on student learning.

Interleaving

One of the clearest memory effects is for interleaving. The idea is to overlap two or more topics in time. (While I think integrating the topics is also worthwhile -- making connections between topics -- the research shows enormous retention benefits just for mixing the units of study together.) Students might study logarithms for a week, then polynomials for a week, then back to logarithms. All the while, they have mixed review homework. This might sound like it would confuse the students, but because they have so many more opportunities to recall and use the ideas, they remember it much better. The evidence here is solid. Here is one example of its use in a pre-algebra class.

In my own personal experience, I found it was enormously effective. I took my A.P. Statistics class and interleaved the topics: one week of experimental or sampling design, one week of probability, one week of descriptive statistics, and one week of statistical inference, then "unit" test. We kept that rotation schedule, with some minor modifications, all school year. I made no other changes -- I used the same textbook, same problem sets, and same in-class activities and projects. The result? The average of my students' scores on the A.P. exam jumped by almost a whole point (out of 5 possible points on the A.P.). It is not because they understood the material better, but because they had review built-in every month.

I would assume that the best model for interleaving is one where frequency drops off but the difficulty increases. For example, a student might be introduced to a basic logarithm concepts on day one; perhaps the student learns how to rewrite a logarithm as an exponential equation. On days two and four and six, the students sees basic logarithm questions again (say, where the bases and exponents are all nice numbers). Then on days nine and twelve, the student sees medium difficulty logarithm questions (perhaps one of the numbers, a base or exponent, requires some fractional thinking). Four days pass, and on day 16, the student sees a medium-to-high difficulty-level question (now he has to rewrite a logarithmic equation into exponential form where there are no numbers, only variables).

While this may seem like a slow pace for teaching one concept, remember that other concepts are being introduced at the same time. Maybe 50% of the work students do is concepts they only learned in this week of class; 25% might be review from the two or three previous weeks; and the last 25% could be review from a month ago or more. This mixture can keeps homework challenging. (It drives me crazy that every problem in one section requires finding a right triangle; and in another section, an isosceles triangle. It is ridiculously too easy to solve homework problems if every one uses the same concept.) At the beginning of the year, teachers could interleave new topics with review topics from the previous year.

The simplest way to interleave is simply to do the sections in a typical textbook in mixed order. That is what we did in my Statistics class: chapter 1.1 to 1.3, then 4.1 to 4.4, then 8.1 and 8.2, then back to 1.4 to 1.6, then 4.5 to 4.7, and so on. In this regard, it is simple to implement. A teacher only needs to plot out the sequence of sections for the year, and then make sure no prerequisite skills or concepts are out of sequence. Before assigning homework problems, the teacher can check that the students are able to do each one -- sometimes textbooks do throw in unlabeled mixed review problems. Of course, it is even better if the teacher can integrate the concepts, as well as interleave, if there are times where a connection can be made between two different topics.

Group work

According to Robert Slavin on the BEE website (Best Evidence Encyclopedia), the research comparing different primary and secondary curricula has a clear result: curricula in which the students work in small teams of two to four, perhaps with alternating roles (once as the "teacher" and once as the "student," for example), are the most effective. In other words, collaborative-learning oriented curricula all show greater positive effects than other kinds of curricula. His research doesn't address the why question, but I have some guesses.

First, I think students get more exposure to metacognitive thinking in group work. As they discuss a problem, someone tends to say, "What are we trying to solve?" or "Can we draw a diagram?" Students working on their own tend to try one strategy and succeed or give up; students in groups are more likely to compare and evaluate different strategies.

Second, I suspect curricula with built-in group work tend to ask students more interesting questions. "Solve for x" using a known algorithm is a boring question to discuss; "extend your knowledge of this algorithm to suggest two different methods to find x" in a novel context is a great discussion question. And the questions can be more difficult, since the students have each other to rely upon.

Third, a solid group-work protocol is probably helping the weakest teachers more than the best teachers. The best teachers already know how to draw out metacognitive thinking and already know how to ask interesting, challenging questions. Changing to a different curriculum will help the best teachers only marginally. On the other hand, the weakest teachers have a lot of room for improvement. Yet a new curriculum with topics in a different order or presented differently will probably help them very little. However, a curriculum with group-work built in will help the weakest teachers address their biggest weakness: drawing out student thinking. My point is that the strong evidence in favor of collaborative-learning curricula probably is due to some very differential effects based on teacher quality.

Simple cognitive models

Here's where we get international. Many nations that have very strong mathematics education explain concepts through simple cognitive models that are continually revisited, practiced, and extended. (See Willingham's article, p. 19, for a Singapore example; exit 10A also posted a nice example of a simple visual model for division, before teaching students the traditional division algorithm; many, many more abound.) The idea is to develop easy-to-understand models and use them over and over again to reinforce the concept.

The simple cognitive model approach has been used successfully by many national curricula. What do we have in the U.S.? Dolphins. I wish I were exaggerating. Scroll down to Mr. Aggarwal's statement. The thing that is so frustrating is that there is a simple cognitive model for the distributive property: the box method.

This example is (x+3)(2x + y + 4). The model should be built upon an area model of multiplication. I would start with one-by-one rectangles with two numbers, then move to one-by-one rectangles with one variable, and so on, always making sure students understood the answers to be areas. A question that would test whether students understood this is: "Which is bigger: 6x or 3y?" Of course, the answer is that they are incommensurate, until we know both x's and y's values.

The box method is extensible to factoring, to complex number multiplication, to completing the square, and more. It is firmly rooted in geometric thinking and makes the connection that multiplication is just repeated addition: 3x = x + x + x is a great diagram with which to start using the box method with variables. Strictly algebraic methods tend to be lost on the students, except for perhaps 5-10% who are very algebraically-oriented. But this subset of students will understand a visual or geometric explanation anyway, and the algebraic, procedural approach ought to follow later once the simple cognitive model is fully grasped.

The idea with simple cognitive models is to pick ones that can be re-used; that way, new ideas can be strongly connected back to old ones.

Metacognition

Metacognition matters. Even simple mathematical procedures require mastering a flow of steps. Students who are able to manage themselves through the steps -- recognizing when something has gone wrong, rethinking assumptions, checking solutions for reasonableness, comparing the efficiency of two different methods -- are at an advantage.

On the other hand, it is easy to overstate the importance of metacognition. Students who have poor working memory are going to struggle to add another layer of thinking -- and working memory is more or less fixed. Furthermore, metacognition, also known as critical thinking, is domain-specific. The strategies are different for different subjects.

Yet metacognition is an important set of skills to work on. For the weakest students, it will be a challenge. But that's why metacognition should be deeply ingrained in the culture of doing math. The teacher can model estimating, stating assumptions, and checking solutions. The teacher can expect and require that students always verbalize thinking, and the classroom can have a culture of error-checking and seeking multiple solutions.

Even though metacognition is domain-specific, all we care about is that students learn mathematical problem-solving skills. They should know techniques like: draw a diagram, make a table of values, list knowns and unknowns, mark congruences, solve simpler problems first, try to identify a pattern, add an auxiliary line, de-layer a problem, model with a function, model in a graphing program like GeoGebra, write down theorems that seems applicable, see if a theorem can be extended (e.g., "Can you extend the Pythagorean theorem to three dimensions?"), look for class-inheritance of ideas (e.g., all the properties of isosceles triangles are true for equilateral triangles, too), write clear definitions with clear conditions, look for contrapositives of known theorems, and many more. These techniques are all math-specific. However, the more general skills, such as self-reflection, can help in many other subjects.

Metacognitive strategies matter, especially for the weakest students. Thinking is hard work, and students need tools to help them.

More testing

There's good research on how effective the pre-test can be. Unfortunately, the pre-test is especially challenging to design for mathematics; students may not even know how to interpret the questions, negating any possible benefit to the early exposure to the concepts.

However, frequent, low-stakes quizzes are also incredibly beneficial to memory and learning, for almost the same reason as the pre-test. Quizzes expose false fluency, the student's false belief that he or she understood the concept.

Traditional vs. modern methods

Some researchers have found data that traditional teaching methods are correlated with high student achievement. Yet other analyses have found the exact opposite looking at the same data. The key issue is lurking variables. Stronger students allow a teacher to default to more traditional methods; weaker students cause the teacher to use more modern methods. Another issue is that this is all self-reported data. Younger teachers, who more recently were in education programs and heard more about the benefits of modern methods, might use or might report using more class time for modern methods. Since these teachers are younger, they are also likely to be weaker as well. Alternatively, it is possible that modern methods work better than traditional lectures in the hands of expert teachers but utterly fail in the hands of weaker teachers. If the weakest teachers can manage an acceptable lecture, then modern methods would have the lower overall average.

A final issue is that the tests may not measure reasoning skills very well. Altogether, the research is good that traditional methods promote memorization. There is a lot of experimental research, not observational studies, on this question. And here is a meta-analysis showing that lecture methods are inferior to more active ones.

Here is another article on the benefits of testing frequently and the benefits of interleaving.