Public intellectuals and statistical thinking

I recently read this article on Identity Protective Cognition. The article is about how people don't really process evidence to decide on a side; they already know their side and are only looking for ammunition. (This is why the switch-sides requirement is so crucial in debate!) It does seem rather hopeless, so, rather than being depressed, let me pile on more to the idea of rationality's limits.

There is plenty of evidence that people can't analyze statistical information properly. Part of it is that people have trouble with doing even simple math, but a bigger problem is that people ask the wrong questions. Even smart people who are mathematically competent ask the wrong questions. In one experiment, described in this article, doctors get conditional probability problems wrong all the time. Here is the problem most doctors got wrong:

"The probability that [a] women has breast cancer is 0.8 percent. If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram. If a woman does not have breast cancer, the probability is 7 percent that she will still have a positive mammogram. Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?"

Some doctors got visibly nervous, anxious, and/or frustrated. Most assume that the answer is near or in between the correct negative rate (93%) and the correct positive rate (90%). Not so. A person need only take a minute to make a two-way table. The 0.8% base rate of breast cancer in the population is crucial to doing the problem correctly, but most people -- doctors included -- ignore it and ask the wrong kind of question.

Not only do many people struggle with mathematical/logical thinking, it is also a fact that our brains vastly prefer stories to data, as Pulitzer prize winners and brain scans can attest. People feel a greater connection to, and feel more informed by, a single anecdote than solid data. Furthermore, it seems like forming stereotypes is a part of that storytelling process. As we assimilate new information, we use stories and judgment. There are so many stories in the news media where the data or studies presented are merely a hook to start opining upon. In my own field of education, two examples jump out. S.A.T. scores are declining? Tsk, tsk. Never mind that more students, ergo more marginal students, are taking the test, so of course averages will go down. The U.S. K-12 education system ranks poorly compared to other developed nations? For shame. It is a shame -- but most would-be reformers sidestep the real issue. Our system is segregated by race and income, and our suburban schools are doing well enough. Yet so many reformers talk about pay-for-performance for teachers or setting up charter schools or a voucher system and ignore desegregation as a necessary and perhaps even sufficient reform.

Perhaps the most frustrating statistical failing of all is when people fail to ask, What exactly is being counted? What is this variable? The researchers said they were measuring positive outlook, but they were really just counting how many people tweeted positive words. (And "I'm not happy" counts as positive words!) This doesn't require any math skills at all! Just critical thinking.

If people struggle so much with data, what hope is there for a rational populace? Dr. Deanna Kuhn has great evidence about how poorly people organize evidence and conduct simple inquiry. Most people looked for "evidence" that confirmed their initial theories, and the "arguments" they constructed were mostly retelling a story they believed illustrated their belief. (This is called scripting.) But she also found that simple, well-applied interventions could make a big difference on both counts. Dr. Kuhn found that students responded quite well to instruction that was specifically focused on critical thinking skills. The record club idea is one worth adapting with your students! I know from my experience as a debater and as a debate coach that the switch-sides, competitive nature of the activity teaches critical thinking skills rigorously. And I know from my experience that a high school statistics course can teach students all the critical concepts they need to correctly interpret conditional probability. But beyond that, learning to think statistically is very foundational to thinking critically about any evidence. Basic statistical concepts are just logic applied to reality.

I think that Identity Protective Cognition and statistical innumeracy are both real, and they are both the default state of the human brain. However, education can make a big difference. Sadly, our education system is mostly failing in these ways. That's the micro side, the analyzing evidence and arguments. What about the macro side? Do most people leave school with a grasp of the big picture, with a sense of how the major forces work in the world?

I read an article about public intellectuals on Wilson Quarterly, which hit a lot of correct notes. Part of the reason America lacks a set of true set of public intellectuals is our preference for pragmatic thinkers; our general anti-academic, anti-elitist sentiment; and our love of technological solutions -- no problem is insoluble once set upon by good old American ingenuity. But I think the most important reason we lack public intellectuals is that Americans do not especially value historical thinking. I am not going to make a hoary "those who don't remember the past are doomed to repeat it" kind of point. It is more complex than that. The problem is that Americans are especially good at distorting historical facts to fit current ideology. I'm sure that for many people, the Founding Fathers of their imagination have little to nothing to do with the reality. Americans like to think that the debate we're having now is the debate we have always been having. The idea that the past really is a strange, different place is forgotten.

I don't believe that Americans are especially aware of how concepts like "freedom" or "equality" have a long history of evolution, of how the concepts have changed over time and do not mean now what they meant two hundred years ago. People are not aware of the assumptions that are buried below the surface in our popular concepts or the difficulties that are papered over. The idea of "rights," such as a person's right to their property, can short-circuit other considerations and options. The trade-offs of any political economy become invisible or perhaps visible but inevitable. This is the work of philosophy (asking ourselves to state everything clearly and unpacking values and concepts), especially after the linguistic and historical turns, those movements to a constructivist and critical version of philosophy. This is important stuff to study, but I know that few kids besides debaters have an opportunity to do much thinking like this in high school. I can imagine a good college course on politics or public policy might, but I can also imagine many courses merely discuss different systems in an empirical way with little analysis of "justice." As a result of this lacuna in our collective education, American politics is dominated by blind hope. Nietzsche said we needed courage instead of hope, in the face of our many challenges. I am trying to be courageous about our education system, as well as our politics.

Debate across the curriculum

There are two interesting curricula I have run across recently. Stanford has an "Reading Like A Historian" curriculum, available free of charge: http://sheg.stanford.edu/rlh. The goal is to ask students to read primary sources carefully, evaluate each speaker's motivations and claims, and arrive at a nuanced, triangulated interpretation of events. They have had excellent results so far.

Sounds a lot like debate to me!

Another curriculum I ran into is Deanna Kuhn's program, which is a philosophy class for middle schoolers. They set this up as an experiment: some kids did philosophy through debate, while the control group kids did philosophy through lecture, reading, and writing alone. The results were quite impressive.

I've been a fan of Deanna Kuhn for at least a decade. She has a new book, Education for Thinking. Here is an excerpt of her writing (but not necessarily her book) from her website:

But aren't children naturally inquisitive? Are inquiry skills something that really need to be developed? The image of the inquisitive preschool child, eager and energetic in her explorations of a world full of surprises, is a compelling one. But the image fades as the child grows older, most often becoming unrecognizable by adolescence, if not middle childhood. What has happened to the "natural" inquisitiveness of early childhood? In part its nurturance into adolescence and adulthood rests on a set of values that parents and teachers must convey and support. But equally important is the channelling of this inquisitive energy into development of the cognitive skills that make for effective inquiry. The skills originate in early childhood, with achievement of the epistemological understanding that knowledge originates in human minds, is fallible, and has the potential for disconfirmation in the face of evidence. Only then does the coordination of theories and evidence that is a hallmark of authentic scientific inquiry become possible. In sum, the so-called "natural" curiosity that infants and young children show about the world around them needs to be enriched and directed by the tools of scientific thinking.

Her book was interesting. She described several investigative tasks; the music club one was easily replicable from her thorough description and sounded quite neat. She also had some interesting things to say about how to structure an introductory debate activity that would be neat for coaches who work with very novice debaters. Dr Kuhn had a key point about bootstrapping: when students knew little about the context and had few critical thinking skills, the teacher needs to have good structure set up so the students can bootstrap their way into both subject and skills.

Dr Kuhn's clearest call for reform was to create a specific focus across all classes on critical thinking skills. As she wrote in the book,

By examining causality in a biological context, a geographical context, a mechanical context, an interpersonal context, a sociological context, and any number of other contexts, students begin to understand and appreciate features of causality itself.

I think the goal of education is not information transfer, per se, but the ability to make a sustained argument, to critically evaluate sources and ideas, and to learn to investigate systematically (which could be a scientific experiment or a comparison of all available historical sources).

Putting the child at the center -- by asking him to debate an issue, to do an experiment, to discover a mathematical idea -- has two main benefits. First, he is actually spending his time on the important task itself (instead of endlessly preparing with rote memorizing for the day -- way off in grad school -- when he might be allowed to investigate anything on his own).

Second, the child is far more engaged in his work, because he has been given agency and purpose. Purpose: answer this question. Agency: it is up to you to answer this question, with some help if you need it, but no one is going to do it for you!