Public intellectuals and statistical thinking

I recently read this article on Identity Protective Cognition. The article is about how people don't really process evidence to decide on a side; they already know their side and are only looking for ammunition. (This is why the switch-sides requirement is so crucial in debate!) It does seem rather hopeless, so, rather than being depressed, let me pile on more to the idea of rationality's limits.

There is plenty of evidence that people can't analyze statistical information properly. Part of it is that people have trouble with doing even simple math, but a bigger problem is that people ask the wrong questions. Even smart people who are mathematically competent ask the wrong questions. In one experiment, described in this article, doctors get conditional probability problems wrong all the time. Here is the problem most doctors got wrong:

"The probability that [a] women has breast cancer is 0.8 percent. If a woman has breast cancer, the probability is 90 percent that she will have a positive mammogram. If a woman does not have breast cancer, the probability is 7 percent that she will still have a positive mammogram. Imagine a woman who has a positive mammogram. What is the probability that she actually has breast cancer?"

Some doctors got visibly nervous, anxious, and/or frustrated. Most assume that the answer is near or in between the correct negative rate (93%) and the correct positive rate (90%). Not so. A person need only take a minute to make a two-way table. The 0.8% base rate of breast cancer in the population is crucial to doing the problem correctly, but most people -- doctors included -- ignore it and ask the wrong kind of question.

Not only do many people struggle with mathematical/logical thinking, it is also a fact that our brains vastly prefer stories to data, as Pulitzer prize winners and brain scans can attest. People feel a greater connection to, and feel more informed by, a single anecdote than solid data. Furthermore, it seems like forming stereotypes is a part of that storytelling process. As we assimilate new information, we use stories and judgment. There are so many stories in the news media where the data or studies presented are merely a hook to start opining upon. In my own field of education, two examples jump out. S.A.T. scores are declining? Tsk, tsk. Never mind that more students, ergo more marginal students, are taking the test, so of course averages will go down. The U.S. K-12 education system ranks poorly compared to other developed nations? For shame. It is a shame -- but most would-be reformers sidestep the real issue. Our system is segregated by race and income, and our suburban schools are doing well enough. Yet so many reformers talk about pay-for-performance for teachers or setting up charter schools or a voucher system and ignore desegregation as a necessary and perhaps even sufficient reform.

Perhaps the most frustrating statistical failing of all is when people fail to ask, What exactly is being counted? What is this variable? The researchers said they were measuring positive outlook, but they were really just counting how many people tweeted positive words. (And "I'm not happy" counts as positive words!) This doesn't require any math skills at all! Just critical thinking.

If people struggle so much with data, what hope is there for a rational populace? Dr. Deanna Kuhn has great evidence about how poorly people organize evidence and conduct simple inquiry. Most people looked for "evidence" that confirmed their initial theories, and the "arguments" they constructed were mostly retelling a story they believed illustrated their belief. (This is called scripting.) But she also found that simple, well-applied interventions could make a big difference on both counts. Dr. Kuhn found that students responded quite well to instruction that was specifically focused on critical thinking skills. The record club idea is one worth adapting with your students! I know from my experience as a debater and as a debate coach that the switch-sides, competitive nature of the activity teaches critical thinking skills rigorously. And I know from my experience that a high school statistics course can teach students all the critical concepts they need to correctly interpret conditional probability. But beyond that, learning to think statistically is very foundational to thinking critically about any evidence. Basic statistical concepts are just logic applied to reality.

I think that Identity Protective Cognition and statistical innumeracy are both real, and they are both the default state of the human brain. However, education can make a big difference. Sadly, our education system is mostly failing in these ways. That's the micro side, the analyzing evidence and arguments. What about the macro side? Do most people leave school with a grasp of the big picture, with a sense of how the major forces work in the world?

I read an article about public intellectuals on Wilson Quarterly, which hit a lot of correct notes. Part of the reason America lacks a set of true set of public intellectuals is our preference for pragmatic thinkers; our general anti-academic, anti-elitist sentiment; and our love of technological solutions -- no problem is insoluble once set upon by good old American ingenuity. But I think the most important reason we lack public intellectuals is that Americans do not especially value historical thinking. I am not going to make a hoary "those who don't remember the past are doomed to repeat it" kind of point. It is more complex than that. The problem is that Americans are especially good at distorting historical facts to fit current ideology. I'm sure that for many people, the Founding Fathers of their imagination have little to nothing to do with the reality. Americans like to think that the debate we're having now is the debate we have always been having. The idea that the past really is a strange, different place is forgotten.

I don't believe that Americans are especially aware of how concepts like "freedom" or "equality" have a long history of evolution, of how the concepts have changed over time and do not mean now what they meant two hundred years ago. People are not aware of the assumptions that are buried below the surface in our popular concepts or the difficulties that are papered over. The idea of "rights," such as a person's right to their property, can short-circuit other considerations and options. The trade-offs of any political economy become invisible or perhaps visible but inevitable. This is the work of philosophy (asking ourselves to state everything clearly and unpacking values and concepts), especially after the linguistic and historical turns, those movements to a constructivist and critical version of philosophy. This is important stuff to study, but I know that few kids besides debaters have an opportunity to do much thinking like this in high school. I can imagine a good college course on politics or public policy might, but I can also imagine many courses merely discuss different systems in an empirical way with little analysis of "justice." As a result of this lacuna in our collective education, American politics is dominated by blind hope. Nietzsche said we needed courage instead of hope, in the face of our many challenges. I am trying to be courageous about our education system, as well as our politics.

Debate shouldn't be sophistry

In high school humanities, that is, English, history, and perhaps a semester of civics, ethics, or politics, a student is supposed to pick up five skills:

1. empathy for others, especially those who are different; 
2. ethical reflection; 
3. a understanding of social and/or government functioning; 
4. expository writing skills: putting together a coherent argument with evidence; and 
5. critical reading skills: analyzing texts carefully. 

It has always seemed to me that doing competitive debate does a really good job teaching a student all but the first. However, I have grown more concerned as I age that what debate teaches is very deep but misses some topics. I haven't become a gray-haired proponent of "cultural literacy" in my dottage: I still think that because debate topics are current, they are inherently more interesting and relevant to students than fusty old treatises. If you think high school students ought to read Aristotle and Plato fully, then I'm not your man. I believe they're far more likely to enjoy learning a little philosophy in order to apply it to current controversies. No, my concern is that, while it is good that debaters learn a lot about their topics, they are missing a few key ideas they ought to be taught specifically. My key recommendation is that second-year debaters take some kind of general critical thinking course that fills in these gaps.

One area I've written about before is logic. It is shocking to me that debaters would not know Toulmin's model, how to model definitions with Venn diagrams, or how to model causal arguments with Ishikawa diagrams. While it's not necessary to get into the complexities of propositional logic and truth tables, the basics ought to be covered.

Another area I've mentioned before too is statistics. Debaters are woefully uninformed about how statistics are collected and interpreted. To be blunt, they'll quote just about anyone, whether or not the method makes a lick of sense. Nothing too much is required: just a little knowledge about sampling methods, sources of sample bias, how to understand different correlational studies (like regressions), and how to understand experiments.

But where I think debaters come closest to outright sophistry is on critical arguments. Critical authors can shed light on complex issues. At their best, debaters quote these authors' anthropological, cultural, economic, and psychological investigations to unpack the standard assumptions most public policy authors approach a topic with. The critical authors can help all of us get behind the positions to see how our assumptions and values shape what we believe and advocate for. At their worst, debaters quote these authors to confuse and bury their opponents under a flurry of ten-cent words.

Why is it that outlandish impacts rule the day in policy debate? This is a bit of an overstatement. But I might argue that the development of critiques was probably driven by outlandish brink-based impacts. Many critiques re-introduce linear impacts into the debate round, i.e., endemic problems that are worsened by the plan. For example, racism exists before the plan, but plan makes it worse. Yet no judge would vote for this argument stated as a simple linear disadvantage. Add on the ten-cent words about critical legal studies and it's a go. The problem for debate theory is that the voting issue on a critique brings in a lot of philosophical and debate theoretical baggage: Is the judge voting against the Affirmative's discourse? Or is he voting against the Affirmative because the false assumptions identified by the critique show the case's logic is suspect (e.g., racist arguments undermine a civil rights plan)? Or is the judge simply voting that the critique's "impact" is bad (e.g., racism bad), like a linear disadvantage?

I believe many critiques only work by stringing together a theoretically inconsistent position: the link is about the topic itself, almost a counterwarrant (a guaranteed link, no matter what the plan actually does, e.g., attempting to improve women's literacy in Africa is based on sexist assumptions); the answer to comparative arguments is about pre-fiat, in-round discourse; and the implication is about real-world, post-fiat, non-comparative assessments of the critiqued impact. I can see some truth in the criticism that critiques are utopian counterplans: even thinking about the topic leads to racism -- just look at how the Affirmative speaks -- but by rejecting this, we can magically live in a world where racism is alleviated. Anyway, back to the main point:

If critiques are just bringing in linear impacts, why not just cut straight to running a linear disadvantage? The answer is that deprived of all the hoopla and sleight of hand (of critique theory and the critical author's fancy words), I believe a linear impact would never win a round.

Part of my solution is to teach every debater in a critical thinking course and to grind to a halt the competitive advantage of poorly-argued critiques. If a debater knows how to construct an argument, she can hammer away at the gaps in a critique. If a debater has a good sense of the basic flavors of philosophy and recognizes a nihilist argument when he hears one, great. Furthermore, many debaters go on to be judges, so getting this basic knowledge in them early is important.

But the second part of my solution is to ask judges to (a) stop rewarding sophistry and (b) recognize that linear disadvantages ought to win sometimes. This is not a theory problem or a rules problem. There is nothing that says linear disadvantages are weak. It is not a problem in the topic literature -- debaters are usually stretching the literature as it is. I am skeptical of the "literature checks abuse" argument. Most debaters' positions are undercut by their own authors; no one thinks the horrible impact scenarios are very likely. We the debate community need to recognize that:

     impact = mag x (duration) x prob / timeframe.

Humans are bad at estimating probability. But debaters and judges ought to make an effort to do better. Unfortunately, as it is, we've decided between offensive and defensive arguments that defense never wins. Never.

If debate could keep critiques straightforward and bring back good old-fashioned linear disadvantages, I would have a lot more confidence saying to other humanities teachers that competitive debating does indeed teach social/governmental knowledge, expository writing skills, and critical reading skills. As it is, I worry that we're doing our own thing that's becoming too divorced from reality.

Fitting a normal curve

I show my students histograms of more or less normally distributed real-life data. I have found it difficult, though, to get a Normal curve that fits nicely on top of the histogram. Is there a way to do a best-fit regression in this situation? I looked around and can't find one, so here's a procedure I came up with. I'm not sure if it's the best possible fit, but it's a good fit.

Using the fact that a normal distribution is given by the equation you can work backwards to see how to make the data linear. That is, is a linear transformation of normally distributed data. Here is a histogram.


I used the midpoint of each bin as the x data, and then I transformed the y data as described.


The one trick is that, for x values below the mean, the transformed y data points need to be negative. That can create a little ambiguity for the middle bin, but it's not too hard to tell here that 2.5 is a little below the mean. If in doubt, try both. I ran a linear regression on and found that This can be transformed back into which is almost ready to graph. All it's missing is the leading coefficient. A little work shows that the standard deviation is 0.88, and therefore the final equation is Here's the histogram, with the overlaid normal curve, which does not fit especially well.
However, this shows that this real-life data does not exactly follow a normal distribution, since this is about as well as we could hope it would fit.

Two activities for a statistics course

Finding good activities in a statistics course is easy, if you have access to the Internet and all the data it contains! Both of these activities were enormously successful with my students; they found the research very easy to do and fascinating.

I-85%

The U.S. interstate system takes remarkably straight paths between major cities: specifically, I believe that the true distance between two cities is, on average, only 6/7 of the interstate driving distance. Here's a statistical activity I made for my students:

Interstates

If they do a decent sample size, say n = 25, they will almost certainly reject the null hypothesis I state in the problem.

A Wikipedia theory

A tip of the hat to XKCD. The theory is simple: On any Wikipedia page, click on the first link that is not in italics or in parentheses; repeat; you'll eventually end up on the philosophy page.

My students tested this out (n = 160 trials). They found that they did always end up on the philosophy page, with a mean of 16 links required (and s = 4.6). Neat.

Researchers found that the number of link steps between two topics indicates how strong or weak the connection is.