May 2019
Volume 34 Number 5
[Don't Get Me Started]
Calc or Stats?
By David S. Platt | May 2019
.jpg "David S. Platt")
My daughter Annabelle is now in the spring of her senior year of high school. Most seniors slack off, but that’s not her way. She’s taking both AP Calculus and AP Statistics courses, and is now trying to finish them off.
She absolutely loved calculus, saying, “Derivatives are the meaning of life.” But statistics bore the living crap out of her. (I can’t repeat her exact language in this column, and I can’t imagine where she learned those words.) And yet, I can’t help wondering.
I’ve trained as an engineer, worked my entire career in technical fields, taken calculus through multivariable. But never, not once, have I fired it in anger against a target that I needed to kill in order to eat. The concepts of differentiating and integrating, yes; the actual operations, never. When was the last time you took the cross product of a vector field, or even a common garden variety derivative? On the other hand, I see percentages and probabilities many times per day, and have never studied them in detail. I’m probably misusing them and making mistakes. As are you.
The results of this misprioritization aren’t benign. Here’s an example from Zak Kohane, whom you may remember from my earlier articles (December 2017, msdn.com/magazine/mt814422). He and his colleagues published a research letter in “JAMA Internal Medicine,” entitled “Medicine’s Uncomfortable Relationship with Math: Calculating Positive Predictive Value” (bit.ly/2eiI1A6). They asked 61 doctors the following question: “If a test to detect a disease whose prevalence is 1/1,000 has a false positive rate of 5 percent, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person’s symptoms or signs?”
Figure it out yourself. I’ll bet you get it wrong. So did three-quarters of the doctors they asked. Would you say 95 percent? 90 percent? I’ll stop writing to give you time to work it out.
The correct answer is 2 percent. Remember, the disease is rare, 1 in 1,000 people. A false positive test is much more common, 5 percent, or 50 out of 1,000 tests. The odds are 50-to-1 that this positive value indicates a testing error. Not that this comforts the patient, or the doctor.
Zak and his co-authors wrote: “With wider availability of medical technology and diagnostic testing, sound clinical management will increasingly depend on statistical skills …. Specifically, we favor revising premedical education standards to incorporate training in statistics in favor of calculus, which is seldom used in clinical practice.”
Why don’t technical people study statistics? Partly because it’s not on the advanced math track in high school, so students who take it suffer the perception of being misfits who couldn’t hack the calculus track. But watching Annabelle’s virtual high school video, I’m convinced that the main reason is that current presentations of it are painfully boring. I tried one of their lessons, and couldn’t stay awake.
Why are things this way? I’ve written for years in this space of the enormous educational opportunities, once basic classes are taught online (see msdn.com/magazine/hh708761 and onward). Why isn’t the best statistics teacher in the universe making this video? Or even one in the top half?
I could do far better myself. If there’s one thing I’ve learned in the teaching and presentation business, it’s this: Know your audience, because they’re not you. What do high schoolers think about? From mentoring the robotics class, I know: sex, money and misbehavior. I could design the most killer stats class ever for high school and early college. I’d tap their natural proclivities by using Charles Wheelan’s “Naked Statistics” (Norton, 2014) as the textbook. His chapter on live beer tasting commercials during the Super Bowl would fit right in. I’d work in supplemental readings from Darrell Huff’s 1954 classic, “How to Lie with Statistics.”
Instead of showing one dry equation, then another, and then another, I’d start with a real-life situation, explain what we want to know about it and why, then develop the math from there. For example, “Here’s how the lottery and casinos are taking you for suckers. And here’s how you turn the tables on them.”
I say to Zak, you’re a doctor, and you want other doctors to take their medicine. How about a spoonful of sugar to help it go down? Who’s got the money, and the nerve, to pay me to do this?
David S. Platt teaches programming .NET at Harvard University Extension School and at companies all over the world. He’s the author of 11 programming books, including “Why Software Sucks” (Addison-Wesley Professional, 2006) and “Introducing Microsoft .NET” (Microsoft Press, 2002). Microsoft named him a Software Legend in 2002. He wonders whether he should have taped down two of his daughter’s fingers so she would learn how to count in octal. You can contact him at rollthunder.com.