The title of this post will be familiar to educators experienced in the academic study of statistics. At the introductory level, students in the field are warned of a common problem: misinterpreting correlation (when variables track each other impressively) with a spurious inference of causation.
A typical example used in statistics classes is an awesome correlation between roosters crowing and the sunrise. B follows A consistently, you can bet on it. However, we know, based on experience, that there is no need to worship the Magic Rooster. Our distant ancestors may not have possessed such insight. As Homer Simpson would say, “Stupid ancestors!”
Please have a look at Matt Reed’s perceptive piece in Inside Higher Ed., on how this dilemma can play out at a community college. He cites as an example the problem of evaluating a commendable strategy of tutoring at-risk students. As he points out, the real cause of reported student improvement could be prior traits of those who volunteer for tutoring. By definition, a volunteer for extra work is exceptional, no?
This particular problem is pervasive in untangling educational strategies to decide what works.
Dr. Reed doesn’t discuss the subject, but let’s extrapolate about dual credit. An oft-cited correlation exists between dual credit enrollment and subsequent student success. But why? Is it because of superior pedagogy in dual credit classes? Many would argue otherwise, especially in college courses blended with high school classes. If you talk with practitioners, the leaning environment can turn into a mess quickly in a high school classroom.
Even as dual credit has become mainstream, with a substantial proportion of community college enrollment occupied with dual credit courses, participation still requires students and/or parents to come forward—to volunteer. Such individuals already fit any honest predictor of success. This is not to suggest that cherry-picking is intentional, merely that typical high school kids may not take advantage of dual credit opportunities. But they still show up at our schools as freshmen after they graduate, woefully unprepared for collegiate-level work. Now let’s compare these more typical students with dual credit kids, and it’s not difficult to see what’s going on.
It is certainly proper to reject anecdotes and impressions as evidence and to insist on data-driven analyses. But statistical studies often contain flaws, too. It’s part of the deal in a complicated world. But we should admit it.