Quick thoughts on the “replication crisis” and calls to make the field more mathematically rigorous

If you aren’t familiar with the “replication crisis,” in social psychology, start here, here, and here.

I consider the courses I took in college on quantitative and qualitative methods the most important of my undergraduate years. I learned thereby a great many important things about how not to conduct an experiment and how to think about experimental methodology (not to mention statistics.)

If I were putting together a list of “general education” requirements I wanted all students to to take in order to declare them well-educated and ready to go out into the world, it’d be a course on Quantitative and Qualitative Methods. (Much like current “gen ed” and “distribution requirements,” the level of mathematical ability required would likely vary by field, though no one should be obtaining a college degree without some degree of numerical competence.)

But the real problem with the social science fields is not lack of rigorous statistical background, but overwhelming ideological conformity, enforced by the elders of the fields–advisers, hiring committees, textbook writers, journal editors, etc., who all believe in the same ideology and so have come to see their field as “proving” their ideology.

Ideology drives both the publication biases and the wishful thinking that underlie this crisis. For example, everyone in “Women’s studies” is a feminist who believes that “science” proves that women are oppressed because everyone they know has done studies “proving” it. You’re not going to find a lot of Women’s Studies professors aiming for tenure on the basis of their successful publication of a bunch of studies that failed to find any evidence of bias against women. Findings like that => no publication => no tenure. And besides, feminist professors see it as their moral duty to prove that discrimination exists, not to waste their time on studies that just happened not to be good enough to find the effect.

In the Social Sciences more generally, we get this “post modern” mish-mash of everything from Marxists to Freudians to folks who like Foucault and Said, where the goal is to mush up long-winded descriptions of otherwise simple phenomena into endless Chomsky Sentences.

(Just reading the Wikipedia pages on a variety of Social Science oriented topics reveals how very little real research or knowledge is generated in these fields, and how much is based on individual theorists’ personal views. It is often obvious that virtually anyone not long steeped in the academic literature of these fields would not come up with these theories, but with something far more mundane and sensible. Economists, for all their political bias, at least provide a counterpoint to many of these theories.)

Obviously different fields study different aspects of phenomena, but entire fields should not become reduced to trying to prove one political ideology or another. If they are, they should label themselves explicitly, rather than make a pretense of neutrality.

When ideology rather than correctness become the standard for publication (not to mention hiring and tenure,) the natural result is incorrectness.

More statistical knowledge is not, by itself, going to resolve the problem. The fields must first recognize that they have an ideological bias problem, and then work to remedy it by letting in and publishing work by researchers outside the social science ideological mainstream. It is very easy to think your ideas sound rigorous when you are only debating with people who already agree with you; it is much more difficult to defend your views against people who disagree, or come from very different intellectual backgrounds.

They could start with–hahahaha–letting in a Republican.

7 thoughts on “Quick thoughts on the “replication crisis” and calls to make the field more mathematically rigorous

  1. I used to follow a sociologist. I knew he was ideological and gave him a lot of leeway, but in one of his perennial ‘how to make sociology popular again’ posts, he said one of the things sociologists should do is embrace positivism. I realized he wasn’t just ideological, but stupid. Short bus stupid. No longer able to climb the ivory tower, lest they should be triggered by the view, they are now in ivory bubbles, wobbling around talking about science.

    Anyway, the all the social sciences should be familiar with Von Mises- at least his methodology. That would keep them honest.


  2. As a (fellow?) Sociologist, I actually think the field is exactly where it ought to be. “Conservative” research basically just repeats things that used to be common sense, while everything else is focused on putting a scientific veneer on the prevailing dogma. HBD suggests that we aren’t changing much, just the stimuli we are confronted with. Why worry about statistical validity for research that was 10 years obsolete at the time it was published?


  3. This post seems a tad unfair, no? It’s not like before modern political correctness sociology was making consistently brilliant work. Studying the rules of human beings is very very hard – in part due to the constraints of what experiments we can perform, partly due to how ideologically contentious any results would be, but mostly I think due to how complex and chaotic human beings are.

    In the absence of being able to find rigorously validated results, but continued insatiable curiousity about how humans work… you’re going to get a lot of dreck that tells you more about the speaker than about the subject.

    Social sciences are very very hard and produce almost no reliable results. This is the cause of absurdly politicized fields, not the result.


    • The problem with social science is that last part. Science it is not. The whole range is social sciences is composed of people pushing ideologies, doing market research, or looking for some status points by being a “scientist”.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s