Monday, April 03, 2006

attack of the replicants

So, as readers know, I have decided to roll the dice on being labeled a crank* by arguing that quantitative sociology should attempt to muster some pretense of keeping up with movements in other social science disciplines to increase the transparency and accountability of data analysis (see here for standards of data availability in the flagship journal of economics, or here for extensive and influential debate about replication that commenced more than a decade ago in political science). One of the more interesting arguments from my detractors is that requiring to deposit their code for others to examine and learn from is a crass thing to ask of true professionals.

My first reaction to this argument was to think that it's an interesting feature of certain predominantly-left enterprises that they are enthusiastic about demands for greater transparency and disclosure in other professions but see the Greater Good served by a clubbish and insular "trust us, we're professionals" attitude in their own. My second was to recall this actual response received by a graduate student in response to a request of a professor of sociology for information relevant to reproducing results in a published article:
I am getting a bit tired of everyone trying to get in on the bandwagon to "replicate" my work. I am tired of people taking advantage of my generosity trying to make a fast buck and a name for themselves. I am wise to your deception. What makes you think you could fool me?

Ever heard of an original idea? You go out and have an idea of your own. Why don't you try that, instead of appropriating someone else's idea? [...]

In case you didn't know, you are in the business where ideas matter, your own ideas, not someone else's. Please leave me, and anyone else with original ideas, alone.
I guess one could say here that the graduate student could file an "ethics complaint" against the professor, although the ASA code of ethics is extremely vague about what help the professor is obliged to give, and it's always good to propose to people in subordinate positions that they should be comforted by the existence of quasi-litigious solutions that could be avoided in the first place if the organization/discipline had better practices.

* Update: Earlier, I said "precocious crank," because I associate crank-dom with being older than I am, so that if I was labeled a "crank" (as opposed to just, say, "crazy"), it would be an early achievement on my part. This apparently confused readers, so I revised it.


dorotha said...

wow. what a generous and kind professor. wha? why such a, um, jerk about it? especially since we are taught, since undergrad, that part of science is replicability and that sociology IS A SCIENCE.

okay, i'm a cruddy and qualitative* researcher and no one would want my data, but i hereby promise to make it as available as i possibly can.

* being cruddy has nothing to do with being qualitative.

Anonymous said...

I'm confused by your opening line. It makes it sound like you're precocious, and becoming a crank; or maybe you're a crank and becoming precocious. Could you clear this up?

As for resistence to sharing... just don't be like Merton and think that just because scientists say they are a community of ideas that they actually are. I think it's a decent idea, but logistics notwithstanding, it just isn't going to happen.

jeremy said...

Revision made. By "precocious" I just meant that I feel a little young to be establishing myself as a crank.

Anonymous said...

Wow, love the professor's reply. You gots to drop a dime on this character. Any hints? Come on, now, this could be fun.

jeremy said...

Absolutely not. (Other than that the professor has published in major sociology journal[s], which I note only to pre-empt the potential naive counterargument that no one who would be publishing at that level would send an e-mail like this.)

captain crab said...

Crank isn't too bad. You could be a:

grouch, bear, bellyacher, complainer, crab, crosspatch, curmudgeon, faultfinder, griper, grouser, growler, grumbler, grump, kicker, lemon puss, malcontent, moaner, sorehead, sourpuss, or a whiner!

You'll have to ask pj how I recieved my moniker!

A+ said...

to pre-empt the potential naive counterargument that no one who would be publishing at that level would send an e-mail like this.

Am I way off base to suspect that indeed, those publishing at that level might be even more apt to send an email like that?

Anonymous said...

Dude, just be scrupulously thorough with your own research -- and crank though you may well be -- let it stand. If it's good, you won't need to point out another's shortcomings. The contrast will do that for you.

jeremy said...

This isn't really about how well any of my work "compares" to other people's work. This is about the credibility of collective work.

That said, I have convinced myself that information about analyses are good for individual practice even in the absence of standards for collective practice.

Anonymous said...

Yeah, but it isn't coming across that way, in your blog, at least. Sounds not 'cranky' but 'prissy'.

jeremy said...

If by "prissy" we mean the dictionary definition of "prim and precise", then, where inference in quantitative social science is concerned, I am prissy and proud. If this really puts me at odds with prevailing thought in sociology, then, well, I refuse to conclude the problem is with me.

Anonymous said...

I agree with you about the value of making the programs we use for our data analysis publicly available. (If nothing else, imagine how much glee Guilli Jasso could get out of finding errors in her critics' analyses.) Unfortunately, I see the discipline moving in the opposite direction, thanks to the popularity of pull-down menus in our statistical software packages. If you ask me (and you didn't) pull-down menus are the scourge of accountability and replication. KAW.

Derlierprossy said...

I understand where your coming from, and I also simply appreciate the blog. For what its worth I will be back again soon, really innovative. Bye for now.