Monday, November 06, 2006

big three, big whoop

Okay, so I'm really not interested in belaboring this, but in my recent post on the Notre Dame "productive faculty" rankings, some commenters have implied that I am ignorant about journal impact factors and that if I looked at them I would see what an obvious "Big 3" there were in sociology. I don't get what other people are looking at and, if people are going to hector me about being ignorant about evidence, I want to know what this evidence is. I pull up the ISI Web of Knowledge impact factors right now (note, not based on something somebody remembers reading sometime somewhere), and this is what I see:
1 AM J SOCIOL - 3.262
2 AM SOCIOL REV - 2.933
3 ANNU REV SOCIOL - 2.521
4 SOCIOL HEALTH ILL - 2.169
5 SOC PROBL - 1.796
6 SOC FORCES - 1.578
7 BRIT J SOCIOL - 1.490
8 LAW SOC REV - 1.396
9 SOC NETWORKS - 1.382
10 J MARRIAGE FAM - 1.350
Just to be clear, this is only incidentally relevant to the point I was arguing, which was more about whether there was a "Big 3" or a "Big 2" followed by subfield-dependent ambiguity as to how SF was regarded relative to the top journal in that subarea. It's weird regardless how some people believe that if you question whether there is a "Big 3" or just a "Big 2," people think showing something is #3 is a knockdown counterargument. I am sure they would also argue that there are a "Big 3" of human sexes, alongside "male" and "female", if one type of intersexedness was demonstrably more common than others.

(Note: I have problems of my own with journal impact factor measures anyway--although not nearly as large as the problems I have with the equation of citation counts with the merits of individual scholars.)

23 comments:

Anonymous said...

Michael Allen's most recent analysis, published in Footnotes (2004?) has Social Forces third. Also, on the list posted here, Social Forces is the third in terms of anonymous peer-reviewed, generalist journals. Still, ASR/AJS do appear to constitute a kind of 'big 2'.

jeremy said...

My point, yet again, is not that SF is not the top generalist journal that is not AJS or ASR. (That said, many people would call Social Problems a generalist journal.) Remember also, SF publishes the most articles, and so is most important for the Notre Dame rankings.

dorotha said...

while i barely give a crap, i understand what you are getting at. props for being persistent about this issue in the face of the social forces contingency.

Anonymous said...

Wow, you wouldn't think the Annual Review would score so highly. It's readable, relevant, and includes minimal regression coefficients.

I don't have any sophisticated measurement formulas to back this up, but as a general sociologist, I find ASR, Social Forces, and to a lesser extent, AJS practically worthless. {Perhaps I deserve to be flamed for such blasphemy... I'll take it like a man.}

There may be one article in each journal, per year (give or take) that I find, interesting, important, and useful. As penance, I've been trying to give ASR a chance... I take each issue and try to learn something outside my area of emphasis (punishment and social control). But effort required to understand these articles is not proportional to the payoff in knowledge gained. In contrast, I always learn something useful from an investment in reading an Annual Review essay.

From a sociology of knowledge perspective, it strikes me that the big three will always be ASR, AJS, and SF, because everyone knows those are the big three, regardless of impact scores, citation counts, or some other measurement. That is to say, the the big three have been institutionalized as such.

None of the above should be taken as a specific criticism of:
- the journals, their editorship, or readership;
- any particular paper and its respective authors;
- any readers of this blog who happen to find the meaning of life, the universe, and everything in the pages of ASR.

Tis one man's opinion, nothing more and nothing less.

Anonymous said...

"Social Forces. The 'modal sexually-ambiguous genetalia' of sociology journals". Has a nice ring to it.

Anonymous said...

That's all fine and good Corey, but you are admittedly confused.

Anonymous said...

Although one may debate the validity of impact factors for accurately reflecting the potential influence of articles published in particular journals, they should never be the sole measure of the prestige of publications. (I'm not saying that you are making that claim, Jeremy - I understand that you are simply making a point about the gap that exists between the top two journals in your field and whatever comes next).

On the subject of impact factors, check out the numbers for the most prestigious scientific journals - Nature and Science. They are many times greater than those of any social science journal (as far as I know). In large part this may reflect the fact that refereed journal articles dominate the publishing paradigm in the sciences to a greater degree than they do in the social sciences. But they also reflect the fact that scholars from many fields read Nature and Science. In my field (anthropology), students and professors in the most prestigious departments distinguish themselves by publishing in Nature and Science. Writing the cover article in Nature is like having your picture on the cover of Rolling Stone. It counts for much more than an article in American Anthropologist (our flagship journal).

The Annual Review in any field has a high impact factor. But, as the the journal title implies, the articles are reviews not research articles. They may be widely cited but they don't make original contributions. In my field, its considered nice if you are invited to write a piece for the Annual Review (I did one the year after I finished my Ph.D.). But its not perceived to be the home run that a research article in Nature or Science is.

Citation statistics (which I know you also mentioned) should be considered as well.

One of the things that I found interesting when I was considering the relative prestige of journals is the number of libraries that hold a particular journal. This can be found out by searching WorldCat. In general, I think it is safe to say that the more prestigious journals are held by more libraries.

One of the surprising things I discovered recently is that certain of my colleagues (in my department) publish articles in journals with ridiculously low impact factors. These journals are also held by few libraries.

Anonymous said...

Still, low impact is better than no impact.

Felix said...

Compare the AJS/ASR impact factor of 3.2/2.9 to that of the American Economic Review = 1.8. This points to the importance of discipline-specific tendencies to cite more or less works by others. In AER, you can get away with making an original contribution while citing only 5 works of others (that had better be truly relevant). In sociology, I wager that ASR/AJS wouldn't even send your stuff out for review unless you have a good number of references, 30-70 seems like a good starting point. I'm not sure the discipline is helped by requiring authors to track down that very first time a sociologist has said something vaguely related to every half sentence in the manuscript. (But I'm young, so I try to cite as many potential reviewers as possible and generally strive to link my interests back to Durkheim.) - - OK, this was a bit rambly. Anyway, I'm with Jeremy regarding suspicion of the one-stats-fits-all strategy of ascertaining quality. If I want to know if somebody is doing good work on my own objective Felix Quality Criterion, I read their work.

Anonymous said...

Jeremy,
You make some really good points.

Here is the Allen article that some have mentioned.

http://www.asanet.org/footnotes/dec03/journalinfltable.pdf

He does find that SF is number 3 in centrality. But, your point about the distance between SF and ASR/AJS is well-taken. As well, Allen looks at 1999-2001 and it could be that SF has declined since that period. My own look at web of sci does not support that point, however. Aside from 1999, it appears that SF has actually been higher in the last couple of years than it was in say 2000-2002. If I understand correctly, Allen's measure is lowered if a journal ups its number of articles.

Of course, we should appreciate that impact factors fluctuate from one year to the next. So, any one year might not be a fair "test" of the impact of a journal. Moreover, SF's publishing a lot of articles depreciates its impact factor as that is a rate per article. Maybe the best way to measure impact factor would be some average of the past ten years.

But, I agree with a previous poster in that we can't underestimate the role of the institutionalization of prestige in these journals. Surely, this has a consequence on reviewers (and their expectations for what to accept), departments (and how they evaluate their faculty), and authors (in their choices of where to submit). Of course, that gets us back into the thicket of prestige subjectivity that ND had intended to challenge.

Last but not least, if you really want to debunk the ND article (as it appears you do), the really potent critique is the neglect of books. Surely, we can't underestimate the impact of all the influential books that places like Berkely write, despite their limited presence in the "top 3" of soc journals.

Maybe the best way to read the ND article is for what it is. They are showing that some depts are underappreciated relative to their actual productivity - albeit with a limited measure of productivity.

Anonymous said...

"Surely, we can't underestimate the impact of all the influential books that places like Berkely write, despite their limited presence in the "top 3" of soc journals."

Truer words have never been spoken!

Anonymous said...

asr, ajs, and sf are controlled by upper class white men (here and there a woman who is like-minded will be put in charge) and surprise surprise, the majority of people who publish artices in these journals are upper class white men (and a surprising number of like-minded women due to the demographic structure of discipline)

they like to publish boring articles that only other sociologists care about ...public has no use for most of the rubbish printed in "the big three", but every once in a while a study relevant to public will be printed

Minimal publication of qualitative findings and impression management that sociology is a "hard science" by printing numerous formulas that researcher got from the latest (spss, stata, sas) manual

career significantly enhanced by one or more publications in "big three"

It may be a better choice to get some of our trees and wild life back and compel ASR, AJS, SF (if they must) to just start posting the journal as a series of PDF files online

let's face it, when you die, your (but only a select few) articles in any of these journals may get some discussion in graduate seminars, but no one outside of the discipline will likely be impacted in any substantive way by anything that you were ever able to "get published" in ajs, asr, or sf

I know many of the white-male- capitalist-free-market-power structure- of-sociology readers will burn me at the stake on the job market for saying these things, b/c it is a big deal to call out the power structure on its b.s., so I will post anonymously.

Anonymous said...

upper class? please.

Anonymous said...

strategy one: denial

Anonymous said...

Ummmm. ASR and AJS are not targeted, written, marketed or edited for the general public. They are ACADEMIC journals. Thus, they may not be sitting around on people's coffee tables, but I think the topics are "relevant." Let's see, the latest issue of ASR has an article about public debate after 9/11, Stonewall, the spread of sit-ins in the 60s, fame in baseball, child's life chances, among others. These topics are not "relevant" to the public?

Anonymous said...

Bush isn't racist, he's got Condie Race on his staff.

Anonymous said...

The upper class all male power structure of sociology?

This is the most politically left field in the social sciences. It's also one of the most integrated; a field where most people truly care about equal opportunity and diversity. Women are very successful in sociology, especially as compared with other disciplines. If you feel oppressed by SOCIOLOGY of all things you have lost all sense of proportion.

Anonymous said...

you're a dreamer

Anonymous said...

If you feel oppressed by SOCIOLOGY of all things you have lost all sense of proportion.

I'm not the original anonymous, so perhaps I'm wrong in characterizing the argument, but it's possible that what's so troubling is that sociologists (and maybe others, I have no idea) are liberal on paper, but not too many actually walk the talk, as it were. I know plenty of male sociologists who, when presented with anything having to do with gender, immediately discount it as being an overreaction, and even have the audacity to assert that "it's just as bad for men." The first few times, sure, maybe it's true; this situation or that could be "just as bad" for men, or whites, or whatever. But every single time?

Further, the assertion here that because sociology is "more integrated" than other comparable disciplines, that we should shut up and take it, is so ludicrous as to be laughable.

Anonymous said...

"Further, the assertion here that because sociology is "more integrated" than other comparable disciplines, that we should shut up and take it, is so ludicrous as to be laughable."

Well, I don't know where you got the idea that someone told you to "shut up," but it's certainly true that sociology is one of the most race and gender-integrated fields in academia. Don't forget it wasn't long ago that academia was 95% white males.

Anonymous said...

I agree that sociology is an integrated field, but it seems like it wouldn't be that difficult to figure out if the publications in the top journals are as gender or racially integrated.

I'm new at this, but my few trips to ASA meetings show great gender and racial gaps as far as sections go, and that's got to play some part in where they work and publish, their prestige in the discipline, and so forth.

Anonymous said...

"Shut up and take it" is a saying, a turn of phrase. It's taking issue with the implication that, "things are much better now than they used to be, so why all the grousing?" I mean, do we say the same thing about racism? Okay, many of us do say the same thing about racism. But I'm merely attempting to say that "better than before" doesn't necessarily mean "good."

Anonymous said...

Seems to me like sociology is integrated not because sociologists are so gender and race neutral but because the discipline is losing its prestige, relative to, say, Econ or the sciences. If you look at undergraduate majors, they tend to be more minority and women too.