October 4, 2006

Note to ISI: Please, dear god, stop trying to predict the Nobels

First, the facts related to the science Nobels:

Physics

ISI Predictions


  • Guth/Linde/Steinhardt
  • Fert/Gruenberg
  • Desurvire/Nakazawa/Payne

Actual Winner: Mather & Smoot


Medicine or Physiology

ISI Predictions

  • Chambon/Evans/Jensen
  • Jefferys
  • Capecchi/Evans/Smithies

Actual Winner: Fire & Mello

Chemistry

ISI Predictions

  • Marks
  • Evans/Ley
  • Crabtree/Scheiber

Actual Winner: Kornberg

Update: Economics

ISI Predictions

  • Bhagwati/Dixit/Krugman
  • Hart/Holmstrom/Williamson
  • Jorgenson

Actual Winner: Phelps


So, they got every single one completely wrong (Including the economics prize, not initially part of my post). Now, I don't think they do this bad every year with their predictions, but hopefull their completely, ridiculously, stupidly, ignorantly, shamefully awful performance this year will convince them that the science Nobels aren't awarded on the basis of citation analysis. The main point is that influence is not necessarily reflected in raw citation counts. Different fields and subfields can have varying scholarly communication & citation patterns that make it foolish to try and compare different scholars purely on the basis of citation counts. Citation counts do not equal impact. In the same way that librarians would not just use journal impact factors to make journal subscription decisions, I am sure that citation counts play next to no role in the Nobel Committees' decisions.

Please, please, bibliometrics is a good and useful pursuit that can tell us many things. Don't try and tell us that it's any good at measuring true research impact (or true research quality, anymore than the literature prize should be decided by book sales), you only damage your own credibility.

No comments: