Peer-Review

• 10 September 2003
• academia,
• publishing

Thanks much to Jason for his recent entry rounding up some related thoughts at blogs around these parts and raising some interesting questions about their conjunctions. Some of my recent fretting figures into this round-up, in an ironic fashion: I worried about my sense that I was shouting down a well, and lo, but I got a response.

Nonetheless, I remain filled with anxieties (and perhaps this has something to do with the fact that my department met today to debate the Big Question looming in my academic life), a number of which revolve around the relationship between time-devoted-to-blog and need-to-produce-substantive-proof-of-scholarly-activity. I argued, in my personal statement, that the work I do here on^[1] Planned Obsolescence needs to be taken seriously in terms of my research, as the site provides both a site of research, in which I can observe the development of new writing forms first-hand, and a sort of open forum, in which I can test out and receive feedback on the various ideas I’m in the process of forming. In that regard, the blog seems both virtual laboratory and ongoing conference, and needs to be taken seriously by one’s peers.

Ah, but the question of the peers raises the question of peer review. As Jason asks:

Can comments and trackbacks — in some fashion — lead to a sense of peer review (or do they already)? If not that, what kind of peer review could we imagine if it were facilitated by technology? Clearly, there are issues with ignoring the “blind” review process (and if blogs were the model, blind review would be nigh-impossible); E-Bay style procedures are unlikely to benefit academia, since reputation-based evaluation is hardly an effective measure in an environment that, like so many others, can burn you with a wayward comment. The superstars are unlikely to be criticized, and even the newbies will likely find themselves either coddled (because to do otherwise might ruin their career) or cut [again, the Chronicle has some interesting articles on these issues].

As Jason points out, the eBay system of evaluation has proven less than useful — an almost viral grade-inflation has resulted — and as the conversation that resulted from George’s post entitled “Conversation as Game” suggested, Slashdot-style “karma” systems of evaluation are both flawed and massively subvertible. One might argue that a trackback functions like a citation might, except that not all blogging systems trackback (and not all bloggers with trackbackable systems do, either). So what other kinds of peer-review/evaluation systems might we imagine?

• ← Previous
  Things I Wish I Had Written, Part 1
• Next →