kfitz

Tools and Values

I’ve been writing a bit about peer review and its potential futures of late, an essay that’s been solicited for a forthcoming edited volume. Needless to say, this is a subject I’ve spent a lot of time considering, between the research I did for Planned Obsolescence, the year-long study I worked on with my MediaCommons and NYU Press colleagues, and the various other bits of speaking and writing I’ve done on the topic.

A recent exchange, though, has changed my thinking about the subject in some interesting ways, ways that I’m not sure that the essay I’m working on can quite capture. I had just given a talk about some of the potential futures for digital scholarship in the humanities, which included a bit on open peer review, and was getting pretty intensively questioned by an attendee who felt that I was being naively utopian in my rendering of its potential. Why on earth would I want to do away with a peer review system that more or less works in favor of a new open system that brings with it all the problematic power dynamics that manifest in networked spaces?

In responding, I tried to suggest, first, that I wasn’t trying to do away with anything, but rather to open us to the possibility that open review might be beneficial, especially for scholarship that’s being published online. And second, that yes, scholarly engagements in social networks do often play out a range of problematic behaviors, but that at least those behaviors get flushed out into the open, where they’re visible and can be called out; those same behaviors can and do take place in conventional review practices under cover of various kinds of protection.

It was at this point that my colleague Dan O’Donnell intervened; by way of more or less agreeing with me, Dan said that the problem with most thinking about peer review began with considering it to be a system (and thus singular, complex, and difficult to change), when in fact peer review is a tool. Just a tool. “Sometimes you need a screwdriver,” he said, “and when you do, a hammer isn’t going to help.”

Something in the simplicity of that analogy caught me up short. I have been told, in ways both positive and negative, that I am a systems-builder at heart, and so to hear that I might be making things unnecessarily complicated didn’t come as a great shock. But it became clear in that moment that the unnecessary complications might be preventing me from seeing something extremely useful: if we want to transform peer review into something that works reliably, on a wide variety of kinds of scholarship, for an array of different scholarly communities, within a broad range of networks and platforms, we need a greatly expanded toolkit.

This is a much cleaner, clearer way of framing the conclusions to which the MediaCommons/NYU Press study came: each publication, and each community of practice, is likely to have different purposes and expectations for peer review, and so each must develop a mode of conducting review that best serves those purposes and expectations. The key thing is the right tool for the right purpose.

This exchange, though, has affected my thinking in areas far beyond the future of peer review. In order to select the right tool, after all, we really have to be able to articulate our purposes, which first requires understanding them — and understanding them in a way that goes deeper than the surface-level outcomes we’re seeking. In the case of peer review, this means thinking beyond the goal of producing good work; it means considering the kind of community we want to build and support around the work, as well as the things we hope the work might bring to the community and beyond.

In other words, it’s not just about purposes, but also about values: not just reaching a goal, but creating the best conditions for everyone engaged in the process. It’s both simpler and more complex, and it requires really stopping to think not just about what we’re doing, but what’s important to us, and why.

If you’ll forgive a bit of a tangent: I mentioned in my last post that I’d been reading Jim Loehr and Tony Schwartz’s The Power of Full Engagement, which focuses on developing practices for renewing one’s energy in order to be able to focus on and genuinely be present for the important stuff in life. I only posted to Twitter, however, the line from the book that most haunted me: “Is the life I am living worth what I am giving up to have it?”

At first brush, the line produces something not too far off from despair: we are always giving up something, and we frequently find ourselves where we are, having given up way too much, without any sense of how we got there or whether it’s even possible to get back to where we’d hoped to be.

But I’ve been working on thinking of that line in a more positive way, understanding that each choice that I make — to work on this rather than that; to work here rather than there; whathaveyou — entails not just giving up the path not taken, but the opportunity to consider why I’m choosing what I’m choosing, and to try to align the choice as closely as possible with what’s most important.

In the crush of the day-to-day, with a stack of work that’s got to be done RIGHT NOW, it can be hard to put an ideal like that into practice. And needless to say, the opportunity to stop and make such choices is an extraordinary privilege; thinking about “values” in the airy sense that I’m using it here becomes a lot easier once things like comfort, much less survival, are already ensured.

But this is precisely why, I think, those of us in the position to do things like create new programs, or publications, or processes, need to take the time to consider what it is we’re doing and why. To think about the full range of tools at our disposal, and to select — or even design — the ones that best suit the work that is actually at hand, rather than reflexively grabbing for the hammer because everything in front of us has always looked like a nail.

So, an open question: if peer review is genuinely to work toward supporting our deeper goals — not just getting the work done, but building the future for scholarship we want to see — what tools do we need to have at our disposal? What of those tools do we already have available, even if we’ve never used them for this purpose before, and what new tools might we need to imagine?

Webmentions

No replies yet.