Participation Literacy

What is the nature of intellectual work in the peer review process? Reviewing a submission involves drawing on certain criteria (e.g. theoretical content; empirical content; presentation quality; appropriateness for the publication) to evaluate the quality of reasoning and evidence provided, to probe for weaknesses, acknowledge strengths, and question background assumptions. In effect, reviewing is an argumentative process where reviewers are engaging in an imaginary debate with distant authors who are not present to respond to their analysis. This paper-based review model has shortcomings in that questions go unanswered; confusions go unclarified; criticisms go undefended. The dynamic cut-and-thrust of debate normally found in face-to-face contexts such as workshops or conferences is not supported by the paper-based review processes, nor is it yet being realised in the new electronic media.

We are currently rethinking the review process to use new technologies in order to recapture the best features of a dynamic scholarly debate. This rethinking is guided by existing research into hypertext-based, computer-supported collaborative argumentation. Argumentation research is concerned with developing notations and tools to facilitate public debate and negotiation. (Sumner, 1996)

The text above was written 1996 by Tamara Sumner and Simon Buckingham Shum of the Knowledge Media Institute of the Open University. If I would try to analyse the Open Peer Review discourse, the result would not be so flattering for the academic society (from my viewpoint). A search on the Internet for sources of “open peer review” articles resulted in the fact that the open peer review discourse seemed to have had its peak around 1996. Since then Open Peer Review has had a steady stream of voices, such as João Pedro de Magalhães (2004) and Richard Smith (1999).

The problems with the traditional peer review system are linked to the question of how authority is created and distributed. This question is raised by Elisabeth Gulbrandsen, Albert Nsengiyumva, Birgitta Rydhagen and Lena Trojer in ICT, innovation systems and the role of universities in societal development - a (post)colonial strain? (Gulbrandsen et al., 2004):

One important aspect of informal knowledge is the notion of authority or lack of authority in a text. The ability to recognize such authority is hard to make explicit and thus difficult to achieve. “Very few scientists can answer questions about why certain texts give an impression of ‘competence’ while other texts don’t” (op. cit., p. 25, my translation). Gerholm and Gerholm describe this ability as a feeling for how authority is created in a text or a lecture, for what counts as an argument, for the common attitude towards the surrounding world and for the personal style accepted by colleagues. What we want to leave behind as outdated conceptual models may live on as cultural frameworks, showing itself spontaneously in practice as a “theory-in-use”.18

This is not a call for any old or new liberalism, but I think Wendy Hollway makes a point by stating that: “Science as we know it could only become dominant because it was preferred” (Hollway, 1989, p. 11).

This text questions the idea of formal authority as valid judge for knowledge claims. In the Open Peer Review discourse, the word open is a key word, but open can, and does, mean different things. Most often it means transparent as opposed to hidden or closed: “The primary argument against closed peer review is that it seems wrong for somebody making an important judgment on the work of others to do so in secret. A court with an unidentified judge makes us think immediately of totalitarian states and the world of Franz Kafka” (Smith, 1999). As Magalhães says “Anonymity is based on the principle that anonymous reviewers will be more honest and objective. For example, open peer review may hinder junior scientists from rejecting works by more senior colleagues. This cowardly behaviour is nearly exclusive of science. In the arts and even in politics criticism is open and serves an important role in making ideas stronger” (2004). Most voices in the Open Peer Review discourse want to have a transparent peer review process. Starting from a Web 2.0 perspective, open should mean Open in the words every sense. The peer review process should not only be transparent, it should also be open for participation.

Why willingly create borders at all in the review process? For a transdiciplinary mind, this is really hard to understand. In more closed and highly specialised disciplines, such as medicine or physics, the walls are kept to keep the ignorant people out. We all know these explicit walls are unnecessary, because most of us ignorant people would never think of the idea of making a contribution to the discussion, since we know we are ignorant, and we would not want it to be otherwise. But the walls also stop possible critique regarding research methods.

Open Peer Review in the spirit of Web 2.0 would mean complete openness in the research process. No researcher has to be hesitant of “The cult of the amateur” in the researchosphere. In 9 cases of 10, the amateur does not have the right words to understand a research discourse and even less to make a contribution. When an amateur raises his or her voice it is most often wise words worthy of attention. We are fools if we do not take them / us seriously, and still more fools if we try to stop them / us from entering the conversation. We are all amateurs when it comes to most issues, and we are all experts when it comes to some issues.

An Open Peer Review research facility could be realised in many different ways. I am going to sketch a Web 2.0 inspired publishing environment for the transdiciplinary field of Technoscience Studies.

The website would of course be developed on top of an open source environment. The interface would preferably be ajaxian lightweight with instant responses when a link or button is clicked. Anyone could join this community. You would not have to join just to read texts, only to write, but the point is that the communication would be bidirectional so hopefully most of the users would register with the community. The community would have to be open for reading so the search engines could index the site. Your identifier in this community would be your real name, not an avatar or something like that. Your identity is connected to an “about-page”, where you are requested to write about your context. This context is very important, since the context is integrated in everything you write, both your own texts and your comments of other texts.

Everyone has the same right to write articles and comment on other participant’s texts. Both writing texts and making comments are viewed as valuable kinds of participation. An important fact is that commenting on other author’s texts has the same potential value as writing your own. The Web of comments in which a user has participated should be collected by the system and displayed in the same obvious way as the articles of the person in question. The network of a person’s articles and comments is a person’s Web of participation.

Every participated item can be valued by everyone. In practice, this evaluation might be realised by putting an evaluation box in connection to all articles and comments. In this box you could give a quick response to the text, and also see an aggregated view of how other readers have evaluated this article or comment.

With all this user participation the CI machine could be created to do several interesting tasks. You could, for example, let the CI machine work out the most useful users and mark their name with an icon separating them from the others. This kind of hierarchy creation would simulate how hierarchies are created in real academic situations – in the best of all worlds. In real academic life it is not only the value of your texts that places you in the hierarchy. It could for example be more difficult to gain recognition if you are a woman (Wennerås and Wold, 1995) (Wennerås and Wold, 1997), or belonging to some kind of minority. It is also an advantage if you know people in strategic positions. Open peer review and collective intelligence could create a more text related hierarchy – research democracy.

Open Peer Review is the obvious review system for research 2.0, and for future development of research communication.