Wisdom of the crowd versus costs of coordination: The case of Wikipedia.

Prof. Robert Kraut

HCI Institute, Carnegie Mellon University

Fri Mar 14 15:10:00 NZDT 2008 in Room 031, Erskine Building

Abstract

Linus's Law, "Given enough eyeballs, all bugs are shallow." (Eric Raymond, The Cathedral and the Bazaar)

Brooks' Law, "Adding manpower to a late software project makes it later." (Fred Brooks, Mythical Man-Month)

The Internet provides the technology to widely exploit the 'wisdom of the crowd' phenomenon, in which combining judgments from many people allows a crowd to outperform experts. Internet-enabled wisdom-of-the-crowd services has been institutionalized in collaborative filtering systems, distributed brainstorming systems and prediction markets. However, the coordination costs needed to combine individual contributions may undercut the benefits that each contributor makes. Our research examines how the number of editors contributing to Wikipedia articles and the techniques used to coordinate their work influence the articles' quality. Results replicate those of Wilkinson and Huberman (2007), in showing a correlation between the number of editors who work on a Wikipedia article and its quality. However, this correlation occurs because high quality articles attract more editors, not because having many editors work on an article necessarily produces high quality . We show that the use of two coordination techniques improves article quality. Articles improve when editors use explicit coordination, in which they devote energy to discussing articles as they write them. They also improve when editors use implicit coordination, in which a small subset of the editors lead the project by doing most of the work. Use of these coordination techniques improves quality most early in an article's history, when the article is initially of low quality, and when more people are editing. Although we developed these results in the context of Wikipedia, we believe they apply generally to attempts to harness many people to do work that requires integrated planning and execution.