Dear colleagues,
Which would you trust more to select the right answer from a series of options: an expert or a large crowd of non-experts? There is evidence from the television show Who Wants to be a Millionaire?: the crowd is far better. Is there something deeper here?
Scott Page at the University of Michigan has run simulation experiments examining decision-making by groups of 10-20 agents. Surprisingly, the groups did better when some of the agents were not-so-smart than when all of the agents were smart. Mixing in not-so-smart agents made the groups more diverse, adding information and improving decision-making.
The Iowa Electronic Markets, with about 800 traders betting on the outcome of elections, usually comes up with more accurate predictions than the major polls. Furthermore, the market forecasts are much less volatile than polls.
These are a few of the many examples given by James Surowiecki in The Wisdom of Crowds. Surowiecki says there is great power in combining numerous independent judgements: "if you can assemble a diverse group of people who possess varying degrees of knowledge and insight, you're better off entrusting it with major decisions rather than leaving them in the hands of one or two people, no matter how smart those people are. If this is difficult to believe ... it's because it runs counter to our basic intuitions about intelligence and business" (p. 31).
To present his case, Surowiecki presents diverse examples, including Google, gridiron tactics, plank-road fever, US spy operations, Linux, attendance at bars, Quaker businessmen, paying tax, traffic congestion pricing, scientific collaboration on SARS, the fashion clothes company Zara and sharemarket dynamics. He engagingly explains the significance of such examples, citing diverse research findings.
There are four conditions for crowds to make wise decisions: diversity of opinion, independence, decentralisation (so that people draw on local knowledge) and aggregation. The production of free software satisfies these conditions, but the US intelligence community doesn't because there is no means for combining information and judgements. But an attempt to set up a decision market for intelligence purposes, with the market serving to aggregate independent judgements, was fiercely attacked by politicians.
Surowiecki's argument is part of a wider promotion of peer-to-peer alternatives to conventional top-down decision-making. The Wisdom of Crowds is undoubtedly one of the most entertaining accounts in this area.
The Arts Faculty already uses some processes that fit the conditions set out by Surowiecki. For example, in ranking research student scholarship applications, members of the Faculty Research Committee independently rank each application and the rankings are combined to give medians and averages. This system appears to work far better than the previous system in which the conditions of independence and aggregation were not satisfied as well. The implication of Surowiecki's argument is that the FRC could do even better by including more people in the ranking process, including ones with less expertise, so long as they draw on different knowledge and contribute to diversity of opinion. For ranking purposes, a broad cross-section of the faculty is as likely to reach good decisions as a small group of the most experienced members. That's counter-intuitive!
It's possible to imagine using crowd-based decision-making for some other important topics, for example picking the next big trend in student interest or where research breakthroughs can be made. The faculty could set up a decision market, in which members make bets on options, and make its choices based on the state of the market. It sounds weird, but it could be that the first academic groups willing to take the wisdom of crowds seriously will be able to make wiser decisions than their more prestigious peers.
Brian Martin
21 February 2008
James Surowiecki, The wisdom of crowds: why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations (New York: Doubleday, 2004).