For those of us who live elsewhere, the frequency with which mature democracies make questionable, ambiguous or downright foolish choices can either  be read as an encouragement – that every country has  bouts of political myopia – or as proof that democracy teaches by trial and error and that mistakes are a necessary part of the electorate’s education. A case in point is Britain’s recent general election. After six weeks of tireless campaigning, the three main political parties  have had to accept the humbling truth that none of them did enough to convince the public of their fitness to lead the country. Although the Conservatives and Liberal Democrats forged a coalition with unusual speed, and did so with little apparent ill-will, the anxieties of all the parties, and the electorate that refused to grant them a majority, remain essentially unchanged.

At first glance, this doesn’t seem like progress, but the appearance is deceiving. In fact, the delicate balance of power which the election has imposed on Westminster has set the stage for a new kind of politics, one that happens to suit the characters and political temperaments of the current party leaders very well. The new coalition will require constant horse-trading, and this will grant both Cameron and Clegg a licence to move their parties, and the country, beyond a long period of reflexively adversarial politics. This was not a foreseeable denouement, even a day before the election, but with hindsight it does seem like quite a desirable result. The political columnist Matthew Parris has even written that “[i]t borders on the supernatural. No imaginable electoral outcome could have been more intelligently designed for the shape and tone of the government these two men already wanted to lead; no outcome could more securely have protected them from enemies within their own ranks.”

The American satirist H L Mencken once joked that nobody “ever lost money by underestimating the intelligence of the great masses of the plain people.”  It is tempting to feel this way about some recent developments in the United States, where the recent triumph of a “Tea Party” candidate in the Republican Senate primary in Kentucky has prompted a further spate of handwringing among those who fear for President Obama’s chances of winning a second term. The earlier loss of a Senate seat in Massachusetts, the resurgence of Sarah Palin, and the rise of the egregious Glenn Beck have led many pundits to conclude that the American public has become too skeptical of the new president.

According to this analysis, voters would rather elect populist candidates who have little chance of changing anything, than persevere with a reforming president whose efforts have so far produced only modest results. But the truth may be more nuanced. For example, Scott Brown, the new Republican Senator for Massachusetts, the man who broke the Democrats’ supermajority in the Senate, and nearly derailed Obama’s health reform bill, has hardly turned out to be the thorn in Obama’s side that his supporters had hoped. Just a few days ago, Brown was one of only four Republicans in the Senate to help the Democrats pass a financial reform bill.  This is hardly what the Tea Party thought they were getting, but it is quite consistent with Brown’s earlier political choices – he is a socially liberal Republican who has recently voted for a jobs bill and an extension of unemployment benefits.  As for the other rumblings of discontent which Obama faces, these have also extended to a large swathe of Republican incumbents deemed guilty of under-performing. The so-called populist anger may even end up costing Senator McCain his seat in Arizona. In other words, public skepticism is hardly limited to the President’s activities, and it may well end up strengthening his hand in the months leading up to the mid-term elections.

Several years ago the financial journalist James Surowiecki wrote a fascinating book on the way that large groups of people can often exhibit greater practical wisdom than the most intelligent individuals among them. The Wisdom of Crowds argued that in many scenarios, particularly those in which data is incomplete and unclear – such as when crowds at a fairground are asked to guess the number of marbles in a jar, or the weight of a cow – the estimates of large groups of reasonably well-informed people will almost always be more accurate than those of a small number of experts. Surowiecki showed that the same dynamic could be used to make better predictions about the economy, about commodity prices, even about something as difficult to predict as the date of an upcoming terrorist strike. Political outcomes, of course, tend to follow a similar pattern, especially in democratic countries with knowledgeable citizens.  Seen in this light the British election and the fallout from the burgeoning Tea Party movement – that curious mix of political bluster, inexperience and naivety – are not random deviations from the norm, but the wisdom of crowds reshaping political reality, in the same way that Adam Smith believed an invisible hand guided markets and economies.

Perhaps that is the real lesson of political life in mature democracies. Even with flawed voting systems, incompetent politicians, troubled economies and other handicaps, the wisdom of crowds can make an important difference. Democracies do not get to make perfect choices, but they often produce unexpected outcomes that turn out to be far more intelligent than anyone might have predicted.