Chances Are Page 16
Sinclair succeeded: all 938 parishes returned records of their population, classes of inhabitants, agriculture, employment, manufactures, commerce, corn prices, and mortality—with a fine range of further detail besides. The real victory, though, came through the measures Sinclair, armed with his twenty-one volumes of incontrovertible evidence, was able to wring from the government: the creation of a Board of Agriculture, the reduction of coal taxes, the abolition of abusive charges for grinding corn, the increase in schoolmasters’ salaries, the improvement of sheep, and a royal grant of £2,000 to the families of Scottish clergymen. Moreover, the fact that such a complete statistical account could be collected by a single citizen gave the impetus for a national census of the rest of Britain, which would be—not a secret index of national power—but a public guide to legislation.
As the Industrial Revolution took off, the need to investigate the causes of events became ever more pressing. Why was life expectancy twice as high in Rutland as in Manchester? Why was mortality twice as high in London’s East End as in its West? Why did the river washing the Parliament of the world’s most powerful country smell so dreadful that sheets soaked in carbolic had to be hung over the windows? These practical applications required particular facts—and, in the 1830s, England became the world’s great producer of facts.
Parliamentary Committees, Royal Commissions, municipal statistical reports—all freely published, all widely circulating—made early Victorian England a place more self-aware than any before and many since. A contemporary, Douglas Jerrold, said that in 1833 no one thought of the poor, while by 1839 no one thought of anything else. The indefatigable Edwin Chadwick, secretary to the Poor Law Commissioners, bludgeoned local authorities into providing uniform answers and quantifiable reports on the effects of public hygiene on public health. Thanks to these, he was able to confront government with conclusions invincible because armored in fact: for instance, “That the annual loss of life from filth and bad ventilation are greater than the loss from death or wounds in any wars in which the country has been engaged in modern times.”
Reports were a start—but action was the goal. In a neat tale of statistical inference, Dr. John Snow was able to analyze and address one of the worst cholera outbreaks in London’s history. Five hundred people in Soho had died during two weeks of September 1854, all within a quarter-mile of one another. Taking the evidence of 83 death certificates and plotting the addresses of the dead on a map of the area, Snow came up with a visual distribution in two dimensions, centered on the pump at the corner of Broad and Cambridge streets.
Snow took his map to the parish Board of Guardians and got them to remove the handle of the pump; the epidemic died down. Not only were lives saved, but the association of cholera with contaminated water was established—up until then, it was thought possibly to be transmitted through bad smells. The site of the pump is now the John Snow public house; thanks to the efforts of those who put statistics into action, you can safely drink the water there.
The assertive power of statistics was expressed most clearly in the work of Florence Nightingale.
It is as criminal to have a mortality of 17, 19 and 20 per thousand in the Line, Artillery and Guards, when that in civil life is only 11 per thousand, as it would be to take 1,100 men per annum out on Salisbury Plain and shoot them.
The work she did in reversing the horrifying trend of death through illness in the British Army in the Crimea—at one time the rates suggested that the entire army would be dead within a year—was only a beginning. Resistance at the War Office (its minister, Lord Panmure, was known as “the Bison”) was so stubborn that all the lessons of the war could well have been lost. Words, even barbed and swift-shot words, could be countered and baffled by other words, soothing and evasive. Only numbers could make the point stick.
Miss Nightingale knew how to supply numbers in a memorable form (p. 138). Her command of graphics—her misleading “coxcombs,” frightening “bat’s wing” diagrams, and dour “lines” of mortality—made their incontrovertible mark without her presence, convincing the public, the monarch, and even her opponents that dispute with her was futile.
Florence Nightingale called herself a “passionate statistician.” Muffled by her time and gender into having to deal with the world at one remove, she had always been attracted to numerical information as a passport to the real. Her personal religion—an odd amalgam of Unitarianism and Quetelet—sought out the revelations of statistics:
The true foundation of theology is to ascertain the character of God. It is by the aid of Statistics that law in the social sphere can be ascertained and codified, and certain aspects of the character of God thereby revealed. The study of statistics is thus a religious service.
Not everyone, perhaps, would take an interest in statistics so far—but it was this evangelical fervor, this willingness to see the works of God in a table of figures and to act on them as if called to ministry, that goes a long way toward explaining the prevalence of numbers in public life today.
Nineteenth-century reformers were looking for variation, not for continuity. Miss Nightingale’s charts were designed to highlight alarming discrepancies. On a gross scale, when death rates drop 80 percent in a month, the figures require little interpretation: like a trout in the milk, they suggest their own cause. Other problems, though, were less tractable to statistical observation—most of all the problem of poverty.
Charles Booth was both the head of a large Liverpool shipping firm and a man with a heartfelt sympathy for the urban poor. They had, he believed, been failed in two ways: no institution had really bothered to find out how many there were, and no one really cared how they lived—except as a moral example to the rest of us. Faced in 1885 with a survey estimating that 25 percent of Londoners lived in extreme poverty, he said that he thought the authors “had grossly overstated the case”—yet there were no better numbers to go by. Booth’s practical mind went to work; he set up, at his own expense, the systematic study that was to become the three editions and seventeen volumes of Life and Labour of the People in London.
His methods combined the precise and the exhaustive: he looked at housing; he looked at employment; he looked at religion. He and his investigators interviewed all the London School Board visitors about the families in their districts, and accompanied policemen on their beats. Booth himself would take lodgings with poor families to understand better the precise composition of life in some of the more squalid warrens: “Families came and went . . . living in dirt, fond of drink, alike shiftless, shifty and shifting.”
Cross-referencing his material, he had come up with a scale of income from A (“occasional labourers, street sellers, loafers, criminals and semi-criminals”) to H (“upper middle class, servant keeping class”). It gave Booth the idea for a complete map of the metropolis, with every street colored to show its status, from the “vicious” black of Wapping through intermediate blues, purples, and reds to the gorgeous yellows of Mayfair and Belgravia.
Yet, as Booth himself said, there were “shades of black”—and this was his undoing. His decency, his warmth of heart, would not allow him to stop when further differentiation, better evidence, might allow him to understand more. And this was the problem: once beyond obvious discrepancies in social numbers, there was no sure science of variation. The interesting problems, the ones that appealed to active minds, were not those of the normal—what was needed was someone to lead the way down from the middle of the curve.
The combination of higher-than-average intelligence, energy, curiosity, and a large personal fortune can produce remarkable things. Francis Galton had the intellectual and physical vigor that characterized Victorians at their best. His portraits show him high-domed and sideburned, sleek, and alert: a racing version of his cousin Charles Darwin. You might take him for the ideal pattern of sporting country clergyman, the kind who wore riding boots and spurs under his cassock.
It’s not an image taken entirely at random: Galton had been,
in his youth, exalted at the idea of doing some great Good. Restless and dissatisfied, he set off exploring in South West Africa, where, wondering how he could respond to the expected splendor of his welcome by a local king in Ovamboland, he hit on the idea of arriving in hunting pink, riding a bullock. His notes for future explorers also recommended packing a folding camp bath and breaking an egg into each shoe before a long march.
One of Galton’s books on exploration was subtitled Shifts and Contrivances in Wild Countries—and it is easy to be so charmed by the sheer variety of his Victorian enthusiasms that one forgets the keenness with which his mind cut through to the essential task. He first tackled weather, persuading, by sheer force of character, all the weather stations in Europe to tabulate consistently the weather for the month of December 1861. Using his basically visual approach to data, he created symbols that simultaneously encoded wind, temperature, and pressure, thereby—incidentally—providing us with the isobars familiar from every modern weather map. He made photographic composites of “the criminal type”—and left us fingerprints as the basis for identification. He developed a rough-and-ready method for estimating the distribution of height among African tribesmen by lining them up in order and enumerating them by quartiles—the same system now used to rank fund managers. He created a data-entry glove with five counters, so that he could keep surreptitious track of five different variables of the world through which he strode: he measured directly from the streets the apparent rascality of different European nations and the prettiness of British women.
These may sound like the passing enthusiasms of the amateur, but the thread that connects them is ordering: Galton combined a naturalist’s eye for the fall of every sparrow with a scientist’s conviction that classification can reveal truth. All that he needed was an organizing principle general enough to cover even his vast prairies of interest—and he found it in Quetelet’s curve:I know of scarcely anything so apt to impress the imagination as the wonderful form of cosmic order expressed by the “Law of the Frequency of Error.” The law would have been personified by the Greeks and deified, if they had known of it. It reigns with serenity and complete self-effacement amidst the wildest confusion. The huger the mob, and the greater the apparent anarchy, the more perfect is its sway. It is the supreme law of Unreason.
It gave him, for instance, the means to test popular claims of the efficacy of prayer. Rather than enter the argument on its traditional terms, he devised an experiment at once simple and conclusive. The Church of England service includes a prayer for the health and long life of the monarch, in which all the congregation joins. One can therefore assume that the kings and queens of England, as the constant focus of such concentrated prayer power, would—if it were effective—live longer than the mean life span for wealthy, well-fed people of their time. Sadly, they did not. In fact, the mean life span of English monarchs was shorter than that of their gentry, indeed so much shorter as to suggest the counter-efficacy of prayer.
Two things entranced Francis Galton about normal distribution—and neither one was the stability of the average. What he loved was the shape of variation—how each element took its place in order along the curve—and the ability to infer the distribution of a whole population from a sample. Despite Quetelet’s appeals, it had become clear that we would not come to know everything about everybody; conclusions about society would have to rest on the assumption that the data which happened to be available somehow represented the whole.
Heredity was his passion. At heart an inspired experimenter, Galton wanted to give the investigator into heredity and the natural sciences the same tools of evaluation that astronomy and physics enjoyed. Naturalists had listed and described; but the appearance of Darwin and the re-discovery of Mendel suggested laws that could leave traces in the living world similar to those that Newton’s laws had left in the skies.
What—he wanted to know—made a Galton, a Darwin, a Wedgwood? Why had these families risen from the same patch of rural obscurity? Why were they different from the thoughtless masses? And why were there so few of them? Galton constructed time-series of prominent families, tracing how one brilliant generation grew from a single notable great-grandparent and soon withered to a single mildly distinguished great-grandchild (“shirtsleeves to shirtsleeves in three generations”). He put his faith in eugenics (he invented the word)—not in its chilling twentieth-century manifestations, but as a science of human perfectibility. Galton was all in favor of variation if it tended toward the better; he certainly did not share Quetelet’s view of the average man as the perfect embodiment of the just and the good: “Some thoroughgoing democrats may look with complacency on a mob of mediocrities, but to most other persons they are the reverse of attractive.”
Mendelian genetics and Darwinian selection offered abstract explanations for how human characteristics were inherited and how environmental conditions could select for this or that trait—but these obviously took generations to observe and test. Galton wanted to see results now. So, if he could not gauge empirically (as he had tried to) the intellectual development of English Men of Science, he would choose a hereditary phenomenon both easier to quantify and quicker to manifest itself: the size of pea plants.
He took 490 seeds sorted into seven categories by size and sent them off to seven friends to be grown, well away from one another’s influence. The seeds produced plants; the plants, seeds—which Galton’s friends sent back again, carefully labeled so that each batch of seeds could be associated with its progenitor. Galton weighed them and found that, just as the original seeds had been normally distributed around a mean weight, so each of the batches of new seeds were normally distributed—the descendants of heavy seeds around a heavier mean, those of the lighter around a lighter.
This in itself was not a complete surprise; using his quincunx machine, Galton had already seen that if you opened one of the channels at the bottom and let the shot fall through a further field of pins, it would form another bell-shaped normal distribution. Mathematically, you can add many small normal distributions together to produce a larger one—indeed, this quality had made Galton wonder whether this curve was so special after all: couldn’t a normal distribution of observed data be just a combination of lots of little causes blurred into one?
But Galton’s new peas weren’t just normally distributed—they revealed something surprising: the mean weight of each batch wasn’t the same as the weight of the parent seed, but subtly shifted toward the mean weight of the whole parent population. Heavy seed bred heavy seeds—but, on average, a little lighter than their parents; the descendents of light seeds were, on average, heavier—more “normal” from the point of view of their parents’ generation. Galton called this “regression to the mean,” and—while it may not explain why the children of hippie parents grow up to vote Republican, it makes the phenomenon slightly less surprising.
Struck by the results of his pea experiment, Galton collected information on the heights of 928 adults with parents living and willing to be measured. The results are presented in this table, one of the most information-dense summaries of observation ever produced. It shows the normally distributed heights of parents (each couple averaged as a “mid-parent”); the normally distributed heights of children; the number of children of each height for a given height of mid-parent; the lines of regression; and, traced in concentric ovals (only one of which is shown here) the first “bivariate distribution”—that is, the distribution in terms of each other of two normally distributed populations.
What Galton had found was both the foundation of modern statistics and the demolition of the idea of underlying social law. The definitions of a population, of regression, and of variance were all interlinked, parts of the same thing: the nature of a collective quality. They are innate to the data. They say: “Here is a genuine aspect of something,” an adjective with meaning for this collective noun. Measuring heights has meaning; asking people what’s the first number they can think of usually has not. The
re is no necessity for deeper law to explain the existence of normally distributed qualities; it is a law of being a quality.
Not all qualities, though, are normally distributed; a lot of human affairs are asymmetrical, lived on the skew. Most of us are healthier than the average, if we take the number of days a year spent in the hospital as a measure of health. On the other hand, most of us are poorer than the average; if Bill Gates were to walk into a Salvation Army soup kitchen, the diners would take little comfort from the thought that the mean net worth in the room was now billions of dollars. Galton knew that statistics would have to travel beyond the phenomena that conform to the error law. He also knew that he did not have the mathematics to lead it there; like Moses, he saw before him a land he was destined not to enter.
The Joshua to Galton’s Moses was a young man named Karl Pearson. If Francis Galton was the last representative of the great amateur naturalist tradition of gentleman scholars, Pearson was of the rising class of academic professionals: assertive, ambitious, territorial. He had studied in Germany in the 1870s, re-casting his Christian name from Charles, in deference to Marx. He had founded young people’s socialist discussion groups, where men and women could converse unchaperoned over tea and buns. His doctoral subject was political science—but he always understood “science” to be what was then current in German universities: the discipline that has since colonized the world, emphasizing laboratory work, uniform method, careful measurement, clear recording of raw data, and application of overt tools of analysis. This science needed ways to relate measurements, not just correct them, as Galileo and Tycho had done. “Correlation,” a word that Galton had invented almost in passing, was the goal: finding the curve that best approximated the relative values of two or more sets of measurements and judging how far off that curve they lay—thus determining the degree of confidence a scientist could place in the assumption that they meant something to each other.