THE MATHEMATICS OF PREDICTIONS AND ERRORS - THE MATHEMATICS OF RANDOMNESS - The Remarkable Role of Evolution in the Making of Mathematics - Mathematics and the Real World

Mathematics and the Real World: The Remarkable Role of Evolution in the Making of Mathematics (2014)

CHAPTER V. THE MATHEMATICS OF RANDOMNESS

39. THE MATHEMATICS OF PREDICTIONS AND ERRORS

A crucial step forward in the establishment of the link between the concept of expectation and the practical use of statistics was made by Jacob Bernoulli (1654–1705), one of the most prominent members of a family that had a great influence on mathematics. He first analyzed repeated tosses of a coin, assuming that the chances of its falling on either side were equal. By sophisticated use of Newton's binomial formulae, Bernoulli analyzed the following: he examined whether in repeated tosses of a coin the chance that it would fall with a particular side up, say heads, would be close to 50 percent of the total number of flips. He found that the chance grew closer and closer to certainty the greater the number of flips of the coin. Clearly in a given series of tosses of the coin, the proportion of the number of heads to the total number of throws could have any value between zero and one. Nevertheless, as Bernoulli showed, as the number of throws increases, it is almost certain that the number of times heads appears will be close to 50 percent of the total number of throws. These trials are still today called Bernoulli trials, and its mathematical law is called the weak law of large numbers (the formulation of the strong law of large numbers would later require twentieth-century developments).

Bernoulli himself and others who contributed to this innovative research path extended the results to more general cases than repeated flips of a coin, even to the case of repeated sampling from a large population, and to random errors in non-exact measurements. As we have noted previously, in order to assess a physical quantity, the measurement of which entails measurement error, physicists used an average of many measurements. The mathematical result confirmed that the greater the number of repeats or measurements, and on the condition that the repeats are carried out totally independent of each other, the average of all the measurements will be close to the true value with a likelihood that increases and converges to certainty. At the same time, Bernoulli discussed the question of what creates the different probabilities and how confident can we be in their numerical values. He was apparently the first to distinguish between a priori probability, which we can calculate and derive from the conditions of the experiment, and a posteriori probability, which we see after performing a series of experiments. He turned the development of methods to calculate a posteriori probabilities into an objective, and it played an important role in future progress.

Bernoulli's weak law of large numbers is one of the limit rules that refer to statistical aspects of large samples or many repetitions of trials. Already then discrepancies were found between claims regarding large numbers and human intuition. Further on we will discuss in greater detail the discrepancy between intuition and the mathematics of randomness, but here we will give just two examples.

The first discrepancy is generally referred to as the gambler's fallacy. Many gamblers continue betting even if they are losing, believing that the laws of large numbers ensure that they will get their money back eventually. Their mistake is that the law says only that on average their wins will be close to expectation, but it does not relate to the amount of the win or loss. Even if the average of the wins and losses in a series of games is one dollar, the loss itself, in the case of many repeated bets, may be ten thousand or a million dollars. The difference between the average and the actual value gives an enormous advantage to the gambler who is wealthy and who can finance the loss until the probability turnaround arrives. That difference led to the bankruptcy of many gamblers who did not have deep enough pockets. Evolution did not give us the intuitive understanding of the difference between the average and the value itself when dealing with large numbers; the reason, apparently, is that in the course of evolution humans did not encounter examples of so many repetitions of events.

The second discrepancy between intuition and probability concepts is known as the St. Petersburg paradox, named after the city in which Daniel Bernoulli, nephew of Jacob Bernoulli, presented the problem to the Imperial Academy of the Arts. Remember that Hyugens referred to the expectation of a lottery as a fair measure of the cost of participating in it. That approach proved itself as an accounting basis for calculating loan repayments or the price of participating in a lottery. Now consider a lottery in which a coin is tossed many times, say a million. In this, if the first time the coin falls tails up is on the nth throw, that is, until then it fell n – 1 times with heads up, the participant receives 2n dollars. A simple calculation shows that expected winnings are one million dollars! Would you agree to pay one hundred thousand dollars, or even ten thousand dollars, to participate in this lottery? I do not know anyone that would agree to do so, and this difference between the theory and the practice is the paradox. Daniel Bernoulli had a social explanation for this, and we will discuss it in the next chapter. A different explanation, which we will also expand on in due course, is the gulf between mathematics and intuition. The latter tells us that the coin will not fall on the same side a large number of times and ignores the chance, albeit a small one but with high winnings, that such an event will occur.

The search for methods of calculating probabilities in situations with incomplete information at the outset, a subject raised by Jacob Bernoulli, led to the development of a technique based on a mathematical theorem known as the central limit theorem. The first move in that direction was made by the French mathematician Abraham de Moivre (1667–1754). He spent many years in England, much of the time with Newton, after being exiled there as a result of the persecution of the Huguenots in France. Whereas Jacob Bernoulli examined the extent of the deviations of the average from expectation and showed that most of the deviations are concentrated around zero, de Moivre decided to study the distribution of those deviations, in other words, how they divide between relatively large deviations and medium and small ones. He focused on Bernoulli's coin tossing experiments in which, let us say, heads gives a prize of one, and tails yields nothing. He found, using a mathematical calculation, that if the size of the deviation from the average is divided by the square root of the number of throws (i.e., instead of dividing the total winnings by n after n flips of the coin to find the deviation from expectation, he divided by √n), the distribution becomes closer and closer to a bell shape. If the coin is not equally balanced, with the chance of heads say a, the shape of the bell will depend on the value of a, but if the result is divided by √a(1 – a), which in due course came to be known as the standard deviation, the distribution obtained is bell shaped independent of the value of a. De Moivre actually calculated the shape of the bell obtained, shown in the diagram together with the formula (which is unimportant for our current purposes).

images

It is not clear whether de Moivre realized the implications of his discovery for the theory of statistics and its practice, but several famous mathematicians generalized de Moivre's limit law and found that it applies far more widely. The research reached its pinnacle in the work by Pierre-Simon Laplace (1749–1827), who as well as proving the wider applicability of the central limit theorem also laid the foundations for its uses in statistical analyses. Laplace was born in Normandy, France, and his family intended that he become a priest, but his mathematical inclinations prevailed, and he was accepted to carry out research under D’Alembert. He completed a research project in mechanics, a study that earned him a position in the military academy as a mathematics teacher and an artillery officer. There he befriended Napoleon Bonaparte, a friendship that certainly did him no harm in the politically stormy atmosphere in France at that time. He survived the French Revolution by maintaining a low profile and even became head of the French Academy. His book on the analytical theory of probability, published in 1812, he dedicated to Napoleon.

At about the same time, in a book published in 1810, Gauss himself presented the same limit law. Gauss, who clearly was familiar with de Moivre's work, focused his attention on a different aspect than did Laplace. Gauss was very interested in the results of measurements and in the question of how to find the closest value to the correct one in measurements that include random measurement errors. With that in mind, Gauss developed a system that still today is called the least-squares method, and, based on the assumption that the errors in the calculations are random and independent, showed that the average was the value that predicted the correct value with the greatest degree of accuracy. He extended the system to more complex calculations and, in that framework, even proved the central limit theorem. The bell-shaped distribution is today called the normal distribution and also the Gaussian distribution, in recognition of his contribution.

Laplace and Gauss, as well as many of their research colleagues, realized that the statistical methods could be used to answer questions beyond the field of gambling and flipping coins. If a particular outcome is the result of many occurrences that include randomness and those random occurrences are independent, the distribution of the outcomes will be similar to the normal distribution around the average value of the outcome. Laplace, who was also interested in and contributed much to astronomy, used this technique to analyze deviations of the planes of orbit of the planets from one, middle, plane. The planes of orbit of the planets around the Sun almost coincide, and their deviations from one middle plane are very small. Are those deviations random, or do they have another cause? By using the statistical technique he had developed, Laplace showed that the deviations are a close approximation to the expected distribution based on the central limit law, and therefore it was highly probable that they were random deviations from one orbit plane. The expression “highly probable” is an indication that this is a statistical result and not a mathematical certainty. At the same time Laplace showed that the planes of orbit of the various comets do not comply with the expectation based on the central limit law, and he therefore concluded that those deviations were not caused by random deviations from one plane. Gauss also used the least-squares method for calculations in astronomy. At that time the asteroid Ceres, which moved on an orbit that disappeared behind the Sun, was identified, and the question was when would it reappear on the other side of the Sun. Little data had been calculated regarding its path, and what there was included many measurement errors. Gauss applied his method and predicted with surprising accuracy Ceres's continued route, a prediction that justifiably brought him worldwide renown.

The work of Gauss and Laplace brought the mathematics of randomness and its uses in statistics to a center-stage position in science. Since then, the central limit law has been found to be correct, with minor changes, in more general cases than those analyzed by Laplace and Gauss. Especially worthy of mention are the Russian mathematician Pafnuty Chebyshev and his students Andrei Markov and Aleksandr Lyapunov. They were active in the second half of the nineteenth century and firmly established the central limit theorem without the assumption that the random events that lead to deviations from the average have equal distributions, that is, they have the same characteristics of randomness. Such a general rule was important to justify its uses for situations that appear in nature. The lack of interdependence in nature between the random events can be justified, but it is more difficult to justify the assumption that the random characteristics are equal for all events. The work of the Russian mathematicians reduced the gap between the mathematical theorem and its possible applications, and thus the central limit theorem, together with other limit theorems, became the norm for the various uses of statistics.

The main use of limit theorems is to estimate statistical values such as the average, the dispersion, and so on, of data that have random inaccuracy. It is generally difficult to assess whether the errors are random or not. Even if the errors are random, however, absorbing and understanding the technique required for the use of the mathematics that developed involves difficulties that can themselves cause errors. Again, the difficulties derive from the way our intuition relates to data. We will mention two such difficulties.

We are exposed to huge numbers of statistical surveys in our day-to-day lives. When the results of a survey are published, it is usually done in the following form: The survey found that (say) 47 percent of the voters intend to vote for a particular candidate, with a survey error of plus or minus 2 percent. Yet only part of the conditions and reservations about the results are presented in the survey report. In effect, the correct conclusion from the survey would be that there is a 95 percent chance that the proportion of voters who intend to support that particular candidate is between 47 percent plus 2 percent and 47 percent minus 2 percent. The 95 percent bound is quite normal in practice in statistics. The survey can be devised such that the chance that the assessment derived from the survey is correct is 99 percent (it will then cost more to carry out the survey), or any other number less than a hundred. The condition of 95 or 99 percent is not published. Why not? The fact that the declared limits of the results, that is, that the resulting interval of 45 to 49 percent, applies with only a 95 percent probability could have great importance. The reason would seem to be the difficulty in absorbing quantifications.

There is another aspect of statistical samples that is difficult to understand. Let us say that we are told that a survey of five hundred people selected at random in Israel is sufficient to ensure a result with a plus or minus 2 percent survey error (with 95 percent accuracy). The population of Israel is about eight million. What size sample would be required to achieve that level of confidence in the United States with 320 million inhabitants? Would it need to be forty times the sample size in Israel? Most people asked that question would answer intuitively that a much larger sample is needed in the United States than in Israel. The right answer is that the same size sample is required in both cases. The size of the sampled population affects only the difficulty of selecting the sample randomly. Once we have sampled correctly (and most survey failures derive from the inability to sample correctly), the size of the survey error is determined only by the size of the sample. This is another example of the discrepancy between human intuition and mathematical results. Evolution indeed did not prepare us for large samples.