Normal distribution and its history

00Comment iconComment iconComment iconComment icon

Lots of problems calculating probability? Your problems are over, the Central Limit Theorem has come to help you!

Writer image

Translated byEditorial

Writer image

Revised byEditorial

Edit Article

Introduction

Throughout the previous lessons, discrete random variables were studied, which adapt very well to a series of practical problems. For some of these distributions, there are tables and graphs that make it easier to calculate and convey information about their parameters.

Probabilistic models for discrete random variables

Discrete uniform distribution

This is the simplest case of a random variable, where each possible value occurs with the same probability.

\[ P(X = x_i) = p(x_i) = p = \frac{1}{k}\]

As an example, what is the probability of each number when rolling a die?

Number on the dieProbability
11/6
21/6
31/6
41/6
51/6
61/6

Or graphically:

die probability
die probability

Bernoulli distribution

The Bernoulli distribution, named in honor of the Swiss scientist Jakob Bernoulli, represents a choice. These are experiments in which the results either have or do not have a certain characteristic. Example:

- A coin is tossed: the result can be heads or not (therefore, tails occurs).

- A die is rolled: either face 5 occurs or not (therefore, the other possible faces occur)

- a piece is chosen at random from a batch containing 500 pieces: this piece is defective or not;

- a person chosen at random from among 1,000 is or is not male;

- a person is chosen at random among the residents of a city and it is checked whether they are in favor of a municipal project or not.

For each experiment there is an occurrence of success (value 1) or failure (value 0).

We assume p as the probability of success occurring. Thus:

\[ p(1) = P(X = 1) = p \]

\[ p(0) = P(X = 0) = 1 - p \]

It can be seen that if the probability of success is p, the probability of failure is 1 - p.

The variance for this case will be the difference between the whole minus the mean:

\[ Var(x) = (p - 0) · (1 - p) + p · (p - p) = p · (1 - p) \]

The notation for a Bernoulli trial is as follows:

\[ X \sim Ber(p) \]

Binomial distribution

When repeating the Bernoulli trial n times, or, alternatively, having obtained a sample of size n from a Bernoulli distribution, a particular sample will consist of a sequence of successes and failures; assuming that these repetitions are independent, that is, the result of one trial does not interfere with the result of another. This repetition can be modeled from the binomial distribution. In summary, it requires the following characteristics:

a. The experiment must be repeated, under the same conditions, a finite number of times (n).

b. The repeated trials must be independent, that is, the result of one must not affect the results of the successive ones.

c. In each trial, one of the two possible results must appear: success and failure.

d. During the experiment, the probability p of success and the probability q (q = 1- p) of failure will remain constant.

The following situations with these characteristics can be found:

- a coin is tossed three times; what is the probability of getting two heads?

- a die is rolled five times; what is the probability of getting face 5 at most three times?

- ten pieces are drawn at random, with replacement, from a batch containing 500 pieces; what is the probability that all are defective, knowing that 10% of the pieces in the batch are defective?

- five people are chosen at random from among 1,000; what is the probability that two are male?

- it is known that 90% of the people in a city are in favor of a municipal project. Choosing 100 people at random among the residents, what is the probability that at least 80 are in favor of the project?

Note that in the last two cases, the fact that we are drawing individuals from a very large set implies that we can assume the drawings are practically independent. If the set is small, these data should be handled by another distribution: the hypergeometric distribution

An interesting characteristic of the situations considered is that there is interest only in the total number of successes and not in the order in which they occur. The probability of a sequence with k successes occurring in a sample of size n is:

\[ \text{Probability of success: }p^{k} \]

\[ \text{Probability of failure: }(1 - p)^{n - k} \]

\[ \text{Probability of a sequence with k successes occurring: }p^{k} * (1 - p)^{n - k} \]

It remains to know how many sequences with the specified property we can form. It is easy to see by simple combination that there are:

\[ \binom{n}{k} = \frac{n!}{k!(n-k)!} \]

Consequently:

\[ \text{Probability of k successes occurring: }P(X = k) = \binom{n}{k} p^{k} (1 - p)^{n - k} \]

With the following notation:

\[ X \sim B(n, p) \]

Examples

Example 1: a coin is tossed three times; what is the probability of getting two heads?

Using the formula, we can obtain the following table:

No. of successesProbabilityp = 1/2
01/8
13pq³3/8
23p²q3/8
31/8
Coin probability with 3 samples
Coin probability with 3 samples

Example 2: ten pieces are drawn at random from a batch. Build the probability table of successes if 50% of the pieces in the batch are defective.

Using the formula, we can obtain the following table:

No. of successesProbabilityp = 50%
0q¹⁰0.09%
110pq⁹0.97%
245p²q⁸4.39%
3120p³q⁷11.71%
4210p⁴q⁶20.50%
5252p⁵q⁵24.60%
6210p⁶q⁴20.50%
7120p⁷q³11.71%
845p⁸q²4.39%
910p⁹q0.97%
10p¹⁰0.09%
Coin probability with 10 samples
Coin probability with 10 samples

Example3: 200 pieces are drawn at random from a batch. Build the probability graph of successes if 50% of the pieces in the batch are defective.

Coin probability with 200 samples
Coin probability with 200 samples

It can be seen that the more samples we add to the problem, the more the distribution seems to take shape!

This analysis led statistics to arrive at the following theorem.

Central limit theorem

“When the sample size increases, the sampling distribution of its mean gets closer and closer to a normal distribution.”

- Central Limit Theorem

But after all, what is a normal distribution?

Probabilistic models for continuous variables

When working with continuous variables, it is assumed that there is an infinity of possible events. As an example, the height of a human being can be: 1.6 or 1.61 or 1.612 or 1.6125 or 1.61256 etc...

Since the probability of an event is its occurrence over the total, the probability of a certain continuous event will always be zero, because:

\[ \text{Probability of a certain continuous event: } \frac{\text{occurrences of this event}}{\text{sum of all occurrences}} = \frac{\text{occurrences of this event}}{\text{infinity}} = 0 \]

To work with the probability of continuous variables, periods are usually considered. Thus, if we want to study the probability of a clock hand being at minute 2:

\[ P(2 \leq X < 3) = \frac{1}{60} \]

By dividing the interval [0, 60[ into small subintervals, we can build a histogram for the probabilities of the random variable X.

> The probability of the interval is equal to the area of that interval

Probability of each minute interval
Probability of each minute interval

The probability graph according to each interval is called the probability density function (p.d.f.) of the random variable X.

For example, one can consider the following p.d.f. and calculate from it the probability of x being between 0 and 0.5:

\[ F(x) = 2x \text{, for } 0 \leq x \leq 1 \]

Probability of each X
Probability of each X

The area would be the triangle formed by the x values between 0 and 0.5:

\[ p(0 \leq x \leq 0.5) = \frac{0.5 * 1}{2} = \frac{1}{4} = 25\% \]

The Gaussian or normal model

Its origins go back to Gauss in his work on errors in astronomical observations, around 1810, hence the name Gaussian distribution for this model.

In more formal terms, the normal distribution is an absolutely continuous probability distribution parameterized by its mathematical expectation (mean) and standard deviation. Because of this, there is an infinity of normal distributions:

Normal distribution
Normal distribution

The normal distribution with zero mean and unit standard deviation is called the centered and reduced normal distribution or the standard normal distribution.

As stated in the central limit theorem, everything that can be considered the sum of a large number of small independent random values approximates a normal distribution. The central limit theorem is valid for every probability distribution with independent and identically distributed variables and finite standard deviation. For example:

- If the variables follow the Bernoulli distribution

- If the variables follow the chi-square distribution with one degree of freedom

- If the variables follow the exponential distribution

Normalization processes can also be created so that variables that do not fit these characteristics come to fit them.

Uses of the Gaussian

- Ballistics.

In the 19th century, to improve the accuracy of artillery fire, many cannon shots were fired. It was observed that the direction and range were similar to normal distributions.

- Intelligence quotient

- Human anatomy (ear size, height, etc...)

- Animal anatomy (size of an egg)

- Economics (commodity price

- Physical signals and measurements (transmission of a signal through an electrical cable; its loss or noise has a normal distribution)

- Etc...

The normal distribution is found everywhere, including at your local gym
The normal distribution is found everywhere, including at your local gym

Criteria for checking normality

The simplest criterion consists of drawing a bar chart of the distribution and visually checking whether the chart is bell-shaped, with symmetry around the center. However, this subjective criterion allows part of the distributions to be eliminated when considered non-Gaussian. More advanced criteria consist of checking normality ranges, statistical normality tests, calculating parameter estimates, etc...