IntroductionIn the previous lessons, we studied discrete random variables, which fit a wide range of practical problems very well. For some of these distributions there are tables and charts that make it easier to calculate probabilities and to communicate their parameters. In this lesson, we start from three discrete models (uniform, Bernoulli and binomial) and arrive at the most important distribution in statistics: the normal distribution, and we tell how it came about.Probability models for discrete random variablesDiscrete uniform distributionThis is the simplest case of a random variable: each of the k possible values occurs with the same probability.\[ P(X = x_i) = p(x_i) = p = \frac{1}{k}\]For example, what is the probability of each number when rolling a die?Number on the dieProbability11/621/631/641/651/661/6Or, graphically:die probabilityBernoulli distributionThe Bernoulli distribution, named after the Swiss mathematician Jakob Bernoulli, describes an experiment with only two outcomes: the result either has a given characteristic or it does not. Examples:a coin is tossed: the result is heads or not (in which case it is tails);a die is rolled: face 5 comes up or not (in which case one of the other faces comes up);a part is picked at random from a batch of 500 parts: it is either defective or not;a person picked at random from 1,000 people is male or not;a person is picked at random from the residents of a city and asked whether or not they support a municipal project.In each experiment there is either a success (value 1) or a failure (value 0).We call p the probability of success. So:\[ p(1) = P(X = 1) = p \]\[ p(0) = P(X = 0) = 1 - p \]That is, if the probability of success is p, the probability of failure is 1 − p.The mean (expected value) of a Bernoulli variable is the probability of success itself:\[ E(X) = 1 \cdot p + 0 \cdot (1 - p) = p \]And the variance is the mean of the squared deviations from that mean:\[ Var(X) = (1 - p)^2 \cdot p + (0 - p)^2 \cdot (1 - p) = p(1 - p)\left[(1 - p) + p\right] = p(1 - p) \]The notation for a Bernoulli variable is:\[ X \sim Ber(p) \]Binomial distributionWhen we repeat the Bernoulli trial n times (or, equivalently, take a sample of size n from a Bernoulli distribution), the result is a sequence of successes and failures. If the repetitions are independent, that is, if the result of one trial does not affect another, the number of successes follows the binomial distribution. In short, it requires the following characteristics:The experiment is repeated, under the same conditions, a finite number of times (n).The repetitions are independent: the result of one does not affect the following ones.Each repetition has one of two possible outcomes: success or failure.Throughout the experiment, the probability p of success and the probability q = 1 − p of failure remain constant.Situations with these characteristics:a coin is tossed three times; what is the probability of getting two heads?a die is rolled five times; what is the probability of getting face 5 at most three times?ten parts are drawn at random, with replacement, from a batch of 500 parts; what is the probability that all of them are defective, given that 10% of the parts in the batch are defective?five people are chosen at random from 1,000; what is the probability that two of them are male?90% of the people in a city are known to support a municipal project; if 100 residents are chosen at random, what is the probability that at least 80 support it?In the last two cases, the draws are made without replacement, but from a very large set, so we can assume they are practically independent. If the set were small, the appropriate model would be a different one: the hypergeometric distribution.An interesting feature of these situations is that we only care about the total number of successes, not the order in which they occur. In a sample of size n, one specific sequence with k successes has:\[ \text{probability of the } k \text{ successes: } p^{k} \]\[ \text{probability of the } n - k \text{ failures: } (1 - p)^{n - k} \]\[ \text{probability of the sequence: } p^{k} \cdot (1 - p)^{n - k} \]It remains to count how many sequences with k successes we can form. By simple combination, there are:\[ \binom{n}{k} = \frac{n!}{k!(n-k)!} \]Consequently:\[ \text{probability of } k \text{ successes: } P(X = k) = \binom{n}{k} \cdot p^{k} \cdot (1 - p)^{n - k} \]With the notation:\[ X \sim B(n, p) \]ExamplesExample 1: a coin is tossed three times; what is the probability of getting two heads?Using the formula, with q = 1 − p, we get the table:No. of successesProbabilityp = 1/20q³1/813pq²3/823p²q3/83p³1/8The probability of getting two heads is therefore 3/8 = 37.5%.Coin probability with 3 samplesExample 2: ten parts are drawn at random from a batch. Build the probability table for the number of defective parts if 50% of the parts in the batch are defective.Using the formula, we get the table:No. of successesProbabilityp = 50%0q¹⁰0.10%110pq⁹0.98%245p²q⁸4.39%3120p³q⁷11.72%4210p⁴q⁶20.51%5252p⁵q⁵24.61%6210p⁶q⁴20.51%7120p⁷q³11.72%845p⁸q²4.39%910p⁹q0.98%10p¹⁰0.10%Coin probability with 10 samplesExample 3: 200 parts are drawn at random from a batch. Plot the probability of the number of defective parts if 50% of the parts in the batch are defective.Coin probability with 200 samplesNotice that the larger the number of repetitions, the more the distribution takes on a bell shape, symmetric around the mean (np = 100).A fair coin is tossed 4 times. What is the probability of getting exactly 2 heads?Answer: 37.5% — there are C(4,2) = 6 sequences with two heads, each with probability (1/2)^4 = 1/16; so 6/16 = 0.375. The 50% is the trap of thinking that 'half the time' is guaranteed: two heads is the most likely result, but it does not have a 50% chance.A Bernoulli variable has probability of success p = 0.3. What is its variance?Answer: 0.21 — the Bernoulli variance is p(1 − p) = 0.3 × 0.7 = 0.21. The 0.09 is p², which is only part of the calculation, and 0.3 is the mean, not the variance.Central limit theoremThis behavior is no coincidence: it is what led statistics to the following theorem.“When the sample size increases, the sampling distribution of its mean gets closer and closer to a normal distribution.”- Central Limit TheoremThe number of successes in a binomial is the sum of n independent Bernoulli variables; that is why, for large n, the binomial approaches the normal, as we saw in the charts. But what exactly is a normal distribution? Before answering, we need to understand how continuous variables work.Probability models for continuous variablesWhen we work with continuous variables, there are infinitely many possible values. For example, a person's height can be 1.6 m, or 1.61, or 1.612, or 1.6125, or 1.61256, and so on.For this reason, the probability that a continuous variable takes exactly one specific value is always zero: infinitely many possible values share a total probability of 1, and a single point has no "width" in which to accumulate probability.\[ P(X = x) = 0 \text{ for any specific value } x \]To work with continuous variables, we therefore consider intervals. So, if we want to study the probability that the minute hand of a clock is between minute 2 and minute 3:\[ P(2 \leq X < 3) = \frac{1}{60} \]By dividing the interval [0, 60) into small subintervals, we can build a histogram of the probabilities of the random variable X. The central idea is: the probability of an interval equals the area under the curve over that interval.Probability of each minute intervalThe curve that describes how probability is spread across the values is called the probability density function (p.d.f.) of the random variable X. The total area under it is always equal to 1.For example, consider the following p.d.f., from which we want to calculate the probability that x is between 0 and 0.5:\[ f(x) = 2x \text{, for } 0 \leq x \leq 1 \]Probability of each XThe area is that of the triangle formed between x = 0 and x = 0.5, whose height is f(0.5) = 1:\[ P(0 \leq X \leq 0.5) = \frac{0.5 \cdot 1}{2} = \frac{1}{4} = 25\% \]Note that the vertical axis of the chart is not a probability but a density: it goes up to 2, while probability only appears as area.With the density f(x) = 2x, for 0 ≤ x ≤ 1, what is P(0.5 ≤ X ≤ 1)?Answer: 75% — the total area under the density is 1 and the part from 0 to 0.5 is 25%, which leaves 1 − 0.25 = 0.75 (or, using the trapezoid, (1 + 2)/2 × 0.5 = 0.75). The 50% is the mistake of thinking that half the interval has half the probability: that only holds for a constant density.The Gaussian, or normal, modelThe normal distribution first appeared in 1733, in a work by Abraham de Moivre, as an approximation to binomial probabilities when the number of repetitions is large, exactly the phenomenon we saw in the charts above. Around 1809, Carl Friedrich Gauss used it as a model for the errors in astronomical observations, hence the name Gaussian distribution. Shortly afterwards, in 1810, Pierre-Simon Laplace proved a first version of the central limit theorem, explaining why this curve appears so often.More formally, the normal distribution is a continuous probability distribution determined by two parameters: the expected value (mean, μ) and the standard deviation (σ). So there are infinitely many normal distributions, one for each pair of parameters:Normal distributionThe normal distribution with mean zero and standard deviation one is called the standard normal distribution.As the central limit theorem states, anything that can be seen as the sum of a large number of small independent random parts tends toward a normal distribution. The theorem holds for independent and identically distributed variables with finite variance, whatever the original distribution. For example, the mean of many variables tends toward the normal:if the variables follow the Bernoulli distribution;if the variables follow the chi-square distribution with one degree of freedom;if the variables follow the exponential distribution.The exception is distributions without a finite variance, such as the Cauchy distribution: for them, the mean of many observations does not approach the normal. For data that are not normal, there are also transformations (such as the logarithm) that bring them closer to normality.For which of these distributions does the central limit theorem NOT guarantee that the mean of many independent observations approaches the normal?Answer: Cauchy — the theorem requires finite variance, and the Cauchy distribution has neither a defined mean nor a defined variance. The exponential and the chi-square are skewed, but they have finite variance, so their mean tends toward the normal.Where the Gaussian appearsBallistics: in the 19th century, to improve the accuracy of artillery, many cannon shots were fired, and it was observed that the deviations in direction and range were distributed much like the normal.Intelligence quotient: IQ scores are built to follow a normal, with mean 100 and standard deviation 15.Human anatomy: ear size, height (separating men and women), etc.Animal anatomy: the size of an egg, for example.Economics and finance: the daily returns of assets and commodities are often modeled by the normal as a first approximation, although in practice they have heavier tails (prices themselves are not normal).Signals and physical measurements: the thermal noise added to a signal transmitted through an electric cable and the measurement errors of instruments approximately follow the normal.The normal distribution is found everywhere, including at your local gymCriteria for checking normalityThe simplest criterion is to draw a histogram of the data and visually check whether it is bell-shaped and symmetric around the center. This criterion is subjective: it helps rule out distributions that are clearly not Gaussian, but not to confirm normality. More rigorous criteria include:the Q-Q (quantile-quantile) plot, in which normal data lie approximately on a straight line;the measures of skewness and kurtosis, which in the normal are 0 and 3 (excess kurtosis equal to 0);the proportion of data in the normal ranges: about 68% within 1 standard deviation of the mean, 95% within 2 and 99.7% within 3;statistical normality tests, such as Shapiro-Wilk, Anderson-Darling and Kolmogorov-Smirnov (in the Lilliefors version).In a large sample that follows a normal distribution, roughly what proportion of the data should lie more than 2 standard deviations from the mean, counting both sides?Answer: 5% — about 95% of the data lie within 2 standard deviations of the mean, which leaves 5%, split into 2.5% in each tail. The 2.5% is the trap of counting only one side; 0.3% corresponds to 3 standard deviations.
ردود الفعل
0
0 تعليقات
كن أول من يعلق
لا شيء هنا بعد.