Skewness in distributionsOnce we know where the data sit (mean, median, mode) and how spread out they are (standard deviation), the next question is about the shape of the distribution: is it symmetric, or does it have a longer tail on one side?When a distribution is symmetric, the mean, the median and the mode coincide. When it is skewed to the left (negative skew), the long tail is on the side of the small values and pulls the mean down: the mean is smaller than the median, which is smaller than the mode. When it is skewed to the right (positive skew), the opposite happens: the mean is larger than the median, which is larger than the mode.So the difference between the mean and the mode already tells us the type of skewness:\[ \bar{x} - Mo = 0 \text{ → no skewness, symmetric distribution} \]\[ \bar{x} - Mo < 0 \text{ → negative or left skew} \]\[ \bar{x} - Mo > 0 \text{ → positive or right skew} \]Examples of the three types of skewnessLet us look at three frequency distributions with the same classes. In all of them, the mode used is the crude mode (the midpoint of the class with the highest frequency), the median was found by interpolation within the median class, and s is the sample standard deviation.ClassesFrequency2 ⊢ 666 ⊢ 101210 ⊢ 142414 ⊢ 181218 ⊢ 226We have:\[ \bar{x} = 12 \]\[ Md = 12 \]\[ Mo = 12 \]\[ s = 4.42 \]Since mean, median and mode are equal, the distribution is symmetric.ClassesFrequency2 ⊢ 666 ⊢ 101210 ⊢ 142414 ⊢ 183018 ⊢ 226We have:\[ \bar{x} = 12.92 \]\[ Md = 13.5 \]\[ Mo = 16 \]\[ s = 4.22 \]Since the mean is smaller than the mode, the distribution is negatively skewed.ClassesFrequency2 ⊢ 666 ⊢ 103010 ⊢ 142414 ⊢ 181218 ⊢ 226We have:\[ \bar{x} = 11.08 \]\[ Md = 10.5 \]\[ Mo = 8 \]\[ s = 4.22 \]Since the mean is larger than the mode, the distribution is positively skewed.A distribution has mean 40, median 43 and mode 46. What type of skewness does it show?Answer: Negative skew, because the mean is smaller than the mode — x̄ − Mo = 40 − 46 = −6 < 0: the long tail is on the left and pulls the mean down. The tempting option, positive skew, mistakes a large mode for a right tail; what matters is where the mean lies relative to the mode.Pearson's coefficient of skewnessThe difference between mean and mode (or between mean and median) is an absolute measure: it is expressed in the units of the data. That is why it has the same shortcoming as the standard deviation on its own — it does not let us compare different distributions to say which one is more skewed.To compare them, we divide the difference by the standard deviation and obtain Pearson's coefficient of skewness (in the version that uses the median):\[ As = \frac{3 · (\bar{x} - Md)}{s} \]From the previous examples:\[ As_1 = \frac{3 · (12-12)}{4.42} = 0 \]\[ As_2 = \frac{3 · (12.92-13.5)}{4.22} = -0.41 \]\[ As_3 = \frac{3 · (11.08 - 10.5)}{4.22} = 0.41 \]The sign gives the side of the skew and the size gives its intensity. A common rule of thumb: |As| below 0.15 means weak skewness (almost symmetric); between 0.15 and 1, moderate skewness; above 1, strong skewness. Examples 2 and 3 have moderate skewness of the same intensity and opposite directions — which makes sense, since one table is the mirror image of the other.A distribution has mean 60, median 57 and standard deviation 12. What is Pearson's coefficient of skewness?Answer: 0.75 — As = 3 · (60 − 57) / 12 = 9 / 12 = 0.75, a moderate positive skew. Forgetting the factor 3 gives 0.25.KurtosisTraditionally, kurtosis is presented as the degree of flatness (or peakedness) of a distribution compared with the normal distribution, and that is how it appears in most textbooks and in the figure below:Type of kurtosisTechnical nameCharacteristicsHighLeptokurticHeavy tails: besides a concentrated core, there are more extreme values than in a normal distribution. It usually has a higher peak.NormalMesokurticTails similar to those of the normal curve.LowPlatykurticLight tails: the data spread more evenly and there are almost no extreme values. It usually looks flat.Examples:Grades on a test in which almost every student scores close to 7, but a few score 0 or 10, far from everyone else → leptokurtic (concentrated core and tails with extremes).Heights of adults of the same sex, whose shape is very close to the normal → mesokurtic.Ages of the participants in an event where every age group appears with similar frequency → platykurtic (close to a uniform distribution).Kurtosis coefficient and excess kurtosisKurtosis is measured by the fourth-moment coefficient, proposed by Karl Pearson in 1905:\[ K = \frac{ \sum (x_i - \bar{x})^4 }{n · s^4} \]Here s is the standard deviation computed with divisor n. Because the deviations are raised to the fourth power, values far from the mean weigh much more than those close to it. That is why this number is tied to the tails of the distribution and not to the shape of the peak: high kurtosis means extreme deviations (outliers) that are more frequent or larger, not necessarily a sharper peak. Describing kurtosis as "flatness" is a simplification — useful for intuition, but imprecise.In practice we use excess kurtosis more often:\[ \text{Excess kurtosis} = K - 3 \]If \( K - 3 > 0 \): leptokurticIf \( K - 3 = 0 \): mesokurticIf \( K - 3 < 0 \): platykurticThe value 3 is the kurtosis of the normal distribution, which is why it is the benchmark for comparison. Watch out when reading software output: many programs (such as Excel and scipy's kurtosis function) already report excess kurtosis, so the normal shows up as 0, not 3.A program reports, for a data set, kurtosis K = 2.1. How is the distribution classified?Answer: Platykurtic, since K − 3 = −0.9 is negative — the comparison with the normal uses the excess K − 3. The tempting option, leptokurtic, looks at the sign of K, which is always positive; only the excess can be negative.Boxplot (box-and-whisker plot) to check skewness and kurtosisIntroduced by John Tukey in 1970 and popularized in his book Exploratory Data Analysis (1977), the boxplot is a visual summary of the distribution of a data set using five measures: minimum, first quartile (Q1), median, third quartile (Q3) and maximum. Its simplicity allows a quick analysis of spread, skewness and outliers.Boxplot and its points explainedElements of the boxplot:Box: goes from Q1 to Q3 and represents the interquartile range (IQR = Q3 − Q1), which contains the middle 50% of the data.Median line: splits the box in two and marks the centre of the distribution.Whiskers: run from the box to the most extreme observed value that is still inside the fences Q1 − 1.5 × IQR and Q3 + 1.5 × IQR. The fences are reference limits; the whisker ends at an actual data point.Outliers: points beyond the fences, drawn individually.The boxplot is great for judging skewness because it immediately shows where the median sits inside the box and how long each whisker is. In a symmetric distribution, the median is roughly in the middle of the box and the two whiskers have similar lengths. If the median is closer to Q1 and the upper whisker is much longer than the lower one, the distribution is skewed to the right (positive). If the median is close to Q3 and the lower whisker is longer, the skew is to the left (negative).Comparisons between distributions using boxplotThe boxplot also gives hints about kurtosis, which is tied to the weight of the tails. A narrow box (small IQR) together with many outliers on both sides points to a leptokurtic distribution: the core is concentrated but the tails are heavy. A box that is wide compared with the total range, with short whiskers and no outliers, suggests a platykurtic distribution: the data are spread evenly and no value is very far from the centre. It is an approximate reading — to quantify it, use the kurtosis coefficient — but it is very useful in exploratory analysis.Building the boxplot step by stepWe will use the following 40 numbers:1, 2, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 391. Put the data in orderThe data are already in ascending order.2. Find the median (Q2)Since we have 40 numbers (n = 40, even), the median is the average of the two central values:Central positions: 20th and 21stValues: 19 and 20Median (Q2) = (19 + 20) / 2 = 19.53. Find the first quartile (Q1)Q1 is the median of the first half of the data (the first 20 numbers):First half: 1, 2, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19Median of the first 20: average of the 10th and 11th values, which are 9 and 10Q1 = (9 + 10) / 2 = 9.54. Find the third quartile (Q3)Q3 is the median of the second half of the data (the last 20 numbers):Second half: 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39Median of the last 20: average of the 10th and 11th values of that half, which are 29 and 30Q3 = (29 + 30) / 2 = 29.5There are other methods for computing quartiles, and programs such as Excel, R or numpy may give slightly different values (for example, 9.75 and 29.25 here). All of them are accepted; what matters is using the same method from start to finish.5. Calculate the interquartile range (IQR)IQR = Q3 − Q1 = 29.5 − 9.5 = 206. Calculate the fences to identify outliersLower fence = Q1 − 1.5 × IQR = 9.5 − 1.5 × 20 = 9.5 − 30 = −20.5Upper fence = Q3 + 1.5 × IQR = 29.5 + 1.5 × 20 = 29.5 + 30 = 59.57. Identify the outliersWe check whether any value is below −20.5 or above 59.5:Smallest value in the data: 1Largest value in the data: 39There are no outliers in this data set, since every value lies within the interval [−20.5, 59.5].8. Find the ends of the whiskersLower whisker: smallest value inside the lower fence → 1Upper whisker: largest value inside the upper fence → 399. Draw the boxplotWe now have all the measures:MeasureValueMinimum (end of the lower whisker)1Q19.5Median (Q2)19.5Q329.5Maximum (end of the upper whisker)39On an axis from 0 to 40, draw the box from 9.5 to 29.5 with a line at 19.5, one whisker from 1 to 9.5 and another from 29.5 to 39. The median is exactly in the middle of the box (10 units on each side) and the whiskers have almost the same length (8.5 and 9.5): the distribution is practically symmetric. Since there are no outliers and the data spread almost uniformly, it is also platykurtic.In a boxplot, the median is very close to Q1 and the upper whisker is three times longer than the lower one. What can we conclude about the distribution?Answer: Positive skew, with a tail on the right — the values above the median spread over a much wider range than those below, so the long tail is on the side of the high values. Negative skew would be the mirror case: median close to Q3 and a long lower whisker.A data set has Q1 = 20 and Q3 = 32. Which of these values would be flagged as an outlier by the 1.5 × IQR rule?Answer: 53 — IQR = 32 − 20 = 12 and 1.5 × 12 = 18, so the fences are 20 − 18 = 2 and 32 + 18 = 50. Only 53 lies outside them. The value 5 looks far away, but it is still above the lower fence, 2.Exercises1) Consider the following results for three frequency distributions:DistributionMeanModeA5252B4550C4846Determine the type of skewness of each one.2) A frequency distribution has the following measures: mean = 48.1; median = 47.9; and standard deviation = 2.12. Calculate Pearson's coefficient of skewness.3) The following measures were found in a frequency distribution: mean = 33.18; mode = 27.50; median = 31.67; and standard deviation = 12.25.a) Classify the type of skewness.b) Calculate Pearson's coefficient of skewness.4) A distribution has kurtosis coefficient K = 4.5. Classify the distribution (platykurtic, mesokurtic or leptokurtic).5) Explain in your own words what a leptokurtic distribution with negative skew means.6) Build the boxplot for the following data set: [12, 14, 19, 22, 23, 25, 27, 29, 31, 34, 35, 37, 39, 42, 44, 45, 48, 51, 53, 56, 59, 61, 63, 66, 67, 68, 71, 74, 77, 78, 79, 82, 85, 86, 88, 91, 93, 95, 97]7) Build the boxplot for the following data set: [−15, 5, 29, 31, 32, 33, 35, 36, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 88, 120, 150]Answer key1) A: symmetric (mean equal to the mode); B: negatively skewed (mean smaller than the mode); C: positively skewed (mean larger than the mode).2) As = 3 · (48.1 − 47.9) / 2.12 ≈ 0.28 (moderate positive skew).3) a) Positively skewed (mean larger than the median, which is larger than the mode). b) As = 3 · (33.18 − 31.67) / 12.25 ≈ 0.37.4) K − 3 = 1.5, positive: leptokurtic.5) Personal answer. Example: most of the data are concentrated around high values, there is a longer tail on the side of the low values, and the tails are heavy, that is, values far below the centre appear fairly often.6) n = 39. Median = 56 (20th value); Q1 = 34 and Q3 = 78 (medians of the halves of 19 values, excluding the median); IQR = 44; fences −32 and 144; no outliers; whiskers from 12 to 97. The median is in the middle of the box: the distribution is practically symmetric.7) n = 39. Median = 49; Q1 = 39; Q3 = 60; IQR = 21; fences 7.5 and 91.5; outliers: −15, 5, 120 and 150; whiskers from 29 to 88. The box is narrow and there are outliers on both sides: heavy tails, a leptokurtic distribution.
ردود الفعل
0
0 تعليقات
كن أول من يعلق
لا شيء هنا بعد.