Frequency distribution: collecting data in the field

, محدث •0•0Comment iconComment iconComment iconComment icon

How to build a frequency distribution: classes, limits, width, Sturges' rule, relative and cumulative frequencies, charts and the smoothed curve.

Writer image

تمت الترجمة بواسطةEditorial

Writer image

تمت المراجعة بواسطةEditorial

تحرير المادة

Forms of frequency distribution

Variables are much easier to observe and study when we list the values in order, in a column, and write next to each value the number of times it appears.

Simple frequency

We call frequency the number of times a given value of the variable under study occurs. A frequency distribution is the table (or chart) that shows all these frequencies and how they are distributed.

As an example, let's look at the frequency distribution of the heights of the 33 students in a class, measured in whole centimeters:

Height (cm)Frequency
1511
1520
1532
1541
1554
1563
1571
1582
1590
1605
1614
1623
1632
1643
1651
1661

Frequency by classes

This table is still impractical, because it takes up a lot of space even when the number of observations (n) is small. Whenever possible, especially for a continuous variable, the solution is to group the values into intervals. For example:

Height (cm)Frequency
150 —| 1558
155 —| 16011
160 —| 16513
165 —| 1701

The symbol —| means that the interval does not include the left limit and does include the right one: class 150 —| 155 gathers heights from 151 to 155 cm (1 + 0 + 2 + 1 + 4 = 8 students). It is the same as the interval (150, 155]. The opposite symbol, |—, includes the left limit and excludes the right one, like [150, 155). What matters is making the convention clear, so that no value is counted twice.

The intervals created are called classes: frequency classes, or simply classes, are the intervals into which we divide the range of the variable. The right-hand column of the table above shows the frequency of each class.

In the height table, how many students are in class 160 —| 165?

Loading icon

Answer: 13 — the class excludes 160 and includes 165, so it adds up heights from 161 to 165: 4 + 3 + 2 + 3 + 1 = 13. Anyone who includes 160 (5 students) and excludes 165 gets 17, counting 160 in a class that is not its own.

Frequency terminology

Class limits

The endpoints of each class are called class limits: each class has a lower limit and an upper limit. In the example, the lower limits are 150, 155, 160 and 165; the upper limits are 155, 160, 165 and 170.

Class width

The class width (h), also called class interval or class amplitude, is the size of the interval that defines the class. It is the difference between the upper and lower limits. In the example, all classes have width 5:

\[ h = 155 - 150 = 160 - 155 = 165 - 160 = 170 - 165 = 5 \]

Total range of the distribution and sample range

The total range of the distribution (TR) is the difference between the upper limit of the last class and the lower limit of the first. In the example, TR = 170 − 150 = 20.

Do not confuse it with the sample range (SR), which is the difference between the largest and the smallest observed value. In the example, SR = 166 − 151 = 15. The total range is always greater than or equal to the sample range, because the classes must contain all the data.

Number of classes

The first decision when building a frequency distribution is the number of classes (k). Too few classes hide the shape of the distribution; too many leave many of them empty.

A widely used rule is Sturges' rule (proposed by Herbert Sturges in 1926), which gives the number of classes from the number of observations n, using the base-10 logarithm:

\[ k = 1 + 3.3 \cdot \log_{10}(n) \]

And the width of each class is approximately the sample range divided by the number of classes, rounded up:

\[ h \approx \frac{\text{largest value} - \text{smallest value}}{k} \]

In the height example, with n = 33: k = 1 + 3.3 × log(33) = 1 + 3.3 × 1.52 ≈ 6 classes, and h ≈ 15 / 6 = 2.5, that is, classes 3 cm wide. The table above used only 4 classes 5 cm wide, for simplicity: Sturges' rule is a suggestion, not an obligation.

A survey has 200 observations, with smallest value 12 and largest value 68. Using Sturges' rule, how many classes and what approximate class width should be used?

Loading icon

Answer: 9 classes of width 7 — k = 1 + 3.3 × log(200) = 1 + 3.3 × 2.30 ≈ 8.6, that is, 9 classes; h ≈ (68 − 12) / 9 = 6.2, rounded up: 7. The most common mistake is using the natural logarithm (ln 200 = 5.3), which would lead to about 18 classes.

Types of frequency

Simple or absolute frequency

The simple or absolute frequency (fi) is the number of data points that belong to each class.

Example:

Height (cm)Simple frequency
150 —| 1558
155 —| 16011
160 —| 16513
165 —| 1701

Relative frequency

The relative frequency (fri) is the ratio between the simple frequency of the class and the total frequency. Added together, the relative frequencies equal 1 (or 100%).

Example, with total 33:

Height (cm)Relative frequency
150 —| 1558/33 ≈ 0.242 (24.2%)
155 —| 16011/33 ≈ 0.333 (33.3%)
160 —| 16513/33 ≈ 0.394 (39.4%)
165 —| 1701/33 ≈ 0.030 (3.0%)

Cumulative frequency

The cumulative frequency (Fi) is the total of the frequencies of all values less than or equal to the upper limit of a given class, that is, the frequency of the class plus those of all previous classes.

Height (cm)Cumulative frequency
150 —| 1558
155 —| 16019
160 —| 16532
165 —| 17033

Relative cumulative frequency

The relative cumulative frequency (Fri) is the cumulative frequency of the class divided by the total of the distribution.

Example, with total 33:

Height (cm)Relative cumulative frequency
150 —| 1558/33 ≈ 0.242 (24.2%)
155 —| 16019/33 ≈ 0.576 (57.6%)
160 —| 16532/33 ≈ 0.970 (97.0%)
165 —| 17033/33 = 1 (100%)

Reading the table: 57.6% of the students are at most 160 cm tall.

A distribution has frequencies 4, 10, 16 and 10 in classes 1, 2, 3 and 4. What is the relative cumulative frequency of class 3?

Loading icon

Answer: 75% — the total is 4 + 10 + 16 + 10 = 40; the cumulative frequency of class 3 is 4 + 10 + 16 = 30, and 30/40 = 0.75. The answer 40% is the simple relative frequency of class 3 (16/40), without accumulating.

Graphical representation

Besides the histogram (and the bar chart, when there are no classes), a frequency distribution can be represented in the following ways.

Frequency polygon

The frequency polygon is a line chart in which the frequencies are plotted on perpendiculars to the horizontal axis, raised at the midpoints of the class intervals. To close the polygon, a zero-frequency class is added before the first class and another after the last one.

Example (with another set of heights, in 4 cm classes; in the figure, Estatura = height and Frequência = frequency):

Image content of the Website

Cumulative frequency polygon

The cumulative frequency polygon (or ogive) is drawn by plotting the cumulative frequencies on perpendiculars to the horizontal axis, raised at the upper limits of the class intervals; it starts at zero at the lower limit of the first class.

Example (with another data set, in 6 cm classes; in the figure, Frequência acumulada = cumulative frequency and Classes das alturas = height classes):

Image content of the Website

The shapes of frequency curves

Bell-shaped curves

They have a maximum value in the central region. They are the most common in nature. Examples: heights, students' grades, daily changes in stock prices.

They can be symmetric:

Image content of the Website

Or asymmetric (skewed):

Image content of the Website

A skewed curve with the longer tail to the right (like the one in the figure above) has positive skewness; with the longer tail to the left, negative skewness.

J-shaped curves

They correspond to extremely skewed distributions, with the point of maximum frequency at one of the ends. They are common in economic and social phenomena. Examples: the distribution of personal income (many people with low income and fewer and fewer people in the higher brackets) and the number of accidents per driver (most have none). When the maximum is on the right, the curve is a J; when it is on the left, a reverse J.

Image content of the Website

U-shaped curves

They correspond to distributions with maximum frequencies at both ends and a minimum in the middle. They appear in very specific situations, such as the mortality rate by age: it is high among newborns and the elderly and low among young adults.

Image content of the Website

In a country, most people earn up to twice the minimum wage, and each higher income bracket has fewer people than the one before. What shape does the income frequency curve have?

Loading icon

Answer: Reverse J, with positive skewness — the maximum frequency is at the left end (low incomes) and the long tail goes to the right (high incomes). Negative skewness would be the opposite: a long tail to the left.

Transformations of the frequency table

Smoothed curve: when the sample curve does not come out as expected

If you make a histogram of the heights of 100 people, it may have irregular peaks and valleys. Smoothing the curve after obtaining the frequencies of a sample is a common technique in statistics, especially with histograms and empirical distributions. The reasons are:

  1. Reducing noise and random fluctuations;
  2. Seeing the shape of the real distribution better;
  3. Making comparisons and interpretations easier;
  4. Estimating probabilities in a continuous way.

In Brazilian introductory statistics textbooks, the smoothed curve (curva polida) is obtained with a weighted average of each class and its two neighbors, in which the class itself has weight 2:

\[ fc_i = \frac{f_{i-1} + 2f_i + f_{i+1}}{4} \]

At the ends, zero frequency is used for the classes that do not exist. With the heights from the example (8, 11, 13 and 1), adding an empty class before (145 —| 150) and another after (170 —| 175):

ClassfiSmoothed frequency fci
145 —| 1500(0 + 0 + 8)/4 = 2.00
150 —| 1558(0 + 16 + 11)/4 = 6.75
155 —| 16011(8 + 22 + 13)/4 = 10.75
160 —| 16513(11 + 26 + 1)/4 = 9.50
165 —| 1701(13 + 2 + 0)/4 = 3.75
170 —| 1750(1 + 0 + 0)/4 = 0.25

Note that the total is still 33 and that the sharp drop from 13 to 1 has become a gradual descent. The smoothed curve is drawn by joining these values at the class midpoints, as in a frequency polygon.

Other smoothing techniques are kernel density estimation (KDE), moving averages and splines (non-parametric regression). The simple moving average (SMA) of order n is widely used for series over time:

\[ SMA_t = \frac{x_t + x_{t-1} + x_{t-2} + \dots + x_{t-n+1}}{n} \]

Where:

xt = value at time t;

n = number of periods considered.

Suppose we collected the following daily visit counts for a website over 10 days:

Day (t)Visits (xt)
1120
2150
3130
4170
5160
6140
7190
8180
9170
10200

With n = 3, the formula can only be applied from day 3 onward:

  • Day 3: \( SMA_3 = \frac{130 + 150 + 120}{3} = \frac{400}{3} \approx 133.3 \)
  • Day 4: \( SMA_4 = \frac{170 + 130 + 150}{3} = \frac{450}{3} = 150 \)
  • Day 5: \( SMA_5 = \frac{160 + 170 + 130}{3} = \frac{460}{3} \approx 153.3 \)

... and so on up to day 10:

Day (t)Visits (xt)SMA(3) (smoothed curve)
1120-
2150-
3130133.3
4170150.0
5160153.3
6140156.7
7190163.3
8180170.0
9170180.0
10200183.3

See in the chart (which continues the series up to day 22; Acessos = visits, Curva Polida = smoothed curve, Dia = day) how the smoothed curve does not have the sudden jumps of the original series.

Smoothed curve vs. actual sample curve
Smoothed curve vs. actual sample curve

The frequencies of four consecutive classes are 6, 10, 4 and 8. What is the smoothed frequency of the second class, using fc = (previous f + 2 × f of the class + next f) / 4?

Loading icon

Answer: 7.5 — fc = (6 + 2 × 10 + 4) / 4 = 30 / 4 = 7.5. The value 6.7 would be the simple average of the three classes, (6 + 10 + 4) / 3, which does not give weight 2 to the class itself.

Other transformations

Besides smoothing, there are several mathematical transformations applied to data to improve their distribution, linearity or interpretation. The main ones are:

TransformationFormulaWhen to useExample of application
Logarithmic\( \log(x) \) or \( \ln(x) \), only for x > 0Data with positive skewness (long right tail).Income, city sizes, stock prices.
Square root\( \sqrt{x} \)Count data (Poisson) or moderate skewness.Number of clicks, accidents per day.
Inverse\( \frac{1}{x} \)Inverse relationships (e.g. time versus speed).Time between events, failure rates.
Box-Cox\( \frac{x^\lambda - 1}{\lambda} \) (and \( \ln x \) when \( \lambda = 0 \)), only for x > 0Bringing data closer to a normal distribution; \( \lambda \) is estimated from the data themselves.Skewed data or data with non-constant variance.
Standardization (z-score)\( \frac{x - \mu}{\sigma} \)When a comparable scale is needed (mean 0, standard deviation 1).Machine learning, scale-sensitive algorithms.
Min-max normalization\( \frac{x - \min(X)}{\max(X) - \min(X)} \)Data in a fixed interval (e.g. from 0 to 1).Neural networks, images (pixels).

Exercises

1) The grades obtained by 50 students in a class were:

1 2 3 4 5 6 6 7 7 8

2 3 3 4 5 6 6 7 8 8

2 3 4 4 5 6 6 7 8 9

2 3 4 5 5 6 6 7 8 9

2 3 4 5 5 6 7 7 8 9

It was decided to divide the grades into 5 classes (0 |— 2, 2 |— 4, 4 |— 6, 6 |— 8 and 8 |—| 10). Answer:

a) Complete the frequency distribution of the classes.

b) What is the sample range?

c) What is the total range of the distribution?

d) What is the number of classes of the distribution?

e) What is the upper limit of the fourth class?

f) What is the upper limit of the class of order 2?

g) What is the width of the second class interval?

2) Given the grades of 50 students:

84 68 33 52 47 73 68 61 73 77

74 71 81 91 65 55 57 35 85 88

59 80 41 50 53 65 76 85 73 60

67 41 78 56 94 35 45 55 64 74

65 94 66 48 39 69 89 98 42 54

Build the frequency distribution, with 30 as the lower limit of the first class and 10 as the class width.

3) The results of rolling a die 50 times were:

1 1 1 1 1 1 1 2 2 1

2 2 2 2 2 3 3 3 3 3

4 4 4 4 4 4 4 4 4 4

5 5 5 3 3 3 5 5 5 5

5 5 6 6 6 6 6 6 6 6

Build a frequency distribution without class intervals.

4) Given the frequency distribution:

xFrequency 1Frequency 2
322
455
5129
6106
782
831

Determine, for each of the two distributions:

a) the sum of the frequencies;

b) the relative frequencies;

c) the cumulative frequencies;

d) the relative cumulative frequencies.

5) Build the smoothed curve for the frequency distribution:

iClassesFrequency
14 |— 82
28 |— 125
312 |— 169
416 |— 206
520 |— 242
624 |— 281

6) Name the type of curve for each of the following distributions:

a) number of married women aged 15 to 30, in a given population, classified by the number of times they have married;

b) grades of students in the final year of secondary school, in a given population;

c) accident mortality rate, by age group;

d) parking time of motor vehicles in a congested area;

e) number of able-bodied men, by age group, who are unemployed.

7) The numbers below are the liquidity ratios obtained from the balance sheets of 50 companies:

3.9 7.4 10.0 11.8 2.3 4.5 10.5 8.4 15.6 7.6

18.8 2.9 2.3 0.4 5.0 9.0 5.5 9.2 12.4 8.7

4.5 4.4 10.6 5.6 8.5 2.4 17.8 11.6 0.8 4.4

7.1 3.2 2.7 16.2 2.7 9.5 13.1 3.8 6.3 7.9

4.8 5.3 12.9 6.9 6.3 7.5 2.6 3.3 4.6 16.0

a) Build a distribution with class intervals of width 3, so that the lower limits are multiples of 3;

b) Draw the corresponding histogram.

8) The degree of cloud cover, recorded in tenths, occurs according to the distribution below:

Cloud coverFrequency
0 |— 0.5320
0.5 |— 1.5125
1.5 |— 2.575
2.5 |— 3.565
3.5 |— 4.545
4.5 |— 5.545
5.5 |— 6.555
6.5 |— 7.565
7.5 |— 8.590
8.5 |— 9.5145
9.5 |—| 10676

Draw the corresponding histogram. (Note: the first and last classes have width 0.5, half of the others.)

9) Considering the distribution below:

ClassesFrequency
1 |— 27
2 |— 33
3 |— 410
4 |— 511
5 |— 612
6 |— 737
7 |— 835
8 |— 945
9 |— 1039
10 |— 1130
11 |— 1225

Draw the corresponding histogram.

10) The table below shows a frequency distribution of the areas of 400 plots of land:

Area (m²)Number of plots
300 |— 40014
400 |— 50046
500 |— 60058
600 |— 70076
700 |— 80068
800 |— 90062
900 |— 100048
1000 |— 110022
1100 |— 12006

Using this table, determine:

a) the total range;

b) the upper limit of the fifth class;

c) the lower limit of the eighth class;

d) the midpoint of the seventh class;

e) the width of the second class interval;

f) the frequency of the fourth class;

g) the relative frequency of the sixth class;

h) the cumulative frequency of the fifth class;

i) the number of plots whose area is less than 700 m²;

j) the number of plots whose area is 800 m² or more;

l) the percentage of plots whose area is greater than or equal to 900 m²;

m) the percentage of plots whose area is at least 500 m² but less than 1000 m².

Answer key

ما رأيك في هذا المقال؟

0%
تقييم المستخدمين (0)

شارك هذا المقال!

المواضيع estatísticacoletadados