IntroductionFor any two paired data sets, there are two ways of describing how they interact.The first is to measure how strongly the variables are associated. For that we use correlation, which we have already studied: the size of the correlation coefficient indicates the strength of the association.The second is to describe the relationship between the two variables with an equation that tells us how much one variable changes, on average, when the other increases or decreases. That is what regression is about.Functional and statistical relationshipsAs we know, the perimeter and the side of a square are related. The relationship between them is perfectly defined and can be expressed by a mathematical sentence:\[ 2p = 4l \]Where 2p is the perimeter and l is the side.Given any value of l, we can determine the value of 2p exactly.Now consider the relationship between the weight and the height of a group of people. This relationship is not of the same kind as the previous one: it is much less precise. People of different heights may have the same weight, and people of the same height may have different weights. Even so, on average, the taller the person, the heavier.Relationships like perimeter–side are called functional relationships, and those like weight–height are called statistical relationships. A statistical relationship can be summarized by a function that describes the average behaviour; when that function is a straight line and there is a single explanatory variable, we have simple linear regression.Objectives of simple linear regressionPredict values of a dependent variable (Y) from an independent variable (X).Know how much changes in X affect, on average, the value of Y.Simple linear regressionSimple linear regression is used to examine the relationship between one dependent variable and one independent variable. Once the analysis is done, the regression equation can be used to predict the dependent variable when the independent one is known. Regression goes beyond correlation by adding this predictive ability.People use the idea of regression intuitively every day: anyone who goes to the supermarket knows that the more items in the cart, the bigger the bill tends to be; anyone who drives knows that the longer the trip, the more fuel it will take. How easy it is to wake up in the morning often depends on how late you went to bed the night before. Quantitative regression adds precision to these intuitions by building a mathematical formula that can be used to make predictions.For example, a medical researcher may want to use body weight (independent variable) to predict the most appropriate dose of a new drug (dependent variable). The goal of regression is to find the formula that best describes the relationship between the two variables. That formula can then be used to predict the dependent variable when only the independent one is known: a doctor could prescribe the right dose based on a person's weight.The regression line (also called the least-squares line) shows the expected value of the dependent variable for each value of the independent variable. Technically, it is the line that minimizes the sum of the squared residuals (the vertical distances between the points and the line). It is the line that best fits the data in a scatter diagram.In the regression equation, the slope of the line (a) is the change in y divided by the change in x: how much y rises (or falls) when x increases by one unit. The intercept (b) is the point where the line crosses the y-axis, that is, the predicted value of y when x = 0. The slope is also called the angular coefficient and the intercept the linear coefficient (or constant). Since a regression model is almost never a perfect predictor, the equation also has an error term.In the regression equation, y is always the dependent variable and x is always the independent one. Here are three equivalent ways of writing a simple linear regression model:\[ y = (\text{slope} · x) + \text{intercept} + \text{error} \]\[ y = (\text{angular coefficient} · x) + \text{linear coefficient} + \text{error} \]\[ y = a · x + b + e \]Simple linear regression chooses a and b so as to make the sum of the squared errors, \( \sum{e^2} \), as small as possible. One consequence of this method is that the mean of the errors of the fitted line is zero.In the regression line y = 3 · x + 10, what does the number 3 mean?Answer: How much y increases, on average, when x increases by one unit — 3 is the slope (angular coefficient). The value of y when x = 0 is the intercept, 10.Calculating the coefficients a and bWe first calculate the slope a. Like correlation, it is obtained from the covariance:\[ a = \frac{\sigma_{x,y}}{\sigma_{x}^2} \]The slope is the covariance divided by the variance of the independent variable. Note that, unlike the correlation coefficient, which divides the covariance by the standard deviations of both variables, here only the independent variable appears in the denominator. That is why a still carries the units of the data (units of y per unit of x), while r is dimensionless.Once we know a, we calculate b by taking the mean of both sides of the equation:\[ y = a · x + b + e \]\[ E[y] = E[a · x + b + e] \]\[ E[y] = E[a · x] + E[b] + E[e] \]\[ \bar{y} = a · \bar{x} + b + 0 \]\[ b = \bar{y} - a · \bar{x} \]In other words, the regression line always passes through the point of the means \( (\bar{x}, \bar{y}) \).Some textbooks give the direct formulas in terms of sums:\[ a = \frac{N · (\sum{X · Y}) - (\sum{X}) · (\sum{Y})}{N · (\sum{X^{2}}) - (\sum{X})^{2}} \]\[ b = \frac{(\sum{Y}) - a · (\sum{X})}{N} \]With the slope and the intercept, we build the line that best describes the relationship between the two variables:It is the best line in the least-squares sense: no other line produces a smaller sum of the squared vertical distances between the points and the line.In a study, the covariance between X and Y is 12, the variance of X is 4, the mean of X is 5 and the mean of Y is 20. What is the regression line?Answer: y = 3 · x + 5 — a = 12 / 4 = 3 and b = 20 − 3 · 5 = 5. The option y = 3 · x + 20 uses the mean of Y as the intercept, forgetting to subtract a · x̄; the option 0.33 inverts the division.Why divide the covariance by the variance of X?We want a to represent "how much Y changes, on average, when X increases by one unit".Covariance alone does not give us that, because it mixes the scales of the two variables: its unit is the unit of X times the unit of Y. If we measure X in centimetres instead of metres, the covariance becomes 100 times larger, even though the relationship is exactly the same.Solution: divide the covariance by the variance of X, whose unit is the unit of X squared. The result is in units of Y per unit of X — exactly what a slope should be. If X is measured in centimetres, the slope becomes 100 times smaller, as it should: one extra centimetre changes Y much less than one extra metre.Think of it this way:Covariance = "how much do X and Y vary together?"Variance of X = "how much does X vary on its own?"Dividing one by the other, we calculate:\[ a = \frac{\text{how much } X \text{ and } Y \text{ vary together}}{\text{how much } X \text{ varies on its own}} \]This answers the question: "for each unit X varies, how much does Y vary, on average?" The standardized, unit-free measure of the relationship between X and Y is the correlation r; the two are linked by \( a = r · \frac{\sigma_y}{\sigma_x} \).Correlation vs. linear regressionThere are real differences between correlation and linear regression, but in simple linear regression they complement each other. The square of the correlation coefficient (r²) equals the regression's coefficient of determination, and in many studies this is the most important result.The main difference is one of role: correlation treats both variables the same way (rXY = rYX) and only measures the association, while regression picks one variable to explain the other — the regression of Y on X is different from the regression of X on Y. Choosing which variable is the dependent one is the researcher's decision; regression alone does not prove that X causes Y.Many people run the regression after analysing the correlation, once they have seen, even in the scatter diagram, that there is a linear relationship. It is worth calculating both, because they answer different questions: r² tells us what fraction of the variation in Y is explained by the line (with r = 0.29, for example, r² = 0.084, so only 8.4% of the variation is explained); regression tells us how much Y changes when X increases by one unit — a relationship that may be unreliable when r² is low, since much of the variation is left unexplained.The correlation between hours of study and grade is r = 0.6. What fraction of the variation in grades does the regression line explain?Answer: 36% — the explained fraction is the coefficient of determination, r² = 0.6² = 0.36. The 60% option confuses r with r²; the 40% option is 1 − r, which has no meaning here.Will the data always form a straight line?No. A great way to check whether the line is a good fit for the data is the coefficient of determination, r². In simple linear regression it is the square of the correlation, so checking the fit of the regression is very similar to checking the correlation between the data. It is also always worth looking at the scatter diagram: a low r² can hide a strong but curved relationship.Although in simple linear regression r² equals the square of the correlation coefficient, the coefficient of determination is defined differently: it is the fraction of the total variation in y explained by the line, that is, one minus the variation left around the line divided by the total variation:\[ r^2 = \frac{\text{variation explained by the line}}{\text{total variation}} = 1 - \frac{\sum{(y - \hat{y})^2}}{\sum{(y - \bar{y})^2}} \]where \( \hat{y} \) is the value predicted by the line. An r² of 1 means all the points lie on the line; an r² of 0 means the line explains nothing beyond the mean.This is also why linear regression is known as least-squares regression: it makes the sum of the squared errors, \( \sum{(y - \hat{y})^2} \), as small as possible, which makes r² as large as possible among all lines.Exercises1) What is the difference between correlation and linear regression?2) Draw the scatter plot and the line that best fits the data below:Month (X)Demand (Y)124002210032550427005275062800730003) In the previous exercise, the covariance of the two data sets is 485.71 and the variance of the independent variable (month) is 4 (both with divisor n). What are the slope and the intercept of the regression line?4) A shop recorded how many swimsuits it sold on 4 days of the year. Calculate the equation of the line that best fits the following points:Temperature (X)Swimsuits sold (Y)20102525303035405) In the previous exercise, how many swimsuits should the shop expect to sell, on average, on a 40-degree day?6) It is said that the more you study, the higher your grade. A student who doubted this recorded the study time and the grade of each classmate. The mean study time was 4 hours and the mean grade was 7. The covariance of the two data sets was 8 and the variance of the study time was 5. Using linear regression, what is the line that best explains the relationship between study time and grade?7) In the previous exercise, Zezinho got a grade of 5.4. According to the line, how long did he study? If he needed a grade of 8.6 to pass, how long should he have studied, according to the line?Answer key1) Correlation measures the strength and direction of the linear association and treats both variables equally; regression picks a dependent variable and gives the equation to predict it from the independent one.2) and 3) a = 485.71 / 4 = 121.43 and b = 2614.29 − 121.43 · 4 = 2128.57, that is, Y = 2128.57 + 121.43 · X (r² ≈ 0.79).4) Y = 1.9 · X − 265) 1.9 · 40 − 26 = 50 swimsuits6) a = 8 / 5 = 1.6 and b = 7 − 1.6 · 4 = 0.6, that is, Y = 1.6 · X + 0.67) 5.4 = 1.6 · X + 0.6 gives X = 3 hours; 8.6 = 1.6 · X + 0.6 gives X = 5 hours. (Using the line of Y on X "in reverse" is an approximation: to predict hours from the grade, the correct approach would be to fit the regression of X on Y.)Bibliographic referenceshttp://www.biostathandbook.com/linearregression.htmlhttps://www.statpac.com/statistics-calculator/correlation-regression.htmhttp://www.kean.edu/~fosborne/bstat/09rc.htmlhttps://edisciplinas.usp.br/pluginfile.php/1479289/mod_resource/content/0/regr_lin.pdf
ردود الفعل
0
0 تعليقات
كن أول من يعلق
لا شيء هنا بعد.