Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 1
1. What is the nature of heteroscedasticity?
2. What are its consequences?
3. How does one detect it?
4. What are the remedial measures?
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 2
The Nature of Heteroscedasticity
• One of the important assumptions of the classical linear regression model is that the variance of each disturbance term ui, conditional on the chosen values of the explanatory variables, is some constant number equal to s2.
• This is the assumption of homoscedasticity, or equal (homo) spread (scedasticity), that is, equal variance.
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 3
The Nature of Heteroscedasticity
• Symbolically,
• When the variance is nonconstant then we have heteroscedasticity:
niuE i
, … ,2 ,1 )( 22 s (11.1.1)
22 )( ii
uE s (11.1.2)
2
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 4
FIGURE 11.1: Homoscedastic disturbances
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 5
FIGURE 11.2: Heteroscedastic disturbances
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 6
The Nature of Heteroscedasticity
• There are several reasons why the variances of ui may be variable:
1. Following the error–learning models, as people learn, their errors of behavior become smaller over time.
In this case, is expected to decrease (Fig. 11.3).2 i
s
3
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 7
FIGURE 11.3: Illustration of heteroscedasticity
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 8
The Nature of Heteroscedasticity
2. As incomes grow, people have more discretionary income and hence more scope for choice about the disposition of their income. Hence, is likely to increase with income.
2
i s
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 9
The Nature of Heteroscedasticity
3. As data collecting techniques improve, is likely to decrease.
2
i s
Thus, banks that have sophisticated data processing equipment are likely to commit fewer errors in the monthly or quarterly statements of their customers than banks without such facilities.
4
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 10
The Nature of Heteroscedasticity
4. Heteroscedasticity can also arise as a result of the presence of outliers.
An outlier is an observation that is much different (either very small or very large) in relation to the observations in the sample (Fig. 11.4).
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 11
FIGURE 11.4: The relationship between stock prices and consumer prices
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 12
The Nature of Heteroscedasticity
5. Another source of heteroscedasticity arises from violating Assumption 9 of the CLRM, namely, that the regression model is correctly specified.
Very often, what looks like heteroscedasticity may be due to the fact that some important variables are omitted from the model.
5
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 13
The Nature of Heteroscedasticity
Recall our study of advertising impressions retained (Y) in relation to advertising expenditure (X).
If you regress Y on X only and observe the residuals from this regression, you will see one pattern.
But if you regress Y on X and X2, you will see another pattern (Fig. 11.5).
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 14
FIGURE 11.5: Residuals from the regression of (a) impressions on advertising expenditure and (b) impressions on Adexp and Adexp2
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 15
The Nature of Heteroscedasticity
6. Another source of heteroscedasticity is skewness in the distribution of one or more regressors included in the model.
Examples are economic variables such as income, wealth, and education.
It is well known that the distribution of income and wealth in most societies is uneven.
6
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 16
The Nature of Heteroscedasticity
7. Other sources of heteroscedasticity: As David Hendry notes, heteroscedasticity can also arise because of
(1) incorrect data transformation (e.g., ratio or first difference transformations).
(2) incorrect functional form (e.g., linear vs. log- linear models).
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 17
The Nature of Heteroscedasticity
• As an illustration of heteroscedasticity, consider Table 11.1.
• This table gives data on compensation per employee in 10 nondurable goods manufacturing industries, classified by the employment size of the firm or establishment for the year 1958.
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 18
The Nature of Heteroscedasticity
TABLE 11.1: COMPENSATION PER EMPLOYEE ($) IN NONDURABLE MFG. INDUSTRIES ACCORDING TO EMPLOYMENT SIZE OF ESTABLISHMENT, 1958
7
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 19
FIGURE 11.6: Standard deviation of compensation and mean compensation
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 20
OLS Estimation in the Presence of Heteroscedasticity
• What happens to OLS estimators and their variances if we introduce heteroscedasticity by letting
but retain all other assumptions of the classical model?
22 )( ii
uE s
• To answer this question, we shall revert to the two – variable model.
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 21
OLS Estimation in the Presence of Heteroscedasticity
• Applying the usual formula, the OLS estimator of b2 is
iii uXY
21 bb
22
22
)(
ˆ
ii
iiii
i
ii
XXn
YXYXn
x
yx b
(11.2.1)
8
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 22
OLS Estimation in the Presence of Heteroscedasticity
• But its variance is now given by the following expression:
22
22
2 )( )ˆvar(
i
ii
x
x s b (11.2.2)
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 23
OLS Estimation in the Presence of Heteroscedasticity
• The usual variance formula obtained under the assumption of homoscedasticity is:
(11.2.3)
2
2
2 )ˆvar(
i x
s b
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 24
OLS Estimation in the Presence of Heteroscedasticity
• Heteroscedasticity Homoscedasticity
22
22
2 )( )ˆvar(
i
ii
x
x s b
(11.2.2)
2
2
2 )ˆvar(
i x
s b
(11.2.3)
• Of course, if for each i, the two formulas will be identical.
22 ss i
9
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 25
The Method of Generalized Least Squares (GLS)
• Why is the usual OLS estimator of b2 not best, although it is still unbiased?
• Intuitively, we can see the reason from Table 11.1.
• There is considerable variability in earnings between employment classes.
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 26
The Method of Generalized Least Squares (GLS)
• If we were to regress per–employee compensation on the size of employment, we would like to use the knowledge that there is considerable interclass variability in earnings.
• We would like to devise the estimating scheme so that observations coming from populations with greater variability are given less weight than those coming from populations with smaller variability.
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 27
The Method of Generalized Least Squares (GLS)
• OLS does not follow this strategy: It assigns equal weight to each observation.
• But a method of estimation, known as generalized least squares (GLS), takes such information into account explicitly and is therefore capable of producing estimators that are BLUE.
10
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 28
The Method of Generalized Least Squares (GLS)
• To see how this is done, let us continue with the familiar two–variable model:
iii uXY
21 bb (11.3.1)
which for ease of algebraic manipulation we write as
iiii uXXY
201 bb (11.3.2)
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 29
The Method of Generalized Least Squares (GLS)
iiii uXXY
201 bb (11.3.2)
where X0i = 1 for each i. The reader can see that these two formulations are identical.
• Now assume that the heteroscedastic variances are known.
2
i s
• Divide (11.3.2) through by si to obtain ….
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 30
The Method of Generalized Least Squares (GLS)
(11.3.3)
which for ease of exposition we write as
i
i
i
i
i
i
i
i uXXY
ss b
s b
s 2 0
1
iii
uXXY 2011
bb (11.3.4)
• What is purpose of transforming the original model?
11
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 31
The Method of Generalized Least Squares (GLS)
(11.3.5)
1
)( cesin )( 1
known is since )( 1
)()var(
222
2
22
2
2
2
iii
i
ii
i
i
i
ii
uE
uE
u EuEu
ss s
s s
s
which is a constant.
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 32
The Method of Generalized Least Squares (GLS)
• GLS is OLS on the transformed variables that satisfy the standard least –squares assumptions.
• The actual mechanics of estimating and are as follows. First, we write down the SRF of (11.3.3)
i
i
i
i
i
i
i
i uXXY
ss b
s b
s ˆˆˆ
2
0
1
1 b̂
2 b̂
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 33
The Method of Generalized Least Squares (GLS)
i
i
i
i
i
i
i
i uXXY
ss b
s b
s ˆˆˆ
2
0
1
or
iii
uXXY ˆˆˆ 2011
bb (11.3.6)
12
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 34
The Method of Generalized Least Squares (GLS)
• Now, to obtain the GLS estimators, we minimize
that is,
22011 2 )ˆˆ(ˆ
iii XXYu bb
2
2
0
1
2
ˆˆˆ
i
i
i
i
i
i
i
i XXYu
s b
s b
ss (11.3.7)
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 35
The Method of Generalized Least Squares (GLS)
• The GLS estimator of is
and its variance is given by
(11.3.9)
2 b
222 )())((
))(())((ˆ iiiii
iiiiiiii
XwXww
YwXwYXww b (11.3.8)
222 )())((
)( )ˆvar(
iiiii
i
XwXww
w b
where .2/1 ii
w s
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 36
The Method of Generalized Least Squares (GLS)
• In OLS we minimize
Difference Between OLS and GLS
221 2 )ˆˆ(ˆ
iii XYu bb (11.3.10)
• In GLS we minimize the expression (11.3.7):
2
2
0
1
2
ˆˆˆ
i
i
i
i
i
i
i
i XXYu
s b
s b
ss (11.3.7)
13
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 37
The Method of Generalized Least Squares (GLS)
Difference Between OLS and GLS
which can also be written as
2
2
0
1
2
ˆˆˆ
i
i
i
i
i
i
i
i XXYu
s b
s b
ss (11.3.7)
2201 2 )ˆˆ(ˆ
iiiiii XXYwuw bb (11.3.11)
where .2/1 ii
w s
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 38
The Method of Generalized Least Squares (GLS)
• Thus, in GLS we minimize a weighted sum of residual squares with acting as the weights.
Difference Between OLS and GLS
• But in OLS we minimize an unweighted or (what amounts to the same thing) equally weighted RSS.
2/1 ii
w s
• To see the difference between the two, consider the hypothetical scattergram in Fig. 11.7.
CHAPTER 11: Heteroscedasticity: What Happens If the Error Variance Is Nonconstant?
ANDREW PAIZIS-QC BASIC ECONOMETRICS 5th Ed. 39
