Serial Correlation in Multiple Linear Regression: Autocorrelation & Autoregressive - Prof., Study notes of Economic Theory

Download Serial Correlation in Multiple Linear Regression: Autocorrelation & Autoregressive - Prof. and more Study notes Economic Theory in PDF only on Docsity! ECON 5340 Class Notes Chapter 12. Serial Correlation 1 Introduction In this chapter, we focus on the problem of serial correlation (a.k.a. autocorrelation) within the multiple linear regression model. Throughout, we assume that all other classical assumptions are satis…ed. Assume the model is yt = x 0 t + t (1) where E(0) = 2 = 0 2666666666664 1 1


n2 n1 1 1


n3 n2 ... ... . . . ... ... n2 n3


1 1 n1 n2


1 1 3777777777775 ; (2) cov(t; ts) = s is called the autocovariance of the errors and s = s= 0 is called the autocorrelation of the errors. Serial correlation is a common occurrence in time series data. Consider an example of a macroeconomic consumption function Ct = 1 + 2t+ 3Yt + t where t = 1950; :::; 1985, Ct is consumption and Yt is income. A plot of the OLS residuals is attached. 2 Time Series Properties of a First-Order Autoregression Assume that the error terms follow a …rst-order autoregressive (AR(1)) process t = t1 + t (3) where t  iid(0; 2), jj < 1 and t = 1; :::; T . Rewrite (3) using repeated substitutions t = t1 + t = (t2 + t1) + t =  2t2 + t + t1 = ... = T tT + XT1 j=0 jtj : 1 Letting T !1 gives t = X1 j=0 jtj which is called an in…nite moving-average process (MA(1)). We can now use the MA(1) representation to calculate several moments of the distribution for t.  Mean. E(t) = X1 j=0 jE(tj) = 0:  Variance. 0 = V ar(t) = E( 2 t ) = E X1 j=0 jtj 2 = 2 +  22 +  42 +


= 2(1 +  2 + 4 +


) = 2=(1 2):  Covariances. s = Cov(t; ts) = E X1 j=0 jtj X1 j=0 jtsj  = s2=(1 2) = s 0: Since the mean does not depend on time and the covariance between error terms only depend on their distance between each other (and not t), we say that t is a (weakly) covariance stationary process. This information can be substituted into (2) to give 2 () = 2 (1 2) 2666666666664 1 


T2 T1  1


T3 T2 ... ... . . . ... ... T2 T3


1  T1 T2


 1 3777777777775 : (4) 3 Ordinary Least Squares We now examine several results related to OLS when autocorrelation is present in the model. It is useful to break these results into two parts –when the model has no lagged dependent variables and when it does. 2 is an estimate of the jth-order autocorrelation coe¢ cient and
= e21 + e 2 TPT t=1 e 2 t is a term that goes to zero as T ! 1. Therefore, in the limit, we know d = 2(1 r1). If H0 is true, we would expect r1 = 0 and d = 2. If  = 1, then we would expect d = 0: If  = 1, then we would expect d = 4. A couple of notes.  The exact distribution for the d statistic depends upon X, and as a result, a unique set of critical values does not exist. Durbin and Watson have, however, developed lower and upper bounds for the true but unknown critical values. If the d statistic falls in between the lower and upper bounds for the critical value, no conclusion can be reached.  Positive autocorrelation (i.e.,  > 0) is much more common than negative autocorrelation, so often the test is one-tailed with the critical values at the lower end of the distribution. 4.2.1 Durbin’s h test Not surprisingly, the DW test does not work in the presence of lagged-dependent variables because the OLS estimates of are biased and inconsistent. Durbin has developed an alternative. The test statistic is h = r1 p T=(1 Ts2c) where s2c is the estimated variance of the coe¢ cient on yt1. The statistic h has an asymptotic standard normal distribution and can be used to test the same H0 as in the DW test. 4.3 Lagrange Multiplier Test The disadvantage of the DW test is that it has an inconclusive region and only works for AR(1) processes. The LM test helps resolve these issues. The hypotheses are H0 : no autocorrelation HA : AR(p) or MA(p): The …rst step is to run the following regression et = xt 0 + 1et1 + 2et2 +


+ petp + t: 5 The test statistic is LM = TR2 which has an asymptotic chi-square distribution with p degrees of freedom. There is a tradeo¤ associated with the choice of p. Choosing too large of a p can cause the test to lose power (i.e., lead to Type II errors). Choosing too small of a p may miss higher-order autocorrelation. 4.4 Box-Pierce Q Test The Box-Pierce Q test is similar to the LM test but it does not control for X. The test statistic is Q = T Xp j=1 r2j which is asymptotically chi-square with p degrees of freedom. A slight variation of the Box-Pierce Q test was suggested by Ljung and Box Q0 = T (T + 2) Xp j=1 r2j T j : 4.5 Gauss Example (cont.) We now perform the three tests for autocorrelation using the U.S. consumption function example. See Gauss example 12.1 for the results. 5 Generalized Least Squares 5.1 is Known The e¢ cient estimator for the model in equations (1) and (2) is ̂ = (X 0 1X)1(X 0 1Y ) = (X 0P 0PX)1(X 0P 0PY ): We need to calculate the transformation matrix P such that the transformed errors are white noise. Toward that end, de…ne the lag operator L to be LjXt = Xtj . The appropriate transformation is (1 L)Y = (1 L)X + (1 L)) Y = X +  6 where Y = 266666664 y1 y0 y2 y1 ... yT yT1 377777775 X = 266666664 x1 x0 x2 x1 ... xT xT1 377777775  =  = 266666664 1 0 2 1 ... T T1 377777775 : The problem is how to transform the …rst observation since y0 and x0 are not observed. One solution is to treat y1 and x1 as starting values and multiply the entire …rst row by p 1 2 so that 1  N(0; 2). This implies that the transformation matrix will be P = 2666666666664 p 1 2 0


0 0  1 0 0 ... . . . ... 0 0 1 0 0 0


 1 3777777777775 : (7) Technically, this means that the transformed model has no constant since the transformed constant has a di¤erent …rst value. Also, remember that the transformation matrix P above is only valid for AR(1) autocorrelation processes. Higher-order process involve more complicated P . 5.1.1 Maximum Likelihood Estimation Start by writing the transformed model as yt = yt1 + x 0 t x0t1 + t: The likelihood (joint probability) function can then be written as L() = f(y1)f(y2jy1)


f(yT jyT1) where we iteratively use the de…nition of a conditional distribution f(x2jx1) = f(x1; x2) f(x1) : The log likelihood function is then lnL() = ln f(y1) + XT t=2 ln f(ytjyt1): 7