多元回歸分析進一步討論課件

1、多元回歸分析進一步討論課件1第五章多元回歸分析論Multiple Regression Analysisw y = b0 + b1x1 + b2x2 + . . . bkxk + uw 4. Further Issues多元回歸分析進一步討論課件2Redefining Variables: An Examplethe determinations of infant birth weightnVariablesqbwghtkg, child birth weight in kilogramsqbwghtg, child birth weight in gramsqbwghtjin, child

2、 birth weight in jinqcigs, number of cigarettes the mother smoked per day while pregnantqpacks, packs of cigarettes the mother smoked per day while pregnant and 1 packs=20 cigsqfaminc, annual family incomenModelqy=b0+b1x+b2faminc+uqy stand for bwghtkg, bwghtg, bwghtkjin; x stand for cigs or packs多元回

3、歸分析進一步討論課件3Redefining Variables, cont.Dependent vars(1) bwghtkg(2) bwghtg(3) bwghtjin(4) bwghtjincigs-0.0131492(0.0025985)-5.06-13.1492(2.5985)-5.06-0.0262984(0.005197)-5.06packs-0.5259676(0.1039397)-5.06faminc0.0026322(0.0008252)3.182.6322(0.8282)3.180.0052644(0.0016564)3.180.0052644(0.0016564)3.18

4、Intercept3.3191141(0.029764)111.513319.141(29.7649)111.516.638282(0.595298)111.516.638282(0.595298)111.51Observations1388138813881388R20.02980.02980.02980.0298SSR448.85423444888542331795.416941795.41694S.E0.56928569.281.13861.1386Changing the scale of the y variable will lead to a corresponding chan

5、ge in the scale of the coefficients and standard errors, so no change in the significance or interpretationnChanging the scale of one x variable will lead to a change in the scale of that coefficient and standard error, so no change in the significance or interpretation多元回歸分析進一步討論課件4Redefining Varia

6、bles , cont.12let take simple regresstion for example the estimated parameter is now we change the scale of y, for example, we change kilograms to grams, then the new dependent vaiiiixx yxxb1 122newoldrable 1000 , so the new estimated parameter is1000 1000if we write the scale as , then we get The n

7、ew variance of thenewii newiinewiinewoldyyxx yxxyxxxxyybbbb222 new 222new22 new2 new residual is 111The new standard error (MSE) is The new is 11ii newi newiinewoldoldinewiuyyyynknknkRuSSRRSSTyy 2222 new222 new old22j new new old new old1The new standard error of is 11The t-stats of ioldiiijnewoldjj

8、jjjjjjnewoldjjuRyyseseSSTRSSTRttseseb bbbbbbb 多元回歸分析進一步討論課件5Redefining Variables , cont.nChanging the scale of the y variable will lead toqa corresponding change in the scale of the coefficients and standard errors, qt-stats and R2 is not changednChanging the scale of one x variable will lead to qa

9、change in the scale of that coefficient and standard error, qt-stats and R2 is not changed多元回歸分析進一步討論課件6Standardized Coefficients (Beta Coefficients)nOccasional youll see reference to a “standardized coefficient” or “beta coefficient” which has a specific meaningnIdea is to replace y and each x vari

10、able with a standardized version i.e. subtract mean and divide by standard deviationnCoefficient reflects standard deviation of y for a one standard deviation change in x 01111111The original OLS equation iswe now standardize the model, and getwhere, is the sample standard deviatiiikikiiiikkkikyyyky

11、yyxxuyyxxxxubbbbb1 122 on for the dependent variable,and is the sample for , we rewrite the equation aswhere, denote the -score of , is the -score of and the new coefficientjjykkyjjjsdxzb zb zb zvzzy zzxbare traditionally called standardized coefficients or beta coefficients.jyjb多元回歸分析進一步討論課件7Standa

12、rdized Coefficients, ExampleThe determinations of wage, wage1.rawnThe population model in level-level modelqwage=b0+b1educ+b2exper+b3tenure+unEstimating the standardized model qzwge=0.449zeduc+0.082zexper+0.331ztenurenWhat the meaning of the estimated parameters?qThe estimated coefficient of zeduc m

13、eans when the educ changed one standard deviation, the wage will change 0.449 standard deviation.nStata commandqreg wage educ exper tenure, beta多元回歸分析進一步討論課件8Functional Formn OLS can be used for relationships that are not strictly linear in x and y by using nonlinear functions of x and y will still

14、be linear in the parametersn Can take the natural log of x, y or bothn Can use quadratic forms of xn Can use interactions of x variables多元回歸分析進一步討論課件9Interpretation of Log ModelsnIf the model is ln(y) = b0 + b1ln(x) + uqb1 is the elasticity of y with respect to xnIf the model is ln(y) = b0 + b1x + u

15、qb1 is approximately the percentage change in y given a 1 unit change in x nIf the model is y = b0 + b1ln(x) + uqb1 is approximately the change in y for a 100 percent change in xnExample: the determinations of wagesqlog(wage)=0.084+0.094educ+0.109log(exper)+0.018tenure多元回歸分析進一步討論課件10Why use log mode

16、ls?nLog models are invariant to the scale of the variables since measuring percent changesnThey give a direct estimate of elasticitynFor models with y 0, the conditional distribution is often heteroskedastic or skewed, while ln(y) is much less sonThe distribution of ln(y) is more narrow, limiting th

17、e effect of outliers多元回歸分析進一步討論課件11Some Rules of ThumbnWhat types of variables are often used in log form?qDollar amounts that must be positive, such as wages, salaries, firm sales, firm market valueqVery large variables, such as population, number of employees, school enrollmentnWhat types of varia

18、bles are often used in level form?qVariables measured in years, such as education, experience, tenure, ageqVariables that are a proportion or percent, such as unemployment rate, interest rate, roe, roa多元回歸分析進一步討論課件12Quadratic ModelsnFor a model of the form y = b0 + b1x + b2x2 + u we cant interpret b

19、1 alone as measuring the change in y with respect to x, we need to take into account b2 as well, sincexxyxxy21212 so ,2bbbb多元回歸分析進一步討論課件13More on Quadratic ModelsnSuppose that the coefficient on x is positive and the coefficient on x2 is negativenThen y is increasing in x at first, but will eventual

20、ly turn around and be decreasing in x21*212at be willpoint turning the0 and 0For bbbbxExample: Kuznetz CurveGini=b0b1gdppc+b2gdppc2+u多元回歸分析進一步討論課件14More on Quadratic ModelsnSuppose that the coefficient on x is negative and the coefficient on x2 is positivenThen y is decreasing in x at first, but wil

21、l eventually turn around and be increasing in x0 and 0 when as same theis which ,2at be willpoint turning the0 and 0For 2121*21bbbbbbxyxx*For example: the cost functionC(Q)=b0 +b1Q+b2Q2+u多元回歸分析進一步討論課件15More on Quadratic Models, ExampleEffects of Pollution and House PricesnVariablesqprice, median hou

22、sing price;qnox, the amount of nitrogen oxide in the air, in parts per million;qdist, a weighted distance of the community from five employment centers, in miles;qrooms, the average number of rooms in houses in the communityqStratio, the average student-teacher ratio of schools in the community.nThe

23、 estimated modelqlog(prie)=13.39-0.902log(nox)-0.087log(dis)-0.545rooms+0.062rooms 2-0.048stratioq (0.57) (0.115) (0.043) (0.0165) (0.013) (0.006)q n=506 R2=0.603qThe estimated turn point is =0.545/(2*0.062)=4.44.4roomslog(price)多元回歸分析進一步討論課件16Interaction TermsnFor a model of the form qy = b0 + b1x1

24、 + b2x2 + b3x1x2 + u nwe cant interpret b1 alone as measuring the change in y with respect to x1, we need to take into account b3 as well, since 212311at above theevaluate typically weon ofeffect thesummarize toso ,xyxxxybbInteraction term多元回歸分析進一步討論課件17Interaction Terms, cont.Example: wage determin

25、ationsnModel with interaction terms of educ and tenureqwage=b0+b1 educ+b2exper+ b3 tenure+b4eductenure+unEstimated model with interaction terms of educ and tenureqwge=1.097+0.457educ+0.021exper-0.097tenure+0.022eductenureq (0.861) (0.063) (0.012) (0.074) (0.006)q n=526 R2=0.3247nThe effect of educ o

26、n wage at the mean of tenure isqdwage/deduc=0.457+0.022tenure=0.457+0.0225.105=0.57qWhether the estimate 0.57 is statistically different from zero? That is, whether b1b4tenure (b1b45.105) is different from zero?qwage=b0+(b1 +b45.105)educ+b2exper+ b3 tenure+b4educ(tenure-5.105)+uqwge=1.097+0.570educ+

27、0.021exper-0.097tenure+0.022eductenureq (0.861) (0.051) (0.012) (0.074) (0.006)q n=526 R2=0.3247qt=11.12, so it different from zero significantly多元回歸分析進一步討論課件18Adjusted R-SquarednR2 is simply an estimate of how much variation in y is explained by x1, x2,xk. That is,nRecall that the R2 will always in

28、crease as more variables are added to the modelnThe adjusted R2 takes into account the number of variables in a model, and may decrease1111122nSSTnSSTknSSRR22211yunSSTnSSRR多元回歸分析進一步討論課件19Adjusted R-Squared (cont)nIts easy to see that the adjusted R2 is just (1 R2)(n 1) / (n k 1), but most packages w

29、ill give you both R2 and adj-R2n You can compare the fit of 2 models (with the same y) by comparing the adj-R2qwge=3.391+0.644educ+0.070exper adj-R2=0.2222qwge=2.2220.569educ+0.190tenure adj-R2=0.2992n You cannot use the adj-R2 to compare models with different ys (e.g. y vs. ln(y)qwge=3.391+0.644edu

30、c+0.070exper adj-R2=0.2222qlog(wge)=0.404+0.087educ+0.026exper adj-R2=0.3059qBecause the variance of the dependent variables is different, the comparation btw them make no sense.多元回歸分析進一步討論課件20Goodness of FitnImportant not to fixate too much on adj-R2 and lose sight of theory and common sensenIf eco

31、nomic theory clearly predicts a variable belongs, generally leave it innDont want to include a variable that prohibits a sensible interpretation of the variable of interest remember ceteris paribus interpretation of multiple regression多元回歸分析進一步討論課件21Standard Errors for PredictionsnSuppose we want to

32、 use our estimates to obtain a specific prediction?nFirst, suppose that we want an estimate of E(y|x1=c1,xk=ck) = q0 = b0+b1c1+ + bkcknThis is easy to obtain by substituting the xs in our estimated model with cs , but what about a standard error?nReally just a test of a linear combination01 1kcqbbb0

33、111110then, we run the regression ,will get the estimated and its standard error.kkkiikkyxcxcuy onxcxcqbbq多元回歸分析進一步討論課件22Predictions (cont)nThe original regression modelqwge=2.87270.5990educ+0.02234exper+0.1693tenureq (0.7290) (0.0513) (0.0121) (0.0216) q n=526 R2=0.3064nWe want predict the wages of

34、 educ=exper=tenure=12, we can easily put the value in the above estimated equation, and get wge=6.614, but we dont know the standard error of the predicted valuenNow, we reg wage on (educ-12), (exper-12), (tenure-12)qwge=6.6140.5990(educ-12)+0.02234(exper-12)+0.1693(tenure-12)q (0.2368) (0.0513) (0.

35、0121) (0.0216) q n=526 R2=0.3064 多元回歸分析進一步討論課件23Predictions (cont)nThis standard error for the expected value is not the same as a standard error for an outcome on ynWe need to also take into account the variance in the unobserved error. Let the prediction error be 2122002000000000110000 so , and 0b

36、bbyseeseyVaruVaryVareVareEyuxxyyekk多元回歸分析進一步討論課件24Prediction interval 12.677 0.551, is which ,063. 6614. 60936. 396. 1614. 6,12expat wagefor the interval confidence 95%get weThen,0936. 30845. 32368. 0So, .0845. 3 and,2368. 0se,614. 6get we whereexample, Forgoing ,for interval prediction95% a have we

37、, given that so ,2122212200000025.00000100tenureereducyseeseyyesetyyyyeteseekn多元回歸分析進一步討論課件25Residual AnalysisnInformation can be obtained from looking at the residuals (i.e. predicted vs. observed)nExample: Regress price of cars on characteristics big negative residuals indicate a good dealnExample

38、: Regress average earnings for students from a school on student characteristics big positive residuals indicate greatest value-added多元回歸分析進一步討論課件26Predicting y in a log model ? calculate then we, log predictedget first andequation estimated the toput just can we, if ?given for epredict th toHowlog:equation estimated get the OLS use weand ,log is model population The2111011