A Study on Teacher Incentives and School Attendance in India, Slides of Microeconomics

Download A Study on Teacher Incentives and School Attendance in India and more Slides Microeconomics in PDF only on Docsity! Education Quality 1 / 49 Docsity.com School quality in Developing Countries • There has been rapid improvement in school enrollment in developing countries over the last 10-15 years. • However these improvements have not been matched by improvement in school quality: • Low learning performance (ASER study in India) • Massive Teacher absence (Chaudhury and other: 24% in India) • Education quality has been an extremely active domain of research, and in particular there are a series of randomized evaluation paper on various issues: • “Production function” issues: class size, textbooks, flipcharts, etc. • Incentives for students, parents, and teachers • School systems: • Pedagogy (curriculum etc.) • Para-teachers vs regular teachers • Parent information/mobilization (report cards, school commitees etc.) 2 / 49 Docsity.com The Context • We worked with Seva Mandir, an NGO in rural Rajasthan • They run 150 “non-formal education center” (NFE): single teacher school for students who do not attend regular school. • Students are 7-14 year old, completely illiterate when they join. • Schools teach basic hindi and math skills and prepare students to “graduate” to primary school. • In 1997, 20 million children were served by such NFEs 5 / 49 Docsity.com The Intervention • Teacher in Intervention school were provided with a camera with non-temperable time and date stamp 6 / 49 Docsity.com The Intervention • Teacher in Intervention school were provided with a camera with non-temperable time and date stamp • Instructed to take a picture of themselves and the children every day (morning and afternoon). A valid pairs of picture has: • Two pictures taken the same day, separated by at least 5 hours each. • At least 8 children per picture • Payment is calculated each month and is a non-linear function of attendance: • Up to 10 days: Rs 500. • Each day above 10 days: Rs 50. • In non-intervention schools, teachers receive Rs 1000, and are reminded by attending at least 20 days is compulsory. 8 / 49 Docsity.com Power • We know that E [Yi (0)|Wi = 1] = [Yi (0)|Wi = 0] • But in a finite sample, it may or may not hold. • Size (level) of a test (e.g. test H0 ATE=0): Probability of a type I error: I reject H0 when H0 is true • Generally we set the size at 5%. • Power of a test: 1-probability of type II error. • Type II error: for a given size, I do not reject 0, when I should have. • Power depend on effect of program, and on precision of the estimate: • Sample size • Level of Randomization: If I randomize at the group level, I need to cluster at this group level: need to adjust power calculation for that (it will depend on size of the group, and expected correlation of outcomes within the group). 11 / 49 Docsity.com The Randomized evaluation Checklist 1 What was the power of the Experiment? • At what level was the experiment randomized? • We need to take into account clustering at that level in computing our standard error • This affect our power as well 2 What the randomization successful (was there balance between treatment and control group in covariates) • Ways to enforce balance: Stratifying (creating block of covariates, and randomize within those) • Ways to check balance: Compare covariates Did we have attrition (lost observations)? 3 4 5 • If so, how did we deal with it? Did we have non-compliance? If so how did we deal with it? • Did we have contagion (externalities) between treatment and control group? 12 / 49 Docsity.com Checking the Balance in the Camera Experiment Treatment Control Difference (1) (2) (3) School Open 0.66 0.64 0.02 (0.11) 41 39 80 Number of Students Present 17.71 15.92 1.78 (2.31) 27 25 52 Teacher Test Scores 34.99 33.62 1.37 (2.01) 53 56 109 Teacher Highest Grade Completed 10.21 9.80 0.41 (0.46) 57 54 111 Table 1: Is School Quality Similar in Treatment and Control Groups Prior to Program? A. Teacher Attendance B. Student Participation (Random Check) C. Teacher Qualifications 13 / 49 Docsity.com The Randomized evaluation Checklist 1 What was the power of the Experiment? • At what level was the experiment randomized? • We need to take into account clustering at that level in computing our standard error • This affect our power as well 2 What the randomization successful (was there balance between treatment and control group in covariates) • Ways to enforce balance: Stratifying (creating block of covariates, and randomize within those) • Ways to check balance: Compare covariates 3 4 5 Did we have attrition (lost observations)? • If so, how did we deal with it? Did we have non-compliance? If so how did we deal with it? • Did we have contagion (externalities) between treatment and control group? 16 / 49 Docsity.com Attrition • At the school level: some schools got lost, for reasons not related to the program At the individual level for the test: we have substantial • attrition • Why is that a potential problem? • When will it be a problem? What should we check? • • percentage attrition is not differential by group observable characteristics of attritors are no different in T and • C group If not what can we do? • • Assume a selection process, and correct for it (we lose main advantage of a random sample) Provide bounds • 17 / 49 Docsity.com Treatment Control Difference Treatment Control Difference ) ) ) ) Took Written 0.36 0.33 0.03 0.61 0.57 0.04 (0.04) (0.05) Math 0.14 0.00 0.14 -0.08 -0.24 0.16 (0.10) (0.15) Language 0.14 0.00 0.14 1.71 1.60 0.11 (0.10) (0.11) Total 0.14 0.00 0.14 0.35 0.24 0.12 (0.10) (0.11) Notes: (1) Test Scores in Panel B are normalized by the mean of the mid-test control. (2) Standard Errors are clustered by school. B. Exam Score Means Mid Test Post Test Table 9: Descriptive Statistics for Mid Test and Post Test A. Attrition Process Percent Attrition 0.11 0.22 -0.10 0.24 0.21 0.03 (0.05) (0.04 Difference in Percent Written of Pre-Test attriters-stayers 0.01 0.03 0.02 0.06 -0.03 0.10 (0.06) (0.06 Difference in Verbal Test of Pre-Test attriters-stayers 0.05 0.08 -0.03 0.02 0.12 -0.10 (0.14) (0.14 Difference in Written Test of Pre-Test attriters-stayers -0.41 -0.23 -0.18 -0.19 -0.13 -0.06 (0.34) (0.29 Attrition 18 / 49 Docsity.com Attendance: Graphical Evidence Note: (1) The program began in September 2003. August only includes the 80 schools checked before announcement of program. September includes all random checks between August 25 through the end of September. (2) Child learning levels were assessed in a mid-test (April 2004) and a post-test (November 2004). After the post-test, the "official" evaluation period ended. Random checks continued in both the treatment and control schools. Figure 2: Percentage of Schools Open during Random Checks 0% 20% 40% 60% 80% 100% Aug 03 Nov 03 Feb 04 May 04 Aug 04 Nov 04 Feb 05 May 05 Aug 05 Nov 05 Feb 06 Month Treatment Control Mid-Test Post-Test 21 / 49 Docsity.com Attendance: tables Treatment Control Diff Until Mid-Test Mid to Post Test After Post Test (1) (2) (3) (4) (5) (6) 0.79 0.58 0.21 0.20 0.20 0.23 (0.03) (0.04) (0.04) (0.04) 1575 1496 3071 882 660 1529 0.78 0.63 0.15 0.15 0.15 0.14 (0.04) (0.05) (0.05) (0.06) 843 702 1545 423 327 795 0.78 0.53 0.24 0.21 0.14 0.32 (0.04) (0.05) (0.06) (0.06) 625 757 1382 412 300 670 Figure 3: Impact of the Cameras (out of at least 25 visits) Notes: (1) Child learning levels were assessed in a mid-test (April 2004) and a post-test (November 2004). After the post-test, the "official" evaluation period was ended. Random checks continued in both the treatment and control schools. (2) Standard errors are clustered by school. (3) Panels B and C only include the 109 schools where teacher tests were available. Table 3: Teacher Attendance Sept 2003-Feb 2006 Difference Between Treatment and Control Schools A. All Teachers B. Teachers with Above Median Test Scores C. Teachers with Below Median Test Scores 0 2 4 6 8 1 4 7 10 13 16 19 22 25 Atte ndance Fre que ncy N um be r of T ea ch er s p re se nt e xa ct ly x tim es Treatment Control 22 / 49 Docsity.com Cheating? Scenario Number Percent of Total School Open and Valid Photos 879 66% School Open and Invalid Photos 179 13% School Closed and Valid Photos 88 7% School Closed and Invalid Photos 191 14% School not open for full 5 hours 43 24% Only one photo 90 50% Not enough Children 36 20% Instructor not in Photo 9 5% Don't Know 1 1% Random check completed after the school closed 13 15% Camera broke/excused meeting 21 24% Teacher left in the middle of the day 54 61% Table 4: Comparing Random Checks to Photo Data for Treatment Schools A. Possible Scenarios B. Out of 179 where School is Open, the photos are invalid because…. C. Out of 88 where School is Closed and the photos are valid….. Figure 4: Difference in the Percent of Open Schools Between Treatment and Control, By Hour 0.00 0.10 0.20 0.30 0.40 8 AM 9 AM 10 AM 11 AM 12 PM 1 PM 23 / 49 Docsity.com Regression Scoreikj = β1 +β2Treatj +β3Pre Writij +β4Pre oralij +β5Writ+�ijk 26 / 49 Docsity.com Test Score results Math Lang Total Math Lang Total (1) (2) (3) (4) (5) (6) (7) (8) 0.04 0.15 0.16 0.17 0.06 0.21 0.16 0.17 (0.03) (0.07) (0.06) (0.06) (0.04) (0.12) (0.08) (0.09) 1893 1893 1893 1893 1760 1760 1760 1760 0.02 0.13 0.13 0.14 0.05 0.17 0.13 0.15 (0.03) (0.07) (0.05) (0.06) (0.04) (0.10) (0.07) (0.07) 1893 1893 1893 1893 1760 1760 1760 1760 0.14 0.13 0.15 0.2 0.13 0.16 (0.08) (0.06) (0.07) (0.14) (0.09) (0.10) 1550 1550 1550 1454 1454 1454 0.19 0.28 0.25 0.28 0.28 0.25 (0.12) (0.11) (0.11) (0.18) (0.11) (0.12) 343 343 343 306 306 306 0.07 0.18 0.18 0.2 0.07 0.22 0.17 0.18 (0.03) (0.07) (0.07) (0.07) (0.05) (0.12) (0.09) (0.09) 891 891 891 891 821 821 821 821 0.02 0.12 0.14 0.14 0.05 0.19 0.16 0.16 (0.04) (0.09) (0.07) (0.07) (0.04) (0.15) (0.10) (0.10) 988 988 988 988 929 929 929 929 B. With Controls A. All Children Table 10: Estimation of Treatment Effects for the Mid- and Post-Test Mid-Test Post-Test Took Written Took Written C. Took Pre-Test Oral D. Took Pre-Test Written E. Girls Notes: (1) The table presents the coefficient estimate of being in a treated school on the sum of a child's score on the oral and written exams. All regressions include controls for the child's learning levels prior to the program. (2) The mid and post test scores are normalized by mid test control group. (3) Controls in Row B include Block, Teacher Test Scores, and Infrastructure Index. (4) Standard errors are clustered by school. F. Boys 27 / 49 Docsity.com Results by Pre-test score Math Lang Total Math Lang Total (1) (2) (3) (4) (5) (6) (7) (8) 0.14 0.13 0.15 0.2 0.13 0.16 (0.08) (0.06) (0.07) (0.14) (0.09) (0.10) 1550 1550 1550 1454 1454 1454 0.19 0.28 0.25 0.28 0.28 0.25 (0.12) (0.11) (0.11) (0.18) (0.11) (0.12) 343 343 343 306 306 306 Table 10: Estimation of Treatment Effects for the Mid- and Post-Test Mid-Test Post-Test Took Written Took Written C. Took Pre-Test Oral D. Took Pre-Test Written 28 / 49 Docsity.com Estimating the impact of teacher absence Method: OLS OLS OLS 2SLS Sample: Control Schools Treatment Schools Treatment Schools All Schools Data: Random Check Random Check Photographs Random Check (1) (2) (3) (4) Took Written 0.02 0.28 0.36 0.26 (0.10) (0.08) (0.11) (0.19) Total Score 0.20 0.39 0.87 1.07 (0.19) (0.21) (0.22) (0.43) N 878 1015 1015 1893 Took Written 0.24 0.51 0.59 0.33 (0.16) (0.15) (0.20) (0.22) Total Score 0.58 1.17 0.98 0.97 (0.35) (0.36) (0.53) (0.47) N 883 877 877 1760 Table 12: Does the Random Check Predict Test Scores? B. Post-test (Sept 03 -Oct 04) A. Mid-test (Sept 03-April 04) Notes: (1) The table presents the coefficient estimate of the teacher's attendance on the sum of a child's score on the oral and written exams. All regressions include controls for the child's learning levels prior to the program. (2) The mid and post test scores are normalized by the mid test control group. (3) Standard errors are clustered by school. Item Cost Camera Cost1 1133 Film Cost 1392 Battery Cost 552 Photo Development and Printing: 1852 Teacher Salaries2 0 Labor Cost to Run Program3 450 Total Costs to Run Program 5379 A. Camera Cost B. Salaries Table 13: Cost of Program Per Center over 12 Month Period Notes: (1) Assumes cameras last 3 years. (2) The average teacher salary was Rs1000 in program. Thus, in the absence of the program, it would be the same. (3) It takes approximately 50 man hours to process 115 schools per month. Assuming that a staff worker is paid Rs 10,000 per month and works a 40 hour week, it takes 1/2 hour of labor at Rs37.5 to complete one center per month. 31 / 49 Docsity.com Monitoring or Incentives? Preliminary Evidence • Are teachers sensitive to increased monitoring or to incentives? • Preliminary evidence based on Regression Discontinuity Design • Consider a case where treatment is assigned when the treatment is assigned based on a strict threshold: • Sharp RD: Wi = 1[Xi > c] • Fuzzy RD: limx c pr(Wi = 1 Xi = x) = limx c pr(Wi = 1 Xi = x) ↓ | � ↑ | • Identification assumption for RD: limx↓c E [Yi (0)|Xi = x ] = limx↑c E [Yi (0)|Xi = x ] • Estimator: we try to approximate: limx↓c E [Yi |Xi = x ] − limx↑c E [Yi |Xi = x ] • In the sharp RD: this will be the treatment effect • In the fuzzy RD: we use the treshold as instrument: compute our friend the Wald estimate. 32 / 49 Docsity.com RD in the teacher case • In practice: We try to estimate a smooth (non-parametric) function of the relationship between Y and X (here: day in the month and whether teacher works). • We then use this to estimate the limits at the threshold, from the left and the right. • When we switch from the last day of the month to the first day of the month: • A teacher who has attended 9 days or less in the rest of the month faces no incentive at the end of month t and faces incentives again at the end of month t + 1. • A teacher who has attended more than 10 days in the rest of the month face a Rs 50 incentives at the end of month t and slightly smaller at the beginning of the next month • Graphical Evidence • Regression: Witm = α+β1m(d > 10)+γF +λ1m(d > 10)∗F +υi +µm�is , (1) 33 / 49 Docsity.com The Model • Each day, a teacher chooses whether or not to attend school, by comparing the value of attending school to that of staying home or doing something else. • State space s = (t, d), where t is the current time and d is the days worked previously in the current month. • Payoffs: • If the teacher does not attend school: µ + �t • Payoff of attending school is calculated at the end of the month according to: π(d) = 500 + max{0, d − 10} (2) T takes value between 1 and T = 25. • • Transitions: Each day, t increases by one, unless t = T , in which case it resets to t = 1. If a teacher has worked in that period d increases by one, otherwise it remains constant. 36 / 49 Docsity.com Value function Given this payoff structure, for t < T , we can write the value function for each teacher as follows: V (t, d) = max{µ + �t + EV (t + 1, d), EV (t + 1, d + 1)}. (3) At time T , we have: V (T , d) = max{µ+�T +βπ(d)+EV (1, 0), βπ(d +1)+EV (1, 0)}, (4) where β is marginal utility of income. EV (1, 0) enters both side and can thus be ignored: we can solve each month independently, backwards from time T . 37 / 49 Docsity.com Identification • Identification is constructive, and based on partitions of the state space. • At time T , the agent faces a static decision; work if: µ + �T + βπ(d) > βπ(d + 1). (5) • The probability of this event is: Pr(work |d , θ) = Pr(�T > β(π(d + 1) − π(d)) − µ) (6) = 1.0 − Φ(β(π(d + 1) − π(d)) − µ), (7) 38 / 49 Docsity.com iid model, with or without fixed effect Simply write the empirical counterpart of the maximization problem. The log likelihood is: N Mi Tm� � � LLH(θ) = [1(work)Pr(work|t, d , θ) i=1 m=1 t=1 +1(not work)(1 − Pr(work|t, d , θ)], where: Pr(work|t, d , θ) = Pr(µ + �t + EV (t + 1, d) < EV (t + 1, d + 1)) = Pr(�t < EV (t + 1, d + 1) − EV (t + 1, d) − µ) = Φ(EV (t + 1, d + 1) − EV (t + 1, d) − µ), (10) 41 / 49 Docsity.com Serial correlation • Both estimation and identification are a little complicated... • Use method of simulated moment: simulate work history for different parameters, and try to match a distribution of days worked at the beginning of the month. • Can introduce heterogeneity by drawing p teacher from a distribution with high outside option, and 1 − p from distribution with low outside option. 42 / 49 Docsity.com Results from the structural Model Model I Model II Model III Model IV Model V Model VI Parameter (1) (2) (3) (4) (5) (6) β 0.049 0.024 0.059 0.051 0.014 0.019 (0.001) (0.001) (0.001) (0.001) (0.001) (0.001) μ1 1.55 2.315 2.063 -0.107 0.012 (0.013) (0.013) (0.012) (0.040) (0.028) ρ 0.682 0.547 0.461 (0.010) (0.023) (0.039) σ1 2 0.001 0.153 0.135 (0.011) (0.053) (0.027) μ2 3.616 1.165 (0.194) (0.101) σ2 2 0.26 0.311 (0.045) (0.051) p 0.047 0.131 (0.007) (0.015) Heterogeneity None FE None RC RC RC Table 6: Results from the Structural Model 43 / 49 Docsity.com Two out of sample tests • Prediction of the number of days worked under no incentives • Model predicts that teachers would work 52% of the time in control group • In fact they work 58% • Predicted difference treatment vs control is 26%, vs 21% in reality • The impact of a change in rule. • Seva Mandir changed rule after experiment was over (and model was estimated!) • New rule: Rs 700 for 12 days of work. Increment of Rs 70 after the 13th day Model does well too. • • Note that all the alternative models do rather poorly in these counterfactuals. 46 / 49 Docsity.com Results from the structural Model Model I Model II Model III Model IV Model V Model VI Parameter (1) (2) (3) (4) (5) (6) Heterogeneity None FE None RC RC RC ∈Bonus 3.52 1.687 6.225 10.08 0.306 0.370 (1.550) (0.098) (0.634) (1.249) (0.038) (0.029) ∈bonus_cutoff -75.49 -16.04 -50.22 -63.11 -1.29 -1.78 (6.506) (1.264) (2.612) (3.395) (0.479) (0.449) Predicted Days Worked 20.50 19.00 15.30 12.15 20.23 21.36 (0.031) (0.062) (0.058) (0.102) (3.512) (0.373) Days Worked BONUS=0 1.60 6.02 1.29 1.318 13.55 11.81 (0.597) (0.234) (0.875) (0.863) (5.251) (0.669) Out of Sample Prediction 26.16 18.886 15.08 12.956 20.86 21.57 (0.059) (0.253) (0.635) (0.520) (3.793) (0.456) Table 6: Results from the Structural Model 47 / 49 Docsity.com Distribution of Days worked under new rule Figure 6A: Predicted Fit From Model V Figure 6B: CounterFactual Fit From Model V Model V -- CounterFactual Fit 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Days Fr ac tio n Empirical Mean 2.5 percentile 97.5 percentile Model V -- Predicted Fit 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09 0.10 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 0.20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 Days Fr ac tio n Empirical Mean 2.5 Percentile 97.5 Percentile 48 / 49 Docsity.com