Clearing the Way for Better Benefit-Cost Analyses

December 23, 2009

Benefit-cost analyses (BCA) — quantifying benefits of interventions, often expressing them in dollars returned per dollar invested — are key drivers of early education policy. They’re widely consulted when early education decisions are debated, but few who use them have much in the way of an understanding of how they come about. A booklet just off the press from the National Research Council goes a long way toward explaining the issues.

Strengthening Benefit-Cost Analysis for Early Childhood Interventions is a summary of a March 2009 workshop where leading practitioners of the discipline, including NIEER Co-Director Steve Barnett, talked about the challenges of generating dependable BCAs and ways to strengthen them. Their discussions provide a window on the science — and art — of conducting BCAs. Here are some key issues:

• BCAs depend on rigorous program evaluations. Of course, the gold standard in rigor is the randomized controlled trial — a method that is not always available. Complicating matters is the fact that the control condition against which interventions are evaluated are seldom composed of kids who had no exposure to early childhood programs. These days, most kids in the general population attend a program of some type. These issues weren’t much of a factor in the era of the Perry Preschool Program — something that makes data from that era all the more valuable.

• Arriving at true program costs is a challenge. Budget figures gathered in advance of program implementation often don’t portray true costs and total costs may not be completely accounted for, particularly when programs involve matching or braided funding. Analysts often end up estimating cost using comparable market costs or deriving other measures such as “shadow prices.” For example, in many developing economies observed wage rates overstate the true marginal cost of labor while observed interest rates understate the true cost of capital. Accurate estimation of cost is one of the most neglected aspects of this work. All too often, cost receives little attention and the cost estimate used has no scientific basis at all. Yet, cost is just as important for arriving at a good decision as benefit.

• Assessing program value is arguably the area where researchers have the most work cut out for them. Some benefits of programs like greater socio-emotional development or better health behaviors are inherently more difficult to put a value on and have probably been under-estimated in the past. Manifestations of their value often don’t occur for years, even decades, in the future. In lieu of very long-term studies we must build on other research, linking pre-K to outcomes—grade retention, behavior problems, achievement, dropout—that other studies in turn link with later education, earnings and employment, mental and physical health, crime, and civic participation.

• Maintaining the integrity of study samples and having robust data available for long-term studies is a growing concern due to degradation of contact information and the growth of privacy concerns.

The presenters pointed to work done in other fields that has the potential to inform BCAs in early childhood education. In health economics, for instance, analysts are measuring the quality and length of lives saved by a health intervention in terms of a Quality Adjusted Life Year (QALY). Researchers now estimate the value of detecting and medically treating lead poisoning at $1,300 per QALY gained. When they factored in the additional cost savings from remedial education not needed when lead poisoning is prevented, they found the intervention was a sound investment.

Other recommendations the group discussed include more standardization of economic measures such as discount rates that analysts apply over time and developing more standardized practices for research procedures in the field.

Rx for Better Urban Schools: High-Quality Pre-K

December 11, 2009

Children’s math scores at fourth and eight grade haven’t progressed appreciably in most urban school districts over the last two years, says the most recent report from U.S. Department of Education. Only four of the 11 urban districts the National Assessment of Educational Progress (NAEP) has been tracking since 2003 showed significant gains. That doesn’t mean progress hasn’t been made. Urban districts, with their higher proportion of minority children and English Language Learners, represent the nation’s biggest education challenge and if we go back to 2003 when NAEP began, the urban districts have made some progress.

Nevertheless the leveling off suggested by the current report should be cause for concern because it tells us more needs to be done to move the needle toward continuing progress in these districts where the achievement gap between blacks and Hispanics and whites remains shamefully wide. We wish an urban New Jersey district were in the report because districts in cities like Newark and Camden have had the benefit of the state’s high-quality Abbott Preschool Program for a number of years. NIEER’s long-term research on the Abbott Program shows children who had two years of the program achieved gains in a variety of math measures including applied problems, calculation and math fluency through second grade.

Secretary of Education Arne Duncan, who ran the Chicago public schools, champions high-quality preschool education as a prerequisite to success in school. That is also a key recommendation in a compelling new report titled “A New Deal for Urban Public Schools” authored by Andrew J. Rotherman and Sara Mead in the Harvard Law & Policy Review. When Secretary Duncan and I released the findings from the State of Preschool 2008 yearbook at the Oyster-Adams Bilingual School in the District of Columbia earlier this year, we read The Very Hungry Caterpillar to a class of enthusiastic kids who shouted out each part of the story as we came to it. We need a lot more of that in urban districts as well as teachers skilled at recognizing and extending the math and science lessons in the caterpillar’s culinary exploits.

Steve Barnett
Co-Director, NIEER

%d bloggers like this: