Why preschool critics are wrong

February 28, 2014

In a recent blog on his Brookings Institution site, Grover Whitehurst claims that the preponderance of scientific evidence does not indicate lasting positive benefits from preschool.  Others disagree, he says, because they are not as “picky” about the evidence. As there is no disputing taste, I assume he means that the research he prefers is more rigorous and relevant, not just better aligned with his personal preferences.  Hence, we would be looking for a valid and reliable process when he arrays the research and grades each study.  In fact, his list of studies and his analyses are appallingly inaccurate for someone who claims to be an expert.  As shown in detail below, Whitehurst omits much of the relevant research, and he misrepresents the studies that he includes. Steven Barnett

Whitehurst claims that “Not one of the studies that has suggested long-term positive impacts of center-based early childhood programs has been based on a well-implemented and appropriately analyzed randomized trial.”   This claim is false based even on the studies he does cite.  His own statements in the blog regarding the Perry Preschool study and its re-analyses by Jim Heckman contradict this claim, as do older analyses demonstrating that minor departures from random assignment in the Perry study had no substantive effects on the results.[1] No study is perfect, so it is seems odd that Perry receives an A- for an inconsequential fault when other less than perfect studies get an A. Then there is the Infant Health and Development program (IHDP) study, which Whitehurst assigns higher grades than Perry, but which he seems to forget when making his “not one” study claim.

Yet Whitehurst’s credibility problem is far more serious when one turns to the studies that are missing from his analysis.

In fact, a number of other well-implemented and appropriately analyzed randomized trials find lasting effects from preschool education.  For example, a study of long-term effects conducted by the Institute for Developmental Studies (IDS) included 402 children randomly assigned to a public school pre-K program or to a control group at age 4 for one year.[2]  A teacher and an aide staffed each preschool classroom of 17 children. Positive effects were found through at least third grade.  Even longer term follow-up indicates adult gains in achievement, educational attainment, and employment, but suffers from severe attrition. So while we can have strong confidence in the results through third grade, we have less confidence in the very long-term results.  However, the findings for adults are consistent with the earlier results in the elementary grades and with findings in Perry and other studies.  Another randomized trial of preschool education is noteworthy because it was conducted with relatively advantaged children, and it also found evidence of lasting effects on achievement into the early elementary grades.[3]

Inexplicably, Whitehurst fails to recognize a large number of studies (once again including well-implemented randomized trials) that compare one form of preschool education to another to study the effects of curriculum, length of day, and other features.  When such studies find lasting differences due to the type of preschool program, from the end of kindergarten to the end of high school, they add to the evidence that high-quality preschool education per se has long-term effects.  This literature includes studies (herehere, and here) over many years, some begun decades ago with very long-term follow-ups and some very recent with much shorter follow-ups.[4]  These studies also add to the evidence for successful scale up in large-scale public programs.

As preschool research is conducted in other countries, not just the United States, there is a broad range of research Whitehurst omits that finds lasting benefits from quality preschool education, including rigorous studies in countries with universal programs and additional well-implemented, appropriately analyzed randomized trials.[5]  When similar outcomes from quality pre-K are found with different populations in different contexts, such studies are confirmatory—not irrelevant.  Taken together, they indicate that the relationships between quality preschool education and long-term outcomes are quite robust with respect to variations in the children and families served.

Returning to the matter of how Whitehurst represents the few studies he prefers to include, the ongoing Tennessee evaluation of pre-K effectiveness is one of only two to receive double A grades.  He calls this study a well-implemented and analyzed Random Control Trial (RCT) and reports that it finds no differences later in elementary school.

This description of the Tennessee study and its findings is shockingly inaccurate.

The Tennessee pre-K evaluation includes both a large randomized trial of pre-K that follows children using the data routinely collected by schools, and a smaller intensive substudy (ISS) in which randomization failed.  The results Whitehurst cites come from that substudy, which is not analyzed as a randomized trial.  Let me quote directly from the study authors: “The nonconsent rates for the two cohorts in the ISS sample mean that we do not have data on the main ISS outcomes for many of the children who were initially randomized, so analysis on the basis of that randomization is not possible.”[6]  To be perfectly clear, because so many fewer parents in the control group agreed to have their children tested in the substudy, randomization was not used to analyze the data.  Instead, the substudy used a quasi-experimental approach of the type that Whitehurst otherwise gives lower grades.

Yet it is not just the Tennessee study’s methods that are misrepresented, but also its results.  While the full randomized trial will not provide test score results until children reach the state’s third grade tests, it does provide results for grade retention.  Let me again quote the study’s authors directly:

For the Intensive Substudy sample, there was a statistically significant difference between the 4.1% of the TN‐VPK participants who were retained in kindergarten compared to the 6.2% retention rate for the nonparticipants. This effect was confirmed in Cohort 1 of the full randomized sample, with retention data still unavailable for Cohort 2 of that sample. In Cohort 1, 4.0% of the TN‐VPK participants were retained in kindergarten compared to 8.0% of the nonparticipants, also a statistically significant difference.[7]

In other words, analyses of both the compromised Tennessee substudy and the full randomized trial find that the pre-K program significantly reduced grade retention in kindergarten.  The only finding from this adequately randomized part of the Tennessee study is this persistent positive effect, and this effect is twice as large as that found in the quasi-experimental substudy (which still finds a significant positive effect of pre-K).  Yet, Whitehurst reported the study found “no differences later.”

I could go on to list additional problems with Whitehurst’s review, but surely these suffice to demonstrate that his summary of the evidence is just plain wrong–not picky.

– W. Steven Barnett, Director, NIEER and CEELO

This entry originally appeared in Valerie Strauss’ education blog “The Answer Sheet” at the Washington Post.

[1] Weikart, D.P., Bond, J.T., & McNeil, J.T. (1978). The Ypsilanti Perry Preschool Project: Preschool years and longitudinal results through fourth grade. Ypsilanti, MI: HighScope. Barnett, W.S. (1996). Lives in the balance: Age 27 benefit-cost analysis of the High/Scope Perry Preschool Program. Ypsilanti, MI: HighScope.

[2] Deutsch, M., Taleporos, E., & Victor, J. (1974). A brief synopsis of an initial enrichment program in early childhood. In S. Ryan (Ed.), A report on longitudinal evaluations of preschool programs Volume 1: Longitudinal evaluations (pp. 49-60). Washington, DC: Office of Child Development, US Department of Health, Education, and Welfare. Deutsch, M. , Deutsch, C. P. , Jordan, T. J. , & Grallo, R. (1983). The IDS program: An experiment in early and sustained enrichment. In Consortium for Longitudinal Studies (Ed. ). As the Twig is Bent: Lasting Effects of Preschool Programs(pp. 377-410). Hillsdale, NJ: Lawrence Erlbaum. Jordan, T. J. , Grallo, R. , Deutsch, M. , & Deutsch, C. P. (1985). Long-term effects of early enrichment: A 20-year perspective on persistence and change. American Journal of Community Psychology13(4), 393-415.

[3] Larsen, J. M. , & Robinson, C. C. (1989). Later effects of preschool on low-risk children. Early Childhood Research Quarterly4, 133-144.

[4] Schweinhart, L. J. & Weikart, D. P. (1996). Lasting differences: The High/Scope Preschool Curriculum Comparison Study through age 23. Ypsilanti, M: High/Scope. Chambers, B. , Cheung, A. , Slavin, R. E. , Smith, D. , & Laurenzano, M. (2010). Effective early childhood education programmes: a best-evidence synthesis. Reading, England: CfBT Education Trust. Robin, K. B. , Frede, E. C. , & Barnett, W. S. (2006). Is More Better? The Effects of Full-Day vs Half-Day Preschool on Early School Achievement. New Brunwick, NJ: National Institute for Early Education Research. Preschool Curriculum Evaluation Research Consortium (2008). Effects of Preschool Curriculum Programs on School Readiness (NCER 2008-2009). Washington, DC: National Center for Education Research, Institute of Education Sciences, U. S. Department of Education. Washington, DC: U. S. Government Printing Office. Clements, D. H. , & Sarama, J. (2011). Early childhood mathematics intervention. Science333 (6045), 968-970.

[5] Ruhm, C. J. , & Waldfogel, J. (2012). Long term effect of early childcare and education. Nordic Economic Policy Review. Economics of Education, 23-51. Apps, P. , Mendolia, S. , & Walker, I. (2013). The impact of pre-school on adolescents’ outcomes: Evidence from a recent English cohort. Economics of Education Review37, 183-199.

Raine, A. , Mellingen, K. , Liu, J. , Venables, P. , & Mednick, S. A. (2003). Effects of environmental enrichment at ages 3–5 years on schizotypal personality and antisocial behavior at ages 17 and 23 years. American journal of psychiatry160(9), 1627-1635.

[6] Lipsey, M. W. , Hofer, K. G. , Dong, N. , Farran, D. C. , & Bilbrey, C. (2013). Evaluation of the Tennessee voluntary prekindergarten program: Kindergarten and first grade follow-up results from the randomized control design (pp. 22-23). Nashville, TN: Vanderbilt Peabody College.

[7] Lipsey, M. W., Hofer, K. G., Dong, N., Farran, D. C., & Bilbrey, C. (2013). Evaluation of the Tennessee voluntary prekindergarten program: Kindergarten and first grade follow-up results from the randomized control design (p. 50). Nashville, TN: Vanderbilt Peabody College.


Reflections on Play: Join the Conversation

February 25, 2014

The early childhood field has a history of conflict over means and goals that periodically erupts into public debates about the role of play versus academics and construction versus instruction. Concerns about whether preschool and kindergarten have become too stressful and regimented are met head on with concerns that they are academically weak and fail to cognitively challenge children. These conflicts have been intensified by increased demands for assessment  and Common Core State Standards driving curriculum in the early grades.

kids playing dressup

Some worry that the push for quality education even partially driven by a desire to improve achievement may deprive children of important childhood experiences. Others worry that unstructured play without teacher engagement does little to develop children’s minds, particularly for children at high risk of academic failure. Fears are further fanned through research with one recent study reporting that kindergarten may be “too easy,”  and another questioning the assumption of causal relationships between play and child development in the areas of creativity, reasoning, and executive function.

If this debate takes place only in the popular press it seems all too likely that we will be propelled into yet another unproductive and oversimplified debate over play versus academics. To promote a better discussion, NIEER will be hosting a conversation beginning with brief blog posts from experts in early childhood education on play; the goals, content, and methods of early education; and what best practice should look like in the early years. We want you to get involved! Leave comments on the blog with your own thoughts on play-based learning in preschool, share resources on Facebook and Twitter, and catch up on our past writings on the importance of play.


The Profound Impact of Early Education

February 10, 2014

Every family in the United States should be able enroll their child in good preschool program, beginning at age three and ought to have access to good child care–including that provided by themselves at home–for infants and toddlers.  The benefits would be profound for our children and the larger society, especially children from low-income families–half of all young children–but not only for them. Today we are far from achieving this vision of a more nurturing society and our progress has been painfully slow over the last two decades.  All levels of government will have to increase their support for young children and families, including the federal government which can best lead the way by priming the pump–providing financial support and incentives that encourage and enable state and local governments to develop sustainable quality programs.  For example, federal matching funds that start off big and gradually decline are well-designed to address the major challenge to state and local funding of pre-K: states must pay for pre-K now, but the off-setting cost-savings grow year by year as children progress through the grades.

Only about half of 3-and-4-year-olds attend a preschool program. From a national observational study where the quality of care for 4-year-olds was directly observed, we know that few children attend good preschool programs. Public programs improve quality somewhat for children in poverty, so children in middle-income families actually attend worse programs on average. However, many young children are in family day care homes that provide even worse care, so much so that the family day care homes attended by most African-American and Hispanic children are of low quality. Our under-funded child care subsidy systems are perversely designed to encourage this and may actually increase the number of children in settings that harm their development.  As Cindy Lamy and I point out in our chapter in the recently published book Closing the Opportunity Gap, edited by Prudence Carter and Kevin Welner, much of the educational failure and inequality that plagues our country is rooted in children’s experiences before they enter kindergarten.

In recent debates some have claimed that the federal government already spends a great deal on the care and education of young children and that most of this is wasted on ineffective programs. These claims are based on faulty math and misrepresentation of the evidence. Take for example, Grover Whitehurst’s estimate that the federal government spends $5,000 on every young child in poverty. He begins with $20 billion in annual spending on children–fair enough, Ron Haskins and I calculated that number together. But there are 5 million children in poverty, which yields $4,000 per child in poverty. And, of course, all of this is not spent just on children in poverty, so it would be much more reasonable to divide by the number of children under 5 in low income families = about 10 million children, and a  figure of $2,000 per child.

How does federal spending on disadvantaged young children’s care and education compare with federal largesse more generally?  Let’s consider two examples. The tax break for capital gains and dividends which allows wealthy hedge fund managers to pay a 15% income tax rate costs taxpayers $83 billion annually. In 2012, the federal government spent more than $20 billion on farm subsidies received by a small, relatively wealthy population. The 2014 farm bill increases so-called “crop insurance” subsidies that are actually open-ended revenue insurance for farmers.  If Congress set evidence-based priorities for all programs based on returns to the taxpayers, young children would see more money, not less.

What about the claims that federal money spent on young children is wasted?  I would have to agree that lost opportunities abound, but not as the critic’s suppose. Let’s get this clear: the Head Start national impact study’s oft cited intent-to-treat estimates grossly underestimate the program’s actual impacts, and even modest Head Start benefits likely generate benefits that exceed costs. Critics also seem to be in some kind of time warp that missed the last decade of Head Start reform and the evidence that these reforms increased effects on language and literacy development (if only Congress would call an expert in early language and literacy development to testify, surely this would be noted).

The biggest problems with federal programs for young children are that they ask too much of too little money. Nevertheless, both child care and Head Start spending could be better focused on learning and teaching. No amount of wishful thinking will permit this to be accomplished by reducing their budgets or just giving the money to states. The first step to improve Head Start should be doubling notoriously low salaries for highly effective teachers. The second step should be to dramatically reduce bureaucratic compliance requirements for any Head Start that agrees to be judged instead by teaching quality and children’s learning gains. The President’s pre-K proposal is in its own way a Head Start reform proposal that puts states in charge of the education of 4-year-olds; those two steps I set out above would go far toward enabling Head Start to integrate with and enhance state pre-K. States like West Virginia and New Jersey have already successfully integrated their programs with Head Start and child care at high standards. Federal policy that followed such leads could support states to significantly improve opportunity for America’s youngest citizens.

– Steve Barnett, Director, NIEER

This entry is cross-posted to The National Journal’s Education Insiders blog  in response to the prompt “The ‘Noble Intention’ of Giving Early Education” from Fawn Johnson.


I’ve Got Two Words to Say

February 5, 2014

We continue to ease into a new year, complete with crossed-out dates on checks, fleeting personal resolutions, and the trek by legislators back to state capitals to do the People’s Work.

It’s been several years since I last walked down the aisle of my state legislature, as finishing touches were made toward establishing voluntary, universal pre-K through public-private partnerships. It was the end of a long walk spanning several years, different Commissioners, and numerous committees. Though the outcome was generally assured, devilish details eventually gave way to compromise, but I felt a sense of relief when I walked out the State House door that last time, knowing that the greater good prevailed.

I learned many valuable lessons during the protracted process, one of which harkens back to my merit-badge days of being a Boy Scout. The motto “Be prepared” could not have been more sage. Detractors sat ready to pounce on any misstep to derail the initiative, and one sure show-stopper would be my inability to provide a concise answer to a very basic question from an audience who did not share my background in child development and early education.

Speech bubble

Between my frequent, often unplanned, visits to committee hearings, I prepped myself to succinctly address several two- or three-word questions I knew would eventually arise. These questions served as mantras when I walked from my office to theirs, specifically:

So what? Why was it worth legislators’ time and the state’s resources to even consider early education? Informing them of relevant research and connecting the dots linking strategies to benefits and costs, both human and financial, was my job. If they were concerned about the “cost of bad outcomes,” here was my opportunity to offer a proven solution. I immerse myself daily in this topic, they don’t.

What about . . . ? It is imperative for every policymaker to consider the merits and faults of policy proposals. There is no shortage of reports or opinions on charged topics such as voluntary universal pre-K, so it’s important to be current in understanding established and emerging research, and recognizing differing viewpoints or conflicting findings. Questions inevitably arise about individual studies whose results contradict the preponderance of scientific findings. Single studies should be read with great interest and skepticism. Meta-analyses provide a stronger, more compelling foundation.

Who cares? I am of the opinion that everyone cares about early education, or will, when they learn the facts. Know your allies, especially those who may be more in the middle. At the same time, some are more resistant for a number of reasons. Don’t discount others who don’t share your beliefs; work with them. It is important to understand the views of all parties, particularly if a “livable compromise” or common ground can be achieved.

Now what? After a compelling case is made and people are on board, they are looking for a quarterback to guide them to the goal line. Momentum is maintained when they can be handed a playbook with next steps and options. You’ll lose them with a tome, so keep it short, perhaps with a one-pager pulled from your back pocket.

How much? I recall an old Snoopy cartoon when he thought “Every time someone has a good idea, somebody has to bring up the budget.” This is part of a legislator’s responsibility. It’s a 2-part question, however. First, people need to know how much it will cost taxpayers for start-up and long-term operation. Second, estimates should be provided for what it is currently costing the state to not have a program, in terms of “bad outcomes” such as special education, retention, drop-out, and incarceration, and how an investment on the front end would yield savings over time.  As methods for estimating cost savings are refined, answers to these questions are expected and easily provided.

Quick questions deserve quick, compelling, accurate responses. Yet time often conspires against conducting the necessary research and making it comprehensible when important policy decisions are on the line. Fortunately, the field has an indispensable resource in the form of translational research centers (TRC) such as the National Institute for Early Education Research, HighScope Educational Research Foundation, and the Frank Porter Graham Child Development Institute. TRCs conduct, evaluate and translate scientifically valid research into practical, evidence-based information essential for policymakers and influencers to establish sound policy. I relied heavily on information from TRCs to inform committee members and state leaders and results are now speaking for themselves in terms of student achievement.

Although emotion and opinion play roles in shaping policy, we are better served when solid research informs conscionable policy. TRCs move us closer in that direction and for that, my last two words are “Thank you.”

Jim Squires, Ph.D. is Senior Research Fellow at the National Institute for Early Education Research and Center on Enhancing Early Learning Outcomes. He formerly served as Early Childhood Programs Coordinator at the Vermont Department of Education when Act 62 was signed into law.


%d bloggers like this: