Printable Version (193 kb PDF)

Giant Steps for the Littlest Children:
Progress in the Sixth Year of the
Abbott Preschool Program

Year Three Initial Update, 2004-2005
Early Learning Improvement Consortium

Released May 18, 2005

School House Graphic

Cynthia Esposito Lamy, EdD, Rutgers University
Ellen Frede, PhD, New Jersey Department of Education and The College of New Jersey
and
Holly Seplocha, EdD and Janis Strasser, EdD, William Paterson State University
Saigeetha Jambunathan, PhD and Jo Anne Juncker, EdD, New Jersey City University
Ellen Wolock, EdD, New Jersey Department of Education

This update presents initial findings of the third year of the ELIC study. In addition, comparisons to similar information collected by the Center for Early Education Research at Rutgers University in 1999–2000 and 2000-2001 are made to measure change over time (Barnett, Tarr, Esposito-Lamy & Frede, 2002). For a full description of the study details please see the Year One report (Lamy, Frede, Seplocha, el. al., 2004).

Presidents, governors, CEOs, police officers, parents, researchers and, of course, educators all agree that participation in high-quality preschool can give children a giant step toward entering school ready to succeed. They all know that it can be the first important advance in closing the achievement gap that exists in education. The Courts, lawmakers and others in New Jersey also understand this and have established the Abbott preschool program in the 31 school districts that serve most of the low-income families in our state.

The Abbott preschool program has evolved in both reach and quality since its inception in the 1999-2000 school year. In the fifth decision of the landmark Abbott v. Burke school funding case, the New Jersey Supreme Court required that three- and four-year-old children in the highest poverty districts receive a high-quality preschool education. Through a Department of Education (DOE) and Department of Human Services (DHS) partnership, these classrooms combine a DOE-funded six-hour, 180-day component with a DHS-funded wrap-around program that provides daily before- and after-care and summer programs. In total, the full-day, full-year program is available ten hours per day, 245 days a year.

Expansion of the program has been rapid. enrollment has expanded from only 19,000 children served in the 1999-2000 school year to over 39,000 this school year. Projected enrollment for 2005-2006 is over 43,000 children – over 80 percent of the total population of three-and four-year-olds in these districts.

The goal of the Abbott preschool program is to prepare children to succeed in school. The key to reaching this goal is to create high-quality programs that reach all children. Districts have shifted their emphasis from setting up basic program components such as staffing and finding places for children to be served to selecting and building programs that offer high-quality learning experiences for all three- and four-year-olds. The Department of Education’s Office of Early Childhood Education (OECE) works with the thirty-one Abbott districts to implement programs that prepare children to enter school with the knowledge and skills necessary to meet the Preschool Teaching and Learning Expectations: Standards of Quality and the kindergarten Core Curriculum Content Standards. Using the Abbott Preschool Program Implementation Guidelines and budget guidance, districts serve children in programs that meet high standards with adequate funding.

To ensure that districts are meeting these standards, the DOE instituted a continuous cycle of program evaluation and improvement called the Self-Assessment Validation System for Abbott Preschool Programs (SAVS). The multi-phase SAVS process focuses on program improvement augmented by fiscal accountability.

Figure 1 shows frequencies of district scores where a "1" is "not yet," a "2" is "in progress" and a "3" is "fully met." No district score is less than 1.5. Although 10 percent, or three districts, scored less than 2 ("not yet" to "in progress"), 90 percent, or 27 districts, scored between "in progress" and "fully met." All districts are implementing at least some of the best practices in each of the component areas with greatest success in Recruitment and Outreach, Mission and Philosophy, and Administration and Staff Qualifications. These are mostly static variables that are easiest to implement and measure. Areas such as services for children with disabilities and English Language Learners are components that need more work.

Figure 1: Range of SAVS Scores

Measuring the Impact

To measure the effects of Abbott preschool, the DOE formed the Early Learning Improvement Consortium (ELIC), a partnership among the department, other education professionals and academic experts at several of our state colleges and universities.

This consortium began research activities in 2002. In the spring of 2003, 2004 and 2005, members of the ELIC conducted classroom evaluations on 13 percent of the Abbott preschool classrooms. Each fall, assessments of kindergartners’ language skills were conducted. In addition, assessments of a random sample of 3,438 three-, four- and five-year-olds were administered in the fall of 2004 to determine the effect of preschool participation on skills at kindergarten entry.

The Center for Early Education Research at Rutgers University measured classroom quality in a subsample of fourteen Abbott districts in 1999 through 2001. For the current report, change in aspects of classroom quality since before 2002 is measured across this subsample.

Overall Classroom Quality

Overall classroom quality is measured using the Early Childhood Environment Rating Scale – Revised (ECERS-R; Harms, Clifford & Cryer, 1998). The ECERS-R rates classroom quality on a 7-point Likert scale, indicating a range of quality from inadequate (1) to excellent (7). The seven ECERS-R subscales are as follows: Space and Furnishings, Personal Care Routines, Language-Reasoning, Activities, Interaction, Program Structure, and Parents and Staff.

Across the original 30 Abbott districts, average classroom quality scores have risen from 3.96 to 4.77 over the past three years, a 20 percent increase over 2003 levels. Figure 2 shows results for all 30 Abbott districts for the three years of ELIC data collection.

Figure 2. Average Total ECERS-R Scores [30 districts]

Figure 2. Average Total ECERS-R Scores [30 districts]

In 2005, average scores for each of the seven ECERS-R subscales increased over previous levels. Figure 3 indicates the change in total ECERS-R scores and ECERS-R subscale scores in a subsample of 14 Abbott districts in which classroom observations were conducted since the 1999/2000 school year, the beginning of the Abbott effort.

Figure 3. Change in ECERS-R Scores, SY2000 – SY2005 [14 districts]

Figure 3. Change in ECERS-R Scores, SY2000 – SY2005 [14 districts]

For easier reading, the following chart lists the yearly average scores displayed in Figure 3 above. Scores are listed for each ECERS-R subscale and for the total ECERS-R scale on the subsample of 14 Abbott districts in which classroom observations were conducted since the 1999/2000 school year. Thus, these averages are slightly different from Figure 2.

 

SY2000

SY2003

SY2004

SY2005

Total ECERS-R

3.86

3.92

4.31

4.75

Space and Furnishings

3.73

3.70

4.00

4.55

Personal Care

3.98

3.71

4.20

4.45

Language and Reasoning

3.74

4.16

4.57

4.93

Activities

3.19

3.37

3.64

4.13

Interactions

4.47

4.83

5.44

5.97

Program Structure

3.81

4.01

4.67

4.94

Parents and Staff

4.59

4.34

4.63

5.12

Figure 4 below shows the distribution of classrooms over the range of the ECERS-R scale for 2005. While 2.5 percent of classrooms score in the inadequate to minimal range, nearly 58 percent score in the adequate to good range and nearly 40 percent score in the good to excellent range.

Figure 4. Total ECERS-R Score –
Percentage of Classrooms Scoring 1-7 in 2005 (30 districts)

Figure 4. Total ECERS-R Score –  Percentage of Classrooms Scoring 1-7 in 2005 (30 districts)

To understand the improvement in Abbott preschool classrooms over the past three years, Figure 5 displays the change in the percentage of classrooms scoring 1 through 7 since SY2003. The percentage of classrooms scoring in the inadequate to minimal range has dropped from over 17 percent in 2003 to 2.5 percent in 2005, while the percentage of classrooms scoring in the good to excellent range has increased from about 13 percent in 2003 to nearly 40 percent in 2005. Thus, in 2002-2003, over 50 percent of the classrooms scored below the midpoint on the seven point scale compared to only 15 percent in 2003-2004. In the current school year 85 percent of the classrooms score above the midpoint.

Figure 5. Percentage of Classrooms Scoring 1 – 7, SY2003 – SY2005 (30 districts)

Figure 5. Percentage of Classrooms Scoring 1  7, SY2003  SY2005 (30 districts)

Teacher Process versus Physical Classroom Environment

To better understand the influence of factors that are in the control of the classroom teacher versus factors that are not in the control of the classroom teacher on classroom quality scores, ECERS-R items directly related to the physical environment of the classroom were included in "environment factor" while items directly related to teacher process were included in "teacher process factor."

Environment factor items measure the extent to which the classroom space is large enough for the number of children enrolled, is in good repair, has enough furniture for everyone, has outdoor space that is safe and accessible to the children, and has enough safe and appropriate playground equipment.

Teacher process factor items measure the extent to which the teacher’s interactions, methods of discipline, and use of language with the children is supportive of children’s development, along with the extent to which the teacher uses the classroom space, materials and activities to support children’s development during the day.

Results indicate that while both factors have improved since SY2000, teacher process scores have improved 29 percent over SY2000 levels while environment scores have improved 16 percent. See Figure 6 below.

Figure 6. Physical Environment and Teacher Process Factors, SY2000 – SY2005 (14 districts)
Support for Children’s Early Literacy Development

Figure 6. Physical Environment and Teacher Process Factors, SY2000 – SY2005 (14 districts)

The extent to which classroom materials, activities and interactions support children’s early literacy development is measured by the Support for Early Literacy Assessment (SELA; Smith, Davidson & Weisenfeld, 2001). The SELA includes six subscales – the Literate Environment, Language Development, Knowledge of Print/Book Concepts, Phonological Awareness, Letters and Words, and Parent Involvement. One additional item measures the extent to which the classroom environment supports language development for children who speak a language other than English. Each item is scored on a scale of 1 to 5 with "1" representing very low quality and "5" representing high quality, or the ideal.

The average total SELA score across the 30 Abbott districts in 2005 is 3.42, a 20 percent increase over the 2003 average of 2.86. Figure 7 indicates the change in total SELA scores and SELA subscale scores in a subsample of 14 Abbott districts in which classroom observations were conducted since the 2001/2002 school year, the first year the SELA was used during preschool classroom observations.

Figure 7. Change in SELA Scores SY2002 – SY2005 [14 districts]

Figure 7. Change in SELA Scores SY2002 – SY2005 [14 districts]

The following chart lists the yearly average scores displayed in Figure 7 above. Scores are listed for each SELA subscale and for the total SELA scale on the subsample of 14 Abbott districts in which classroom observations were conducted since the 2001/2002 school year.

 

SY2002

SY2003

SY2004

SY2005

Total SELA

2.43

2.86

3.11

3.36

Literate Environment

2.69

3.15

3.39

3.76

Language Development

2.76

3.16

3.30

3.56

Print/Books

2.16

2.82

3.08

3.10

Phonological Awareness

1.65

1.99

2.23

2.39

Letters and Words

2.06

2.65

3.05

3.14

Parent Involvement

2.04

2.39

2.70

2.91

Bilingual Support

2.08

2.35

2.61

3.05

Figure 8 below shows the distribution of classrooms over the range of the SELA scale for 2005. Although 25 percent of classrooms score in the low to mediocre quality range, over 50 percent are in the adequate to good range and over 24 percent score in the good to ideal range.

Figure 8. Total SELA Score –
Percentage of Classrooms Scoring 1-5 in 2005 [30 districts]

Figure 8. Total SELA Score – Percentage of Classrooms Scoring 1-5 in 2005 [30 districts]

Improvement in the distribution of classrooms on the SELA over the past three years is shown in Figure 9 below, which displays the change in the percentage of classrooms scoring 1 through 5 since SY2003. The percentage of classrooms scoring in the very low quality range has dropped from 12 percent in 2003 to 2 percent in 2005, while the percentage of classrooms scoring in the good to ideal range has increased from 10 percent in 2003 to 24 percent in 2005. In 2002 – 2003, 83 percent of the classrooms scored below the midpoint on this scale. In the current school year a dramatic shift shows that 75 percent scored above the midpoint.

Figure 9. Percentage of Classrooms Scoring 1 – 5, SY2003 – SY2005 [30 districts]
Support for Children’s Early Mathematical Development

igure 9. Percentage of Classrooms Scoring 1 – 5, SY2003 – SY2005 [30 districts] Support for Children’s Early Mathematical Development

Preschool classroom support for children’s early mathematical skill development is measured by the Preschool Classroom Mathematics Inventory (PCMI; Frede, Dessewffy, Hornbeck & Worth, 2001). The PCMI measures the materials and methods used in preschool classrooms to support and enhance children’s mathematical skills on a scale of 1 through 5. There are two subscales – Materials, and Numeracy and Other Mathematical Concepts. The Materials subscale measures the extent to which materials to explore mathematical concepts are available for children’s use. The Numeracy and Other Mathematical Concepts subscale measures the extent to which teachers support children’s use of mathematical materials and provide the interactions necessary for children to become familiar with mathematical concepts.

The average score on the total PCMI rose from 2.16 to 2.47 from SY2004 to SY2005 across the 30 districts. The distribution of classrooms scoring 1 through 5 also shows improvement over the past two years. Figure 10 indicates the change in the percentage of classrooms scoring 1 through 5. The percentage of classrooms scoring in the inadequate range dropped from 43 percent in 2004 to 24 percent in 2005, while the percentage of classrooms scoring in the fair to good range increased from 56 percent in 2004 to 75 percent in 2005. Although the percentage of classrooms scoring in the good to ideal range doubled, the percentage remains very small moving from 0.6 percent in 2004 to 1.2 percent in 2005. However, the percentage scoring above the midpoint last year was only 13 percent and has risen to 22 percent for this year. See Figure 10 below.

Figure 10. Total PCMI Score – Percentage of Classrooms Scoring 1 – 5, SY2004 – SY2005 (30 districts)

Figure 10. Total PCMI Score – Percentage of Classrooms Scoring 1 – 5, SY2004 – SY2005 (30 districts)

Figure 11 indicates the change in average PCMI Materials and Numeracy and Other Mathematical Concepts subscale scores over the past two years. While both average subscale scores have improved, the average Materials subscale score was initially higher and improved more than the average Numeracy and Other Mathematical Concepts subscale score.

Figure 11. Change in PCMI Subscale Scores, SY2004-2005 (30 districts)
The Effects of Abbott Preschool on Children’s Academic Skills

Figure 11. Change in PCMI Subscale Scores, SY2004-2005 (30 districts) The Effects of Abbott Preschool on Children’s Academic Skills

Over the first five years of the Abbott preschool program, the research methodology used to measure classroom quality and child outcomes was adequate to answer important questions about the status of classrooms and children in the Abbott districts. This year, however, a research design was implemented that provides a stronger, more direct means of measuring the effects of the Abbott preschool program on entering kindergartners’ academic and social skills. The new research design employs a regression-discontinuity statistical design and has recently been used successfully to measure the impact of preschool programs on kindergartners’ skills in other states. This statistical design allows for the comparison of preschool and kindergarten children as if they were one same-age cohort, distributing their age ranges around the birth date cut-off for kindergarten enrollment. For example, in a district with an age cut-off of October 1, a child who turns five on September 29 will be in kindergarten while a child who turns five on October 1 will be in preschool. Their difference in age may be only hours. Selection bias is minimized, as both children have parents who enrolled them in preschool. Thus, the younger child is the comparison for the older child. When, early in the school year, a comparison is made between the kindergartners who attended a preschool program with the preschoolers who have just started the program, the difference between the groups can be attributed to the effects of the preschool program on the kindergartners. Preliminary findings on the effects of Abbott preschool on kindergartners’ academic skills are presented below.

The study found effects of the preschool program on measures of language and literacy skills, but not math skills. Figure 12 displays the statistically significant (p < .04) effect of preschool attendance on average receptive vocabulary scores as measured by the the Peabody Picture Vocabulary Test-III (PPVT-3) (Dunn & Dunn, 1997) and the Test de Vocabulario en Imágenes Peabody (TVIP) (Dunn, Padilla, Lugo & Dunn, 1986) for Spanish-speaking children. This four-point difference represents a difference of nearly four months in vocabulary development.

Early literacy skills were measured with the Print Awareness subtest of the Preschool - Comprehensive Test of Phonological and Print Processing (Pre-CTOPPP; Lonigan, Wagner, Torgeson & Rashotte, n.p.). Although not yet published, the Pre-CTOPPP has been extensively validated on middle-class and low-income samples. The Print Awareness subtest measures children’s ability to distinguish words and letters from pictures, and measures the extent to which children know that letters have distinct names, shapes and sound associations. As shown in Figure 13, the study found a highly statistically significant (p < .000) effect on Print Awareness scores. Kindergartners who attended an Abbott preschool program answered 76 percent of the items on the test correctly, while children who have not had preschool answered only 44 percent correctly.

The study found no statistically significant effect of the Abbott preschool program on children’s mathematical skill development. Mathematical skills were measured with the Woodcock-Johnson Tests of Achievement, 3rd Edition (Woodcock, McGrew & Mather, 2001), Subtest 10, Applied Problems.

Figure 12. Effects of Abbott Preschool on Entering Kindergartners’
Oral Language Skills

Figure 12. Effects of Abbott Preschool on Entering Kindergartners’ Oral Language Skills

Figure 13. Effects of Abbott Preschool on Entering Kindergartners’
Early Literacy Skills

Figure 13. Effects of Abbott Preschool on Entering Kindergartners’ Early Literacy Skills

More Giant Steps to Come

Data collected over six years show sustained and dramatic improvement in the quality of Abbott preschool education and indicates substantial effects for the preschool program for children’s learning. As the largest gains in classroom quality were for the current school year, we expect that the impact for the four-year-old children will be the strongest yet when they enter kindergarten in the fall of 2005.

The OECE’s process of setting standards for child learning and program practice, measuring attainment of the standards and then using the results for program improvement and teaching is a critical factor in the progress we have made. Progress can also be attributed to increased participation rates and the increased expertise of teachers, center and school administrators and others gained through experience over the six years of the Abbott program.

Great efforts have been made at the state and local levels to complement that experience with professional development. Ninety-nine percent of the teachers will have earned a bachelor’s degree by the fall with the financial and technical assistance of the Department of Human Services Professional Development Center. DOE efforts to ensure competence in preschool education have resulted in over 200 district master teachers receiving the status of Early Childhood Professional Development Fellow. All contracting child care center directors have completed the required Directors Academy sponsored by the Department of Human Services.

Other professional development activities sponsored by the DOE include intensive seminars in early language and literacy, appropriate and useful assessment practices, inclusion of children with disabilities, bilingual education, children with challenging behaviors, math and science practices and choosing and implementing effective curriculum models.

Our efforts have focused more heavily on oral language and early literacy since oral language is the base of almost all academic learning. It is not surprising that the program is more successful in those domains than in mathematics. This year our study shows that teachers are increasing their efforts in math. This is likely in part a reaction to learning last year’s results. The DOE and the districts have intensified training in mathematics and anticipate seeing better results in future years.

Using a rigorous, scientifically sound method of assessing the Abbott preschool program, this study shows that Abbott preschool children are enjoying a nurturing, safe and intellectually challenging experience. They have increasingly qualified teachers who are supported by expert classroom consultants and administrators. These experiences are improving their chances of school success and helping to close the achievement gap by providing a giant step as they stride through the starting gate.