The establishment of a community net is essential for program success. If building a closely net community network was required, our recommendation would be to utilize the services of University Outreach Partnerships (517 432-3500) for assistance in strength's based community network building. According to Foster-Fishman et al., (1999) community networked service delivery systems are most effective if they have an interagency collaboration with a seamless delivery system that is functionally integrated and minimizes organizational boundaries allowing clients to move freely between agencies and facilitates sharing of information between service providers; that is a strengths-based, family-centered and that emphasizes consumer competencies and builds on existing strengths. These "principles" are consistent with the ADS and Outreach Asset Impact Model (OAIM) approaches to outcome based evaluation (Fitzgerald et al., 1999), and they reflect the more general literature on evaluation strategies for the 21st century (Chelimsky & Shadish, 1997; Fetterman, Kaftarian, & Wandersman, 1996; Patton, 1997). The first step in an ADS-community partnership is to establish a shared mission statement that details what the partnership plans to accomplish and the responsibilities for all stakeholders. Then a work plan is developed and adopted as an action plan. The work plan provides the details of the evaluation design and time line for accomplishing key components of the evaluation, including provision for continuous quality improvement.
Although
many descriptions of desired outcomes for
child development during the birth to five
age period have been offered, within the context
of early child development program, the National
Research Council/Institute of Medicine (Shonkoff
& Phillips, 2000) summarized them concisely;
such programs must demonstrate improved performance
among enrolled children, and they must demonstrate
reductions in the disparities among children
that are observed at school entry. This is
most often expressed in the context of school
readiness as a desired outcome. School readiness
refers to "A fixed standard of development
sufficient to enable children to fulfill school
requirements and to absorb curriculum content"
(Kagan, 1994, cited in Phillips & Love,
1997, page 127).
School readiness as a comprehensive concept
for the birth to five age group is especially
appropriate because it is a multi-faceted
concept applied to an age period when multiple
systems are developing and integrating rapidly.
According to Phillips & Lowe (1997) school
readiness is operationally defined the development
of a wide range of skills including:
Motor
Skills
Balance and motion
Fine motor skill activities
Gross motor skill activities
Language Skills
Receptive language skills
Expressive language skills
Age appropriate writing skills
Age appropriate reading strategies:
story
telling
letter
recognition
story
listening
word
and movement songs
Arithmetic Skills
Classifying and sequencing objects
Identification of patterns
Understanding numbers
Time concepts
Science Skills
Exploration of environment: expectancies (hypotheses)
Life sciences (care for plants, visit zoos,
etc.)
Physical sciences (solids, liquids, machines)
Earth science (seasons, rocks, soil, stars)
Health science (nutrition, exercise, safety)
Creativity skills
Movement, music
Painting, drawing
Self concept and positive reinforcement
Arts and artists
Social and Emotional Skills (Self regulation)
Self help skills
Social awareness
Supportive classroom
Self-discipline and rule following
There also is increasing evidence to indicate that certain early experiences can either enhance or diminish school readiness, depending upon the quality of the child's early care experiences (Shonkoff & Phillips, 2000). Such indicators of school readiness include those identified by Phillips & Love, 1997): Exposure to Reading at Home, Exposure to Prenumeracy Experiences, Parental Attitudes and Expectancies, Access to Instruction in Native Language, Approaches to Learning, Emergent Literacy and Numeracy Development, and Proportion of Kindergartners "unready" for kindergarten. When teachers are surveyed they place greater emphasis on such issues as health, nutrition, communication skills, curiosity, and attentiveness (self-regulation) Carnegie Foundation, 1991).
We have combined these into five impact areas, each of which must be taken into account when assessing community based preventive/intervention programs targeting the 0-5 population.
FOCUS ON INDIVIDUAL CHILD HEALTH AND SCHOOL READINESS
Table
1 summarizes an assessment schedule for the
child health and school readiness outcomes.
Each of the indicators is associated with
established assessment tools which are administered
according to a predetermined schedule. The
time line in the table assumes recruitment
into the program will occur at birth or before.
Because some infants and toddlers may be recruited
after the newborn period, there will be deviations
from the time line indicated in Table 1. Within
limits, these deviations will be handled statistically
to maximize sample size. Table 2 describes
the practical tasks of data collection, source
of data, and scoring approaches prior to data
base entry. One strength of this evaluation
data set is that it is not wholly dependent
upon parent report for child outcomes. For
home-based programs there are two major sources
of data: parents and home visitors, and for
center based programs there are parents and
teachers. Combined home-center programs have
three sources of data, parents, home-visitors,
and teachers. Center-home combined programs
have an additional advantage in that children
can be observed in at least two key settings,
home and the supplemental child development
setting/school. With respect to the key outcome,
school readiness is operationally defined
as:
| School Readiness | |
|
Language
skills (literacy)
|
Hakuta
&D'Andrea (1992); Snow (1983); National Educational Goals Panel (1993) |
|
Arithmetic
skills (numeracy)
|
Case & Griffin (1990); Siegler & Robinson (1982) |
|
Fine
and Gross motor skills
|
Knobloch & Pasamanick (1974) |
|
Social
and Emotional skills
|
Izard & Malatesta (1987) |
| Overall child health | Phillips & Love (1997) |
| Family stability | Loukas et al. (1999); Petzold (1999) |
| Parental Attitudes and Expectancies |
FOCUS ON FAMILY AND COMMUNITY: CONTEXTUAL INDICATORS
As recently recognized by the National Academy of Sciences report on the status of the science of early childhood development (Shonkoff & Phillips, 2000), scientific study of the factors that affect child development must take into account the broader contexts within which development takes place. This is fundamentally as systems framework for assessing child outcomes. Table 3 illustrates a strategy for assessing county wide indicators (shown as Reduction in Special Education Placements and Family Stability). When random assignment is not possible, we use historic population demographics to establish baseline prevalence rates over the past 5 years for as many indicators as possible. If over time, the sample size becomes sufficiently large, we use such statistical procedures as survival analysis to track changes in indicators as a function of identified events (e.g., date of onset of the prevention program). Although such approaches may not allow causal inferences, they can provide strong correlational support for program effectiveness (Shonkoff & Phillips, 2000). We specifically track the number of special education placements as well as indicators of family stability (divorce, family resources, number of times moved, family density, etc.).
FOCUS ON PROGRAM
One key indicator of program effectiveness is the extent to which the community action plan is implemented. Therefore, we conduct semi-annual focus groups and annual structured surveys to assess the extent to which the community network is effective in enhancing within system referrals, community involvement and networking, and resource sharing. When community networks include a focal telephone contact entry point, we also monitor telephone requests to determine whether requests were responded to promptly and if a referral was made, whether follow-up occurred.
FOCUS ON SUSTAINABILITY
A key to successful prevention programs is the extent to which communities can sustain programs after extramural funding no longer is available. This is a personnel resource issue. A second key involves the extent to which technology transfer has taken place so that individuals at the local program level gain the knowledge resources needed in order to sustain program innovations. It is the latter issue that is directly addressed by the ADS and OAIM approaches to evaluation. We establish identical data bases in the local area and at Michigan State University and we train individuals at the local level to continue data entry and retrieval so that evaluation can be an ongoing component of community programming.
FOCUS ON IMPLEMENTATION: HOME VISITORS,
TEACHERS and PARENTS
Finally, we will conduct independent focus
groups annually with teachers or home visitors
and parents to assure that their assessments
of program effectiveness is an integral part
of the overall evaluation. Moreover, we assess
parental characteristics that may have an
impact on child development so that parental
influences and program influences as be independently
assessed and integratively analyzed. This
five-pronged evaluation plan will generate
a rich multilevel systemic data base from
which both proximal and historical causal
factors can be tracked and linked to ongoing
program outcomes and program improvements.
The evaluation plan adopted for any specific
partner, however, is a joint venture between
the community partner and ADS personnel. This
assures that the design of the evaluation
for a community based program is as much the
product of the community members as it is
that of the evaluator. This assures that evaluation
takes into account local diversity, local
objectives, and local programs goals.
References
Brown, R., & Reed, C. S. (1998/1999). Evaluating services by linking outcome-based and asset-oriented approaches. Best Practices Brief #5. Michigan State University, East Lansing, MI: University Outreach Partnerships.
Case, R., & Griffin, S. (1990). Child cognitive development: The role of central conceptual structures in the development of scientific and social thought. In C. A. Hauert (Ed). Developmental psychology: cognitive, perceptual-motor, and psychological perspectives. Amsterdam, The Netherlands: Elsevier.
Chelimsky, E., & Shadish, W. R. (Eds) (1997). Evaluation for the 21st century. Thousand Oaks, CA: Sage.
Fetterman, D. M., Kaftarian, S. J., & Wandersman, A. (Eds). (1996). Empowerment evaluation: Knowledge and tools for self-assessment and accountability. Thousand Oaks, CA: Sage.
Fitzgerald, H. E., Abrams, L. A., Church, R. L., Votruba, J. C., & Imig, G. L. (1996). Applied developmental science at Michigan State University: Connecting university and community via programs for children, youth, and families. Journal of Research on Adolescence, 6, 55-66.
Fitzgerald, H. E., Abrams, L. A., Andrews, M., Villarruel, F., Brown, R., & Reed, C. S. (1999). CHECKPOINTS: Building capacity to enhance program impact through evaluation. In T. R. Chibucos & R. M. Lerner (Eds). Serving children and families through community-university partnerships: Success stories. (Pps 197-203). Boston, MA: Kluwer Academic Publishers.
Foster-Fishman, P. G., Salem, D. A., Allen, D. A., & Fahrbach, K. (1999). Facilitating interorganizational exchanges: The contributions of interorganizational alliances. American Journal of Community Psychology.
Hakuta, K., & D'Andrea, D. (1992). Some properties of bilingual maintenace and loss in Mexican background high-school students. Applied Linguistics 13, 72-99.
Izard, C. E., & Malatesta, C. Z. (1987). Perspectives on emotional development I: Differential emotions theory of early emotional development. In J. D. Osofsky (Ed). Handbook of infant development. New York: Wiley Interscience.
Knobloch, H., & Pasamanick, B. (1974). Gesell and Amatruda's developmental diagnosis: the evaluation and management of normal and abnormal neuropsychologic development in infancy and early childhood (3rd ed). New York: Harper and Row.
Loukas, A., Twitchell, G. R., Piejak, L. A., Fitzgerald, H. E., & Zucker, R. A. (1999). The family as a unit of interacting personalities. In L. L'Abate (Ed). Family psychopathology: The relational roots of dysfunctional behavior. New York: Guilford.
National Education Goals Panel (1993). Reconsidering
children's early development and learning:
Toward shared beliefs and vocabulary. Draft
report to the National Education Goals Panel
form the Goal 1 Technical Planning Group.
Washington, DC: National education Goals Panel.
Patton, M. Q. (1997). Utilization-focused evaluation. Thousand Oaks, CA: Sage.
Petzold, M. (1999). The concept of the family in family psychology. In L. L'Abate (Ed). Family psychopathology: The relational roots of dysfunctional behavior. New York: Guilford
Phillips, D. A. & Love, J. (1997). Indicators for school readiness, schooling, and child care in early to middle childhood. In R. M. Hauser, B. V. Brown, W. R. Prosser (Eds). Indicators of children's well being (pps 125-151).
Shonkoff, J., & Phillips, D. A (Eds.). (2000). From neurons to neighborhoods. Washington, DC: National Academy of Sciences.
Siegler, R. S.., & Robinson, M. (1982). The development of numerical understandings. In H. W. Reese & L. P. Lipsitt (eds). Advances in child development and behavior. New York: Academic Press.
Snow,
C. E. (1983). Literacy and language. Harvard
Educational Review, 53; 165-189.
