2 Good Outcomes Begin with Good Assessment!What are Outcomes?Outcomes are the benefits that children and families experience as a result of early interventionThe three child outcomes, as measured for accountability purposes are different than IFSP outcomes in that:OSEP’s three child outcomes for accountability apply to all children and reflect global functioning in broad areas of development (social-emotional, knowledge and skills, getting needs met)IFSP outcomes are specific to an individual child, based on his or her individual needs

3 What is Assessment?“Early childhood assessment is a flexible, collaborative decision-making process in which teams of parents and professionals repeatedly revise their judgments and reach consensus about the changing developmental, educational, medical and mental health services needs of young children and their families.”(Bagnato and Neisworth, 1991)Quoted in DEC Recommended Practices, 2005

4 Assessment is…“Assessment is a generic term that refers to the process of gathering information for decision-making.”Mary McLean, 2004

5 Purposes of AssessmentIn the early intervention systemassessment serves several purposes:Identification of children with special needs (including screening, diagnosis, and determination of eligibility for services)Planning instruction or interventionMonitoring of the child’s level of knowledge or progressProgram evaluation and accountability

6 Accountability???Accountability involves embedding ongoing data collection into program practices because the need to examine the effectiveness of the services being provided is ongoingGiven the number of factors that influence the provision of services (e.g., staff changes, reimbursement formulas), it is not reasonable to assume that a specific program that was effective at one point in time will always be effective

7 Assessment for AccountabilityPolicymakers and funders are raising legitimate questions about which programs are effective as they struggle with how to allocate resourcesThe Government Performance and Results Act (GPRA) means Part C Programs need to provide evidence that program participants demonstrate positive outcomesAn important issue for assessment is that the Office for Special Education Programs (OSEP) decided not to require states to use the same assessment tool. Each state is free to chose their own

8 Assessment as a ProcessIt is important to distinguish betweenassessment as a process andassessment as a single instrument or test!Assessment as a process involves multiple ways of collecting information about a child’s performance, including, for example, observation, documentation of the child’s work, checklists, rating scales, portfolios and norm-referenced or curriculum based tests

9 Variation in What Tools are UsedSo far these are the tools we know thatstates are using for accountability:Of 28 states who listed specific assessment instruments:HELP-15 statesBDI/BDI-2-13 statesAEPS-11 statesCreative Curriculum-6 statesELAP-6 statesNot yet determined-23 states

10 Variation in How Assessments Have Been UsedIn the past, there has been much variationin assessment procedures in earlyintervention programs across the country:In some programs, the only formal assessment has been for eligibilityOther states completed assessment annuallySome states have always completed assessment at exit

11 Views on Assessment There have been major changes in last 15years in how assessment of youngchildren is viewed by the early childhoodcommunity:Old position: Do not test little kidsNew position: Ongoing assessment is part of a high quality early childhood program

12 New Thoughts on Assessment“The best way to understand the development of children is to observe their behavior in natural settings while they are interacting with familiar adults over prolonged periods of time”--Uri Bronfenbrenner

13 What Changed our ThinkingThe purpose of assessment was redefinedin the last decade:It’s no longer about sorting, labeling, using to deny access to servicesIt’s now about getting a rich picture of what children can do and can’t do and using that information to help them acquire new skillsIt’s now about monitoring progressNew and different tools became available for generalearly childhood: Curriculum-based assessments were developed, e.g., Creative Curriculum, Work Sampling,etc.

14 What Changed Our ThinkingAssessment had always been seen as a process with multiple purposesDistinctions have been made between good and bad uses of assessment with young childrenGood assessment practices are now widely recognized as a key component of high-quality early childhood programsGood uses are now promoted!For more information: National Association for Education of Young Children (NAEYC) web site (Position statement on Curriculum, Assessment and Evaluation)

15 NAEYC Position StatementPolicymakers, early childhood professionals and otherstakeholders in young children’s lives have a sharedresponsibility to:Make ethical, appropriate, valid and reliable assessment a central part of all early childhood programs.To assess young children’s strengths, progress,and needs, use assessment methods that aredevelopmentally appropriateculturally and linguistically responsivetied to children’s daily activitiessupported by professional developmentinclusive of families

17 DEC Recommended Practices for AssessmentThe Recommended Practices emerged fromfocus groups and are supported by literaturefrom the Division of Early Childhood (DEC):Parents and professionals who participated in these focus groups identified repeatedly two concernsPrinciple stakeholders, parents and family members must play a vital and indispensible role in assessment from beginning to endAssessment methods and materials must accommodated children’s developmental and disability specific characteristics

18 DEC Recommended Practices for AssessmentThe Recommended Practices are organizedaround five statements:Professionals and families collaborate in planning and implementing assessmentAssessment is individualized and appropriate for the child and familyAssessment provides useful information for interventionProfessionals share information in respectful and useful waysProfessionals meet legal and procedural requirements and Recommended Practice guidelines

19 Assessment for AccountabilityThe assessment process we use to determine eligibilityand program planning can be used for accountability!The three child outcomes:Children have positive social-emotional skills (including social relationships)Children acquire and use knowledge and skills (including early language/communication [and early literacy])Children use appropriate behaviors to meet their needsAssessment tools can inform us about children’s functioning in each of the three outcome areasChallenge: There is no assessment tool that assesses the three outcomes directly

20 Additional ChallengesYoung children are unreliable test takersBecause of their limited ability to understand and follow the directions of the testThey may not be able to control their behaviorThey have short attention spansAnd are easily distractedWhat young children demonstrate in a test situation may not be an accurate representation of what they are capable of doing on a daily basis

21 Types of Tests Standardized vs curriculum basedBy definition, standardized tests must be administered the same way to all test takers.The use of a consistent approach during test administration enhances reliability between administrators and confidence that children are responding to the same tasks.Standardized tests usually provide normative data to compare the child’s functioning with that of other children of the same age.

23 Curriculum-based AssessmentsBagnato and Neisworth (1991) define curriculum-based assessment as a “form of criterion-referenced assessment wherein curricular objectives act as the criteria for the identification of instructional targets and for the assessment of status and progress”Some of the newer assessment tools emphasize critical skills to facilitate the development of fuctional behavior

24 Currently Available Assessment ToolsMost assessment tools are organized around developmental domains and are based on sequences of skills that build on each otherThis organization may pose difficulties for accurate assessments of functioning in young children with disabilities because delayed skills in one domain can influence assessment of or functioning in other domainsEach assessment tool sees children through its own lensEach lens is slightly differentLens are not right or wrong

25 Variations Among StatesStates are continuing to make key decisions related to their accountability systems, so any information about state choices must be seen as time-limited and interpreted cautiouslyIn many state, some assessment-related decisions are being made by local programs, so there is variation within the statesThe information available thus far suggests that much of the data on child outcomes will be collected with curriculum-based tools

26 Variations Among StatesMost states are building their accountability systems around assessments that were already being administered for other purposes (such as eligibility determination or program planning)Few have adopted a new statewide measure solely for collecting data for accountability purposes

27 Maryland’s Top FiveFollowing a survey distributed to all local Program Directors in the state, Maryland state Department of Education (MSDE) identified the 5 most commonly used assessment instruments in Maryland.Results from at least one of these assessment instruments (or other instrument if more appropriate), in conjunction with information obtained from other possible sources, including record review, interview, observation, and screening, will be used in providing the information required to complete the Child Outcomes Summary Form (COSF).Ideally, the same tool would be used at entry and exit. However, use the tool that is most appropriate for the child.

29 Key Questions for Accountability:How much and what information will the tool provide about the child’s current level of functioning and the attainment of the 3 child outcomes?Would providers intentionally or unintentionally give children higher scores than warranted, knowing that conclusions about program effectiveness would be based on the information?

30 Assessment Tool LensEach assessment tool carries its own organizing frameworkMost are organized around domains but children integrate skills when performing functional activitiesNo right or wrong lens:How can what the assessment tool provides be used to develop functional outcomes for young children?How can what the assessment tool provides beused as information about the achievement ofthe three outcomes?

31 Critical Assumptions Related to the 3 Child OutcomesAchievement of the outcomes is age based, children of different ages will demonstrate achievement in different waysThere are many pathways to competence for children with atypical development (using sign language, wheelchair). This seems obvious but can get lost when an assessment tool uses a different lens.

32 Remember ThisFlexibility is required in using assessment tool results to make decisions about a child’s functioning on the outcomesTeam needs to decide what information from an assessment tool is relevant to understanding child’s functioning

33 More DEC Recommended Practices for AssessmentAssessment provides useful informationfor intervention!Assessment results are immediately useful for planning program goals and objectivesGood assessment practices assess not only immediate mastery of skill, but also whether child can demonstrate skill consistently across settings and with other peopleGood assessment practices also assess the presence and extent of atypical child behavior that may be a barrier to intervention and progress

34 Using the Assessment Results for OutcomesWhat is an Outcome:Outcomes are the benefits that children and families experience as a result of early interventionThe three child outcomes, as measured for accountability purposes are different than IFSP outcomes in that:The three child outcomes reflect global functioning in broad areas of development (social-emotional, knowledge and skills, getting needs met)IFSP outcomes are specific to an individual child, based on his or her individual needs

35 What Makes an Outcome Functional? Functional Outcomes:Let’s Look at Using Assessment Information to Develop Functional IFSP OutcomesWhat Makes an Outcome Functional?Functional Outcomes:Refer to things that aremeaningful to the child in thecontext of everyday livingRefer to an integrated seriesof behaviors of skills thatallows for the child to achievethe important everyday goals

36 Meaningful OutcomesMeaningful outcomes promote a child’s functioning in three key foundations of early development - social interaction, mastery over environment, and engagement in learning - in ways specific to each family and child(McWilliam, 2002; National Research Council, 2000).

37 Writing IFSP Family Centered, Measureable, Functional OutcomesFamily-centered outcomes are written to address:The desires and goals a family has for them and their childCreated with the family after assessing what activities are meaningful to individual family members.Includes not only activities of daily living but also the family's ability to participate in cultural and social experiences they hold valuable

38 Functional Outcomes“Function” refers to those activities identified by the family that support the development of the child’s physical, social and psychological well being, i.e. the ability to feed oneself, to hold a toy, to communicate, or to playEarly intervention teams do not “treat” the primary diagnosis, i.e. cerebral palsy, autism, but instead treat the functionaldelay or disability

39 Determining Appropriate IFSP Functional OutcomesA few questions to consider:What are the activities the family would like to do that are difficult?Has the family given up on any family outings, experiences or chores?What kinds of things would the family like for the child to do that would make life easier for them or more fun for the child?Read more at MD’s IFSP Tutorial: Assisting Families to Select Functional Outcomes

40 Measureable IFSP OutcomesOutcomes should be written so that all members of the team will understand when the outcome has been met.Outcomes should address one of the following areas:They should enhance the family’s ability to care for or to engage in activity with their childThey should enhance the child’s ability to participate in functional activities (feeding, dressing, moving in his environment, communicating, playing, etc.)They should expand on activity settings in which the child already participatesRead more at MD’s IFSP Tutorial:Measurable Criteria: How we Know we have Achieved Outcomes

41 Effective Strategies on an IFSP Identify How Functional Outcomes Will be AchievedEach strategy should address:Individualization- Do strategies build on child and family interests?Context- Do strategies build on familiar places, people, and routines?Mastery- Do strategies ensure generalization of a child’s actions and interactions in multiple settings and tasks?Collaboration- Do strategies specify who will do what?Read more at MD’s IFSP Tutorial:Effective Strategies Identify How Functional Outcomes will be Achieved

42 Making a Difference! The results of improved assessment andindividual IFSP outcomes developmentand services is two-fold:First and foremost, it ultimately leads to enhanced services for children and familiesWhich, as a result, can’t help but positively impact Maryland’s outcome measurement results!