Overview of Activities

Below are short descriptions of each activity taking place under this grant. More information on select activities is available by clicking on the "read more" link after the activity summary.

﻿﻿Critical review of informal science education (ISE) summative evaluation reports﻿﻿SK Partners examined methodological characteristics of summative evaluations in informal science education (ISE), asking: What are the major types of designs used in summative evaluations, and what kinds of questions can they answer? What are the types of data collection methods and measures used, and how many are self-reports or direct measures? We analyzed all of the summative evaluation reports published in 2012 on the Center for Advancement of Informal Science Education’s (CAISE) website--http://www.informalscience.org, a central online resource of the ISE community. For more information about this work, please see SKP's Room for Rigor: Designs and methods in informal science education (2016). ﻿Read More>>﻿

﻿﻿Case studies of exemplary evaluations﻿﻿To gain a more nuanced understanding of the characteristics of high-quality summative evaluations, SK Partners examined a few projects more closely. Identified through purposive snowball sampling, we used a rubric to review approximately 20 summative evaluation reports with evidence of exemplary characteristics. Based on the results of that review, as well as other considerations (e.g., geographic location), we selected a summative evaluation of a children’s museum exhibition for a case study. Case study data included semi-structured interviews, as well as review of related materials (e.g., reports, proposals, articles, marketing materials, websites), and observations of the exhibition. In addition, we conducted “mini” case studies of three ISE evaluation/research projects that employed randomized experimental designs, which are challenging to carry out in informal settings. Read More>>

﻿﻿Interviews with leading individuals in ISE evaluation-related fields﻿﻿As part of SK Partners' efforts to understand current evaluation issues in informal science education (ISE), we conducted interviews with leaders in the field. We purposely selected a sample of individuals who could provide insights from a range of perspectives; collectively, they have experience with ISE and ISE evaluation as practitioners, evaluators, researchers, funders, and institutional leaders. Topics in these semi-structured interviews included but were not limited to participants’ views on evaluation best practices, challenges for the field, and issues related to the framework’s dimensions. Several participants generously agreed to share the transcripts from their interviews. Read More>>

Exploration of innovative measures of informal learningAs part of SK Partners’ work on enhancing evaluation in informal science education (ISE), we collected a variety of measures, assessments, tools, and techniques for capturing ISE outcomes. We were particularly interested in “innovative,” rigorous assessments that collected multiple data points, did not rely solely on self-report, and suggested new directions while addressing well-known constraints (e.g., free choice) in ISE.Moreover, we sought measures developed for particular purposes that could be adapted for use in different contexts. Finally, we tried to find examples that targeted a variety of common IS outcome types including engagement or interest; cognitive (e.g. content knowledge, skills); behavioral; intrapersonal (e.g., identity, self-efficacy); interpersonal (e.g., social interactions); and demographic (e.g., number of people served, visitor characteristics). Because we were interested in measures that are suitable for informal learning environments, we made special effort to find those that were unobtrusive or embedded in the visitor or learner’s experience.

Concrete exemplars of the framework in practice Through collaborations with informal science providers who are planning, designing, and/or conducting summative evaluations, SK Partners put the framework into practice. The activity was mutually beneficial: we benefited from testing ideas on the efficacy and applicability of the framework in messy, real-world contexts, and providers benefited from access to just-in-time technical evaluation support and training. Read More>>

﻿Research and conceptualization of alternative training and support models ﻿Despite a wealth of available resources such as books, workshops, tutorials, and toolkits, summative evaluation in ISE still presents a challenge. To build capacity, SK Partners believed efforts should include “in-service” and “pre-service” training and education to meet the needs of those currently working in the field as well as those who will do so in future. We researched and conceptualized promising models such as extended courses of study and formal training programs, apprenticeships, and just-in-time technical support structures.