Leverage for Digital On-Ramps

Preface

The information contained in this project appendix was gathered from the original project proposal that was funded in 2012 and interviews with project leaders through the end of 2014. This appendix and the overall design principles database from the DPD project does not reflect further evolution of the project or developments after the final interview that took place in 2014. As of the final interview, Leverage for Digital On Ramps had not yet issued badges or fully developed a badge system but were continuing the development process. Based on this information, we have classified this project as a partial (rather than an implemented or suspended) development of the badge system.

Summary

The Leverage for Digital On-Ramps badging system strives to prepare high school students, grades 10-12, in Philadelphia for entry into college and the workforce. Project leaders explained that they planned to “develop and demonstrate a badging system for 900 students (from two partner high schools) enrolled in Philadelphia Academies, Inc. (PAI) Post-Secondary and Career Readiness (PSCR) Course, a developmental multi-year course designed to provide 21st century and post-secondary readiness skills to high school students” (DML Stage 1 Proposal). A project grantee of the DML/MacArthur Badges for Lifelong Learning competition, the effort seeks to capture the accomplishments of youth that are performed offline and represent their achievements in a digital, accessible, and enduring format.

As part of a larger vision, Digital On-Ramps aims to eventually expand into a citywide consortium that offers digital badges and serves both youth and adults from ages 14 to 65. With an emphasis on workforce development, the badging initiative focuses on “providing simple, personalized, and customized “on-ramps,” or educational/training supports such as basic literacy and math skills, workforce and college readiness training, 21st century skills and digital literacy/competency training” (DML Stage 1 Proposal). The Urban Affairs Coalition (UAC), Philadelphia Academies, Inc. (PAI), and other groups have partnered together in an effort to build and implement the “Leverage™ Badging System by Pragmatic Solutions into the infrastructure of the Digital On-Ramps (DOR) project” (DML Stage 1 Proposal). With this objective in mind, project leaders planned to construct a badge system initially geared toward youth and then apply the emergent model as a template across other contexts. In this way, the initiative moves toward “preparing all Philadelphians to work and compete in the 21st century economy by providing a digital framework for delivering comprehensive education and workforce training to youth and adults” (DML Stage 1 Proposal).

The goals of Digital On-Ramps involve transitioning the PSCR course to a digital format and representing learning in a more detailed and transparent fashion. To this end, the initiative concentrated its efforts on building a digital portal and creating an online version of the course. The Digital On-Ramps platform is “a social network for badge earners who participate in PAI’s Career and College Readiness Curriculum (CCRC),” which is “a series of surveys, quizzes, live events, and other activities that help high school students develop skills and direction for their post-high school careers or college experience” (HASTAC Q&A). This provides a way to record, document, and give credit for the work that youth perform in-person and out in the field. The badging effort described, “By introducing a systematic badging system used across workforce and education systems, DOR would have the capacity to capture and preserve the effort that people put forward toward reaching their goals” (DML Stage 1 Proposal). The badging effort’s intent included the capacity to share badge credentials easily among colleges and employers. The project also highlighted that “badges would create systemic efficiencies, ensuring that people do not lose valuable time and effort if life-disrupting events cause them to start, stop, and re-start efforts to improve their lives” (DML Stage 1 Proposal). The badge system, then, enables youth to tell a broader story about their achievements.

Evolving Practices and Design Principles

What follows is a list of practices as they relate to the general and more specific design principles in each category of practice. The headings name a (a) General Principle, (b) Specific principle, (c) Specific practice. Below each heading, we indicate what stages this principle appeared in the project: as an (a) intended practice, (b) enacted practice, or (c) formalized continuing practice.

Design Principles for Recognizing Learning

Digital On-Ramps designed badges that last permanently and that students could share these with colleges and employers. The project aimed to connect with employers and demonstrate the value of badges beyond the badge system. Moreover, the system aligns badges to P21 21st Century Skills, providing a common framework for recognizing student achievement. Additionally, badges recognize a range of abilities and attributes, including “grit” and workplace skills. The badge development system also enlists teachers in reviewing student performance and issuing badges. However, as of the DPD Follow-Up Interview, the project has not continued to issue badges.

The project planned for the digital badges to have permanence. They explained that “[t]hese permanent, independently maintained, portable badges could be easily shared among prospective schools, programs, or employers” (DML Stage 1 Proposal). The credential is intended to communicate skills and knowledge gained outside of normal school activities to employers. These badges are employed as a means to keep a collection of accomplishments, and this will help participants keep track of their credentials. Ultimately, these badges are aimed to prepare youth for higher education, an internship, or a job. Permanence is particularly important with this group of learners. It is common for them to have difficulty receiving a position for which they are actually credentialed for, because it is difficult to keep track of the physical location of the document. A move to digital badges means learners will never have to go from office to office trying to track down paperwork that validates their knowledge or certifications.

Align badges to standards > Use community and national/international standards > Collaboration with accredited institutions

Intended, enacted, not formal.

Badge awarding is based on standards determined by an accredited institution. The project aligns badges to the standards of P21 21st Century Skills.

Recognize diverse learning > Badges for range of accomplishments

Intended, enacted, not formal.

In many cases badges are awarded for offline accomplishments. Badges recognize both online and offline experiences and accomplishments. In addition, the credentials recognize students for soft skills and qualities of “grit.” The project initially aimed to employ text messaging to log work that students perform in the field, where trainers would give trainees a text code for trainees to record their work and get credit for completing the activity. However, this posed a challenge, as many young people had phones but no data plans (DPD Follow-up Interview). For this reason, the project has not advanced further on this front. However, the badge structure carries out the purpose of demonstrating an array of skills and achievements attained by badge earners.

Use badges as a means of external communication > External collaboration

Intended, enacted, not formal.

Collaborations with outside entities will create a united badging system that helps track accomplishments of disadvantaged individuals, increasing the usefulness of the the credential. The badge system aimed to enable learners to earn points for activities, badges for courses or sets of activities, and then a 4C’s (meta-badges) badge for sets of courses. Only 4C’s badges will be displayed in the badge backpack. Colleges and employers are heavily involved in the design processes and want fewer badges and greater meaning. They are also calling for a standardization for what warrants a badge within the curriculum. In this sense, the credential communicates nuanced meaning to employers.

The pre-existing relationships that the Urban Affairs Coalition has with the Philadelphia Academies Inc. and the Philadelphia Youth Network will help to support the acceptance of badges outside the issuing community and assist learners in finding future learning and career opportunities. The digital badge credential is a part of a united badging system that helps track accomplishments of traditionally marginalized individuals. It explicitly affords for education and career advancement.

The badges are verified by teachers or program leaders which facilitate the experience. In this sense, the credential is validated by a professional. PIA working directly with teachers to implement this as part of their career academy model. There are two components with one out of school program iRemix system and in school component ConnectEd piece (DPD Follow-up Interview).

Design Principles for Assessing Learning in Digital Badge Systems

In addition to fostering the development of “workforce-ready skills” using standardized rubrics and points that add up to badges, Leverage for Digital On-Ramps is teaching teachers to teach and assess “grit.” The intention here is to help earners develop persistence and conscientiousness, so they may be successful in their educational and work situations. The project aims to help earners grow in their skill set and in their perseverance and work habits.

The system designed badges to promote skill and competency development through participation in projects. Digital On-Ramps explained that “[t]hrough a set of very intentional activities, students develop formative experiences in the real world, and realize the value of ongoing learning as skilled workers” (DML Stage 1 Proposal). The project has research partners interested in assessing “grit” and workforce-ready skills and hope to integrate this into the project. “Grit” is a “combination of persistence and conscientiousness” that is “necessary to be successful in education and employment. So there are conversations around how do we teach teachers to have grit and to teach grit to their kids” (DPD Initial Interview). To this goal, the project issues badges to recognize youth for a spectrum of qualities and abilities.

Use rubrics > General rubrics > Rubrics are used to assess learning

Intended, enacted, not formal.

Digital On-Ramps planned to integrate rubrics to measure learning, describing “two assessment tools: 1) A college readiness rubric that documents the post-secondary readiness skills; and 2) Philadelphia Youth Network’s (PYN) Internship assessment used by internship mentors, to support skill development in professionalism, initiative, leadership, communication, and industry awareness” (DML Stage 1 Proposal). Each activity has its own rubric. Some of these come from the state, and others come from outside agencies.

The projects envisioned the incorporation of experts to review the learning represented by badge credentials1. The badging system delineated that “…when a learner completes something they get a star with a tint not solid, but it doesn’t become official until a teacher or expert approves it but get visual recognition” (DPD Initial Interview). In the implemented badge system, the badges are still validated by experts.

Use leveled badge systems > Metabadges > Leveled badges

“[a] student would accumulate a set of badges representing the various skill levels mastered as they move through the course. These skills can be understood as a series of incremental, discrete levels that are acquired across a set of parallel learning tracks. These levels would be competency-based rather than time based with progress marked in small and flexible accomplishments measured by multiple assessments, many of which are embedded within the learning experiences” (DML Stage 1 Proposal).

The project is implementing a point system symbolized by stars. Points are relatively easy to get, and badges are hard. They elaborate that “[a] badge is a representation of a set of skills or knowledge that you have been able to demonstrate, or a process that was multi-faced. Points are things you get or giving it your best” (DPD Initial Interview). In the badge system implementation, the project has not gone forward with leveled badges.

Design Principles for Motivating Learning

This initiative serves a largely underprivileged population. Any given student in this population likely participates in a variety of educational programs, although life circumstances may make it difficult or even impossible for the student to finish. Often, students’ accomplishments in this arena are lost, leaving the student with no visible record of her accomplishment. This is why Leverage for Digital On-Ramps would like to implement digital badges; badges could serve as that visible and lasting record. Badges could also give students an advantage in the initiative’s participation in the Philadelphia Youth Network internship program, a highly-competitive, long-standing program.

Utilize different types of assessment > Expert > Include assessments by experts

Intended, enacted, not formal.

The project planned to employ different kinds of assessments to motivate badge system participants. The enlistment of experts appears to lead youth to focus their effort toward performing the activities.

Display badges to the public > Learners can choose to share their badges with others > Evidence of employability

Intended, not enacted, not formal.

The project aimed to encourage badge system users by enabling them to use digital badges to show evidence of employability. They outlined that “these permanent, independently maintained, portable badges could be easily shared among prospective schools, programs, or employers” (DML Stage 1 Proposal). Badges will motivate earners by providing increased evidence of employability.

Provide incentives > Internships > Internships

Intended, not enacted, not formal.

Badges could motivate students by providing access to career opportunities, such as paid internships. The badging system highlighted:

“A key aspect of this project is to help students identify career interests and expose them to various career pathways. Through a set of very intentional activities, students develop formative experiences in the real world, and realize the value of ongoing learning as skilled workers. Badging strengthens this experience, validating their accomplishments to educational institutions or employers, and serving as a portable, public record of accomplishments to place in their personal profiles and to notify their networks of peer learners. We intend to offer a range of ‘badge privileges,’ While we will conduct student focus groups to determine appealing incentives, our initial thoughts are: additional opportunities, like internships, on-the-job training and apprenticeships, and possibly preferential interviews; and, student access to local attractions, sports and special events. For example, students interested in science careers could receive a “Future Scientist” badges, gaining free entry into the Academy of Natural Sciences or internship opportunities at a science-based firm” (DML Stage 1 Proposal).

Badges will motivate students by providing access to internships and other professional activities. Digital On-Ramps spelled out:

“The Philadelphia Youth Network has 8,000 kids a year that apply for 1,000 internship positions. They’re paid internships, so they’re highly, highly coveted. It’s understandably difficult to decide to gets those valued internships. So, they’re hoping to provide a very clear way for students to achieve badges, and for those badges to be a prerequisite for being considered for these internships. Then the students will want to earn them…The hope is that, even if students didn’t end up getting an internship, that at least this process could be an incentive to getting them better educated…” (DPD Initial Interview).

This process is still being developed by the initiative. The project conveyed, “Ultimately, in the ‘big door’ vision, they’re hoping that there is a tremendous amount of freely available content, that anyone, even community members who are not already plugged into a service agency, that they would want to come and engage the content because they want to earn the badges because they know that employers recognize them” (DPD Initial Interview). Badges are also designed to showcase student skills for the purpose of college admissions, and they appear to motivate engagement by providing evidence of skills and experiences to potential employers and universities. The initiative explained, “The vision is to not lose any information, that anything a learner does is getting preserved” (DPD Initial Interview). Badges boost engagement by providing a permanent record of experiences and accomplishments. In the implemented badge system, the project has yet to continue development on this practice.

Design Principles for Studying Learning

Although Digital On-Ramps did not articulate in their DML proposal specific intentions to conduct research on the badge system, the project collected data on the interactions and gathered feedback to inform the design of their badges. The effort carried out a beta test of the badge system. In addition, they plan to study further the interests of employers with respect to digital badges.

Improve badge impact > Research FOR badges > Evolution of research and evaluation practices

Intended, enacted, not formal.

The project did not explicitly define any research intentions in their DML proposal, but Pragmatic Solutions wants to do something with the data; they are collecting extensive amounts of data. The initiative uses feedback from colleges and employers to define badges. Weekly surveys will collect further information that is beyond the data analytics of the site.

The project planned to map out where participants are spending time on the site and then use this information to relate to other aspects such as how many badges are earned or what type of badges are earned. The badging effort “beta tested the project with 300 students,… observed how long it would take to explain, have the students sign up and any challenges in working in the school environment” (HASTAC Q&A). Specifically, the project also intends to track job/college placement of badge earners and their specific badges (e.g. Learner with badges X,Y, and Z has a career in Forestry now). They plan to track specific career and/or further education outcomes of badge earners outside of the issuing community. They also discussed the possibility of conducting surveys of employers about what they would want to see in the electronic portfolio, with the potential of having informal sit-down conversations (DPD Follow-up Interview).

Notes

1 Digital On-Ramps claimed “… What we have had to do with every single partner – not had to, it has been a pleasure to do that, is actually have a conversation about what it means to assess learning, and what authentic learning means, and what a transfer of learning means. Because we don’t necessarily teach K-12 teachers how to assess, and we certainly don’t teach people in workforce development how to do that” (DPD Initial Interview). The initiative is teaching experts how to assess learning.

Regardless of where you start, it’s more than likely you’ll end up somewhere other than your intended destination. That’s okay. Systems are living things, and your badge system needs to be flexible. You must embrace a bit of chaos in its design.
-Carla Casilli, Director of Design + Practice at the Badge Alliance