stinjitsuhttps://stinjitsu.wordpress.com
A topnotch WordPress.com siteTue, 26 Mar 2019 00:06:17 +0000en
hourly
1 http://wordpress.com/https://s0.wp.com/i/buttonw-com.pngstinjitsuhttps://stinjitsu.wordpress.com
How ranking algorithms might be used to create competency scores for badge earnershttps://stinjitsu.wordpress.com/2019/03/19/how-ranking-algorithms-might-be-used-to-create-competency-scores-for-badge-earners/
https://stinjitsu.wordpress.com/2019/03/19/how-ranking-algorithms-might-be-used-to-create-competency-scores-for-badge-earners/#respondTue, 19 Mar 2019 12:12:26 +0000http://stinjitsu.wordpress.com/?p=81Last year, Concentric Sky announced the release of a tool it calls BadgeRank, which uses an algorithm to rank the relative value of badges using a number of criteria such as endorsements and outcomes (Harman, 2018)(PRWeb, 2018). BadgeRank is intended to be a search engine for open badges, but I think its potential uses don’t end there.

In this blog, I have advocated for a learner-centric approach to badge stacking that I call upside-down stacking. The idea is this: a learner adds a competency statement or goal to a badge pathway and then earns badges to support the claim of the competency statement or show progress toward the stated goal. It’s a flexible approach that enables learners to design their own pathways and earn as many badges and endorsements as they want to support their claims. The more badges and endorsements learners acquire, the more compelling their claims begin to look. To learn more about upside-down badge stacking you can read the blog post or view the presentation slides.

For this post, the important point about upside-down stacking is that it creates clusters of badges around competency statements, but does so in a way that is still compatible with tiered pathway designs.

BadgeRank’s algorithm, in order to actually rank badges, must use criteria to assign a weight to each badge. In a cluster of badges, the weights of the individual badges could be summed in order to determine a weight for the badge cluster. Because each badge cluster is associated with a competency statement, you get a weight associated with that competency statement.

It would take some thinking and refinement, but it’s conceivable that eventually we might be able to use the weight of a badge cluster to at least partly infer the likelihood of a learner’s claim to be developing or already possess a given competency.

The idea of an algorithm generating something like a competency score for a badge earner both excites and frightens me. I welcome your comments.

]]>https://stinjitsu.wordpress.com/2019/03/19/how-ranking-algorithms-might-be-used-to-create-competency-scores-for-badge-earners/feed/0stinjitsuGraph showing four child badges clustered beneath the parent competency. The weights of the child badges are summed in the parent competency.An Open Recognition meeting in the United Stateshttps://stinjitsu.wordpress.com/2019/01/16/an-open-recognition-meeting-in-the-united-states/
https://stinjitsu.wordpress.com/2019/01/16/an-open-recognition-meeting-in-the-united-states/#respondWed, 16 Jan 2019 16:22:39 +0000http://stinjitsu.wordpress.com/?p=79During a #BadgeChat last November, Julie Keane and Nate Otto shared their perceptions of the ePIC conference on open recognition, which they attended last year in Paris, France. I listened eagerly, all the while wishing there were opportunities to attend similar open recognition events here in the United States. As I listened, that wish for nearby events became a nagging question in my mind. “Hey, why aren’t there open recognition events in the U.S.? Or in North America? For that matter, why aren’t there more open recognition events everywhere?”

Here’s what I think we should do: hold an open recognition meeting in the United States. We could do it this June at the Badge Summit conference in Philadelphia, PA (I already asked Noah Geisel).

To my knowledge, there haven’t been any open recognition events in the United States. There have been conference presentations on the topic, and perhaps there have been events that align with open recognition but that go by different names and so fly below my Google radar. If that’s the case, please let me know; I don’t want to miss out!

There are organizations and workgroups in the U.S. that focus on micro-credentialing and open badge technologies. For instance, IMS Global hosts annual and quarterly conferences, as well as working groups that meet to hammer out technological standards. Likewise, there’s RWoT and W3C’s work on Verifiable Credentials, Decentralized Identifiers, Linked data, and other related technologies and standards. Lots of excellent work is happening —and I’m so grateful for all of it! Open technologies are absolutely necessary for supporting an open recognition ecosystem… but are they sufficient to create such an ecosystem?

Creating an open recognition ecosystem will require significant changes in the opinions and behaviors of individuals, the policies and cultures of organizations, and perhaps even support from government policymakers. I expect most folks would agree that open technologies —by themselves— may not be sufficient to make this happen. Bringing about such changes will likely require an additional kind of effort: outreach and advocacy.

I know that many people have been doing exactly that kind of outreach and advocacy for years —and again, I’m so grateful! I expect those same folks have met informally many times in the U.S. and discussed open recognition, probably while attending events organized around open badges/micro-credentialing. Why not bring those open recognition advocates together for a planned event where open recognition takes center stage, even if it’s just a meeting? It may not be an entire event like the ePIC conference, but it’s a place to start. And speaking of ePIC and the open recognition scene in Europe, the Open Recognition Alliance seems to be doing well advancing an open recognition agenda in Europe. Maybe an organized group of recognition advocates could make similar advances in the U.S.? Heck, there are many locales around the world that might benefit from an organized group of local open recognition advocates, responsive to the needs and opportunities of their particular contexts. Let’s have meetings everywhere!

But in the meantime, can we at least have a meeting in the United States? I want to go to one.

]]>https://stinjitsu.wordpress.com/2019/01/16/an-open-recognition-meeting-in-the-united-states/feed/0stinjitsuDesigning a Pathway for Self-Issuing Open Badgeshttps://stinjitsu.wordpress.com/2018/05/21/designing-a-pathway-for-self-issuing-open-badges/
https://stinjitsu.wordpress.com/2018/05/21/designing-a-pathway-for-self-issuing-open-badges/#commentsMon, 21 May 2018 14:29:04 +0000http://stinjitsu.wordpress.com/?p=73In previous blog posts, I described an approach for how the Open Badges 2.0 standard, plus a pathway standard, might enable us to reputably self-issue badges. I’ve decided to test the approach by self-issuing badges as part of my professional performance review. I am designing a professional development badge pathway for myself by identifying professional goals that are related to either my job performance review or to job title progression at the University of Wisconsin-Extension. Then I am aligning those professional goals to recognized standards. Then, when I have the opportunity to demonstrate any of those competencies in my professional work, I will gather evidence, issue myself a badge, and seek endorsements from supervisors and colleagues. When all is said and done, I hope to have earned evidence-backed and endorsed badges that I can share with the world. Compare that to the conventional static performance review document that gets filed away and never looked at again. I like the badging way better. I’m using Badgr’s Pathways tool, but I’d be curious to know how self-issuing badges might work on other platforms too.

In this post, I’d like to share some challenges with which I’m grappling in this little quest.

My first step has been planning the framework and overall vision for how I’ll do this. My challenges in this step seem to revolve around the question of how do I design a pathway that is both a comprehensive representation of my professional learning goals and development, and an integration or synthesis of different pathways and competency frameworks?

Branches of my pathway

When I conceived of this experiment, I thought only of how I might self-issue badges as part of a job performance review. I could keep this experiment dead simple by adding only my 2018 professional goals to the pathway since my goals are, as far as I can tell, all that officially matter for my performance review (see image below).

However, in reality that’s not the only way my organization tracks my performance. UW CEOEL has also defined a set of competencies and behaviors related to job title progression. As I’ve thought more about this experiment, I’ve realized that I’d like my pathway(s) to be a more comprehensive representation of my learning and development goals at CEOEL, so naturally I’d like it to cover title progression as well.

The two pathway branches of 2018 professional goals and title progression competencies spell out goals for me that are fairly constrained and prescriptive. I obviously didn’t write the title progression competencies, and my professional goals needed supervisor approval and were expected to align closely with active CEOEL projects. That said, I have a bit of agency when it comes to what parts of my professional work I decide to offer as evidence of progress toward those goals.

Of course, we could imagine a third pathway branch that represents, not just Justin’s competencies as an instructional designer at UW CEOEL, but Justin’s competencies as an instructional designer… or even Justin’s competencies as a professional person. This third pathway branch, the one that represents any and all instructional design related competencies or experiences that I choose to identify, really opens up my pathway possibilities to the wild blue yonder. I would be designing a pathway branch for my own professional development and trajectory. Okay, no big deal. Before getting started I would simply have to ask myself: What is my trajectory? Where am I trying go with my professional life?

Anyone else feel the need for a 5-minute panic attack break? Just me?

Perhaps the third branch doesn’t even belong with the first two branches. Maybe it would be better off as its own pathway. The first two branches can be seen as more or less aiming at the same practical goal: succeeding or advancing at my current job. The first two afford successful completion. The third branch seems to aim at a larger goal: me becoming the sort of professional instructional designer that I want to be. That’s the sort of goal I hope never affords successful completion -progress toward, yes, but not completion.

Why couldn’t one badge pathway support all three branches? Technically it could. But I wonder if enlarging the scope to include the third branch changes the essential character of what I’m trying to do with the pathway.

Am I using the pathway to envision a path to a goal?

-or-

Am I using this pathways tool as a way to document all my instructional design related knowledge?

To what extent can Pathways do both? Other folks may have a clearer vision of that. For me, it’s still an open question.

Alignment redundancies

There is additional complexity introduced by my design decisions (1) to include goals and competencies from multiple sources, and (2) to align competencies and goals to recognized standards. Doing so introduces alignment redundancies in my pathway. For instance, let’s imagine I decided to pursue a project management certification from PMI. A lot of my work as an instructional designer requires project management skills, and such a certification might make sense. However, because project management is such an important part of my job, there are already goals and competencies defined as part of job title progression that relate to project management. For example, I might have two competency statements related to “scoping” a project appear in two different branches of my pathway.

Eventually, I expect we’ll be able to use the IMS Global CASE (Competency and Academic Standards Exchange) standard to map such redundancies: https://www.imsglobal.org/activity/case. In the meantime, I plan to map such redundancies manually -probably with a spreadsheet- so that I’m sure to associate badges I earn with all relevant goals and competencies.

Align to competency statements or whole frameworks?

Finally, there’s a question about how to incorporate existing external competency standards. As I see it, I am faced with two options for incorporating existing standards.

Option 1: I identify goals for myself and design a pathway to connect to those goals. The pathway(s) I design will be composed of individual competencies and/or experiences each of which is aligned to a recognized standard that I have cherry picked. Those standards may come from a variety of sources. For instance, perhaps I’d use competency statements from the Association for Talent Development’s competency model (https://www.td.org/certification/atd-competency-model) for some of my instructional design competencies, but use the Project Management Institute’s knowledge areas and process groups (https://www.project-management-prepcast.com/pmbok-knowledge-areas-and-pmi-process-groups) for project management related competencies. Stated another way, in this option I take individual competency statements from recognized competency frameworks, but I do not try to incorporate whole frameworks.

This approach offers several advantages. It makes me think about my goals and the concrete steps that might help me achieve those goals. It also offers a fairly straight-forward way to integrate diverse standards frameworks.

Of course, there are also disadvantages. It’s not comprehensive. It doesn’t account for those competencies and experiences that perhaps don’t fit neatly into my plans, but that I might nonetheless wish to document. It also seems to lack an affordance for… serendipity. I have in mind the unexpected leaps and connections that Carla Cassilli so elegantly praised in her blog post (https://carlacasilli.wordpress.com/2013/03/25/badge-pathways-part-1-the-paraquel/).

Option 2: I somehow try to represent a synthesis of many diverse standards in my pathway. So in this case, I might create an entire branch for ATD, and another branch for PMI. Then, when I demonstrate a given competency or have a noteworthy experience in my professional work, I would issue myself a badge underneath the appropriate branches and competency node(s). I would likely need a way to handle overlaps or synonymy between the standards. Once again, I believe that is where the IMS CASE standard would play a vital role.

At a gut level, I’m drawn to Option 2, but I must admit it’s beyond the scope of what I want to accomplish as an individual doing a little experiment. Plus, I get the sense that Option 2 is perhaps more than a pathway. It’s more like what one would do to create a competency profile, or perhaps as part of a comprehensive learner record (https://www.imsglobal.org/activity/comprehensive-learner-record). It would enable me to attach evidence and credentials to as many of my competencies and experiences as is humanly, technologically, and informationally possible. It would also have some interesting potential consequences for thinking about competencies in ways that cut across traditionally siloed domains, e.g. enabling others to see what kind of project manager I might be based on my competencies and experience as an instructional designer.

What are the implications for upside-down badge stacking?

Upside-down badge stacking seems naturally to lead one’s thinking to Option 2 (above). That is, I should just be able to identify recognized competency standards that align to competencies in which I’m interested, then add those standards aligned competencies to my pathway, and then start stacking badges beneath the competencies. If I’m interested in an entire domain, for instance, instructional design, I should be able to identify an appropriate competency framework, then add that framework to my pathway, and then start stacking badges using that framework. The truth is, I’m interested in many domains: project management, systems thinking, marketing, visual design, pedagogy/andragogy, software development, open education, communities of practice, and the list could go on and on. If I’m interested in those domains, I should be able to identify recognized standards, and then add them to my pathway, and then start stacking badges, right?

Do you see the same challenges I’m seeing? Where does it end? How do I make sense of all those overlapping competency frameworks?

For the time being, I’m going to stick with Option 1: identify goals for myself and design a pathway to connect to those goals. That’s what Badgr’s Pathways tool seems designed to do, and that’s what fits the scope of my experiment to self-issuing badges as part of my performance review.

Thanks for reading and please feel free to comment.

]]>https://stinjitsu.wordpress.com/2018/05/21/designing-a-pathway-for-self-issuing-open-badges/feed/1stinjitsupahtway-goals_onlyPathways-goals_title_progEarly Thoughts on Upside-Down Badge Stacking and Assessment Validityhttps://stinjitsu.wordpress.com/2018/05/09/early-thoughts-on-upside-down-badge-stacking-and-assessment-validity/
https://stinjitsu.wordpress.com/2018/05/09/early-thoughts-on-upside-down-badge-stacking-and-assessment-validity/#respondWed, 09 May 2018 15:15:21 +0000http://stinjitsu.wordpress.com/?p=71Lately I’ve been blogging about an “upside-down” approach to stacking badges as a way to reputably self-issue badges. If you have no idea what I mean by upside-down stacking of badges, I invite you to check out my previous post on the topic. In a nutshell, the idea involves learners identifying goals or competencies in which they’re interested, and then acquiring or self-issuing badges as evidence of progress toward those goals or competencies. The learner creates a collection of “evidence” badges, and the more badges they acquire and the more they or their badges are endorsed, the more compelling their claim of competence begins to look.

In this post, I’d like to share some early thoughts about how collections of badges, and using upside-down stacking in particular, relates to assessment validity.

An upside-down approach to stacking badges hinges on the idea that a collection of badges can represent one’s claim to possess a competency or to have progressed toward a goal as well as an individual badge can. I believe that’s an assumption of badge pathways in general. But if that’s the case, I wondered if concerns about badges, over things such as validity and rigor, applied in the same way to collections of badges. My gut sense is that collections of badges may be more resistant to potential problems around validity. However, that “gut sense” might also be called a bias, and so it deserves some scrutiny.

Assessment Validity

What got me thinking about this in the first place was a concern over the validity of claims made by digital badges, and especially of claims made by badges tied to competency-based assessments. This was a concern I saw raised by Dan Hickey in his 2016 blog post “Traditional Approaches to Validity in Assessment Innovation.” In his post, Hickey wrote that he’s surprised credential innovators don’t express more concern over the criterion validity of the claims some credentials make. Criterion validity, as I understand it, refers to how well the results/scores of an assessment correlate to some external or future metric. For instance, the results of a competency-based assessment that assesses a learner’s ability to write professional emails may show a high degree of criterion validity if results from that assessment correlate strongly with, say, that learner’s job performance reviews related to professional email writing.

Hickey’s concern about criterion validity resonates with me because, as a designer of competency-based micro-credentials, I’ve seen first hand the tendency to make competency claims in micro-credentials that feel too strong for the evidence used to support those claims. As an example, imagine a learner who takes a competency-based micro-credential course on the topic of writing professional emails. The learner’s competence in this domain is evaluated by an authentic summative assessment: she has to write an email, the professionalism of which is evaluated by an assessor using a rubric. If the learner scores above a certain threshold, she earns a badge that claims she has demonstrated mastery at writing professional emails. But has she really demonstrated mastery? Is one email, written in the context of an assessment, which may contain instructional prompts, examples, and maybe even a template, sufficient evidence on which to base a claim that our learner will be a competent email writer “out in the wilds” of the workplace?

I wondered if an upside-down approach to stacking badges may address concerns about the criterion-related validity of micro-credential assessments. In one sense I think it might, but in another sense I suspect it may simply shift the concern to the evaluation of collections of micro-credentials.

On the one hand, it may address concerns about criterion validity of competency-based micro-credentials because, with upside-down badge stacking, there needn’t be an artificial threshold that sharply divides learners who “possess” a given competency from those who “lack” that same competency. Learning usually isn’t an all or nothing prospect. It more often exists as a gradation that ranges from novice to master (Dreyfus & Dreyfus, 1980). Of course, badge and pathway designers aren’t naive about this, and there’s currently an effort to make badges reflect that graduated nature of learning by creating badge levels: Learner A earns the “Competent” badge, while Learner B earns the “Really Competent” badge, but Learner C earns the “Holy Competence, Batman!” badge. When badges are stacked “upside-down” beneath a pathway node (which itself may be a goal or a competency statement), there’s less need for such badge leveling. The claim of the competency statement simply looks more and more convincing as the learner acquires more evidence badges and more endorsements to support those badges. That said, there could still be room for badges that recognize when a learner, based on the evidence she has been able to marshall, passes over a competency threshold. Such a badge might be used to recognize additional rights and responsibilities that the learner has earned in an organization, e.g. email mentor, resident expert on writing professional emails, or even the right to email on behalf of the organization.

So, in the case of an upside-down arrangement of badges, the over-arching claim of whether or not an individual learner possesses a given competency needn’t reside within a particular badge or competency pathway. Instead, it can reside as a judgment in the mind of the badge consumer who reviews the competency claim and supporting evidence, and makes up her own mind about whether or not the learner is likely to be competent. Or if the badge consumer is an algorithm, it makes up its “mind” about the learner’s competence. In either case, it might be that a collection of upside-down badges makes a softer and more implicit claim about a competency or goal than does the explicit and all-or-nothing sorts of claims made by many competency-based micro-credentials.

On the other hand, upside-down badge stacking may simply shift concerns about the criterion validity of micro-credentials to concerns about the validity of collections of micro-credentials. However, badge pathways may do the same thing; I’m not sure it’s a characteristic unique to upside-down stacking. Before I continue, I confess that when I’m talking about assessment validity, there’s still too much that I don’t know. Maybe someone can clarify for me: can one talk about the validity of a collection of badge-related assessments in the same way one talks about the validity associated with a single summative assessment? There wouldn’t appear to be a single assessment result with which to correlated to a criterion, so it seems the answer should be “no.” Or perhaps a collection of badge-related assessments might show a degree of validity that is somehow the sum of validity (validities?) of the assessments that make up the collection? In any case, I still think there’s a sense in which the sum of any collection of micro-credentials may or may not correlate with a given criterion.

Perhaps there can be a single score associated with of a collection of badges or pathways with which to correlate to a criterion. For instance, I have heard that Concentric Sky/Badgr is currently experimenting with a search feature it calls “Badge Rank,” which ranks badges based on a number of criteria. Open badges are machine readable, so it’s not hard to imagine that someone could develop a ranking for collections of badges. In other words, as the owner of a collection of micro-credentials, I might get a score that represents the probability of how likely I am to possess a given competency (side note: I don’t think that’s what Badgr’s Badge Rank does. I only mentioned it to point out that technologically the prospect isn’t so far-fetched). The service/program that generates that ranking might be thought of as another kind of badge consumer, and it would almost certainly be an algorithm.

I don’t think I’ve answered any of my questions about validity and collections of badge-related assessments. That’s okay, my goal in this post was simply to share some early thoughts on the topic. At least, that sounds better than saying I didn’t figure anything out.

Incidentally, the notion that collections of micro-credentials, and by extension the owners of those collections, may one day be evaluated and ranked by algorithms is something I hope to hear others eventually weigh in on (and perhaps others already have and I simply missed it). It’s a topic about which I feel ambivalent: if micro-credentialing proliferates then perhaps that kind of scoring will be needed by badge consumers, e.g., human resource professionals. At the same time, it feels like something that has the potential to run counter to the ethos of open badges. Plus, it’s tough to imagine what the far-reaching consequences of such scoring might be.

]]>https://stinjitsu.wordpress.com/2018/05/09/early-thoughts-on-upside-down-badge-stacking-and-assessment-validity/feed/0stinjitsuSelf-Issuing Digital Badges for my Performance Reviewhttps://stinjitsu.wordpress.com/2018/04/13/self-issuing-digital-badges-for-my-performance-review/
https://stinjitsu.wordpress.com/2018/04/13/self-issuing-digital-badges-for-my-performance-review/#commentsFri, 13 Apr 2018 14:42:26 +0000http://stinjitsu.wordpress.com/?p=66In an earlier post I described an idea for how the Open Badges 2.0 standard and a pathways standard might enable a person to reputably self-issue badges. Thanks to some excellent feedback in the comments of that post, I have revised the idea:

Step 1: A learner identifies a competency or goal that she is interested in developing or achieving, or that she believes she has already developed or achieved.

Step 2: The learner adds that competency or goal to a learning/development pathway. The learner may write the competency statement herself, or she may find and use a competency statement already written by an authoritative source, e.g. a recognized competency standard. That competency or goal then acts as a node on her pathway.

Step 3: The learner begins to earn badges that support the claim of the competency statement, or that serve as evidence of progress toward the stated goal.

Step 4: The learner’s supporting badges are endorsed by others.

The competency node is similar to a meta-badge in the sense that it signifies an overarching competency that is supported by a collection of “child” badges. However, it differs from the “conventional” understanding of how a learner earns a meta-badge and progresses along a pathway. That’s because, in what I’m describing, the learner can acquire the competency node before earning the supporting child badges. The learner then earns badges in order to support the claim made by that competency node. The more supporting child badges the learner earns, and the more those badges are endorsed by others, the more credible the claim made by the learner’s competency statement begins to look. Because the supporting badges can be stacked under the competency ad infinitum, I called it downward stackability.

How About We Try It?

I’d like to try this idea, and if you’re looking for a way to get involved with badges, I hope you might try it with me. I work as an instructional designer at the University of Wisconsin, and I have struck a deal with my supervisor and my human resources department to incorporate badging into my annual performance review. Do you have a supervisor, an HR department, or even colleagues that’d be willing to play along? Then you have the required ingredients to try it! What do you have to lose? As far as badge “pilots” go, this one is fairly low-risk, low-cost, and I imagine many working professionals could ask their organization to try something like this. What you get out of the deal is some lasting value from performance reviews in the form of open digital badges. What your organization gets out of the deal is a demonstration of the potential value for training and development that digital badges have to offer. What the digital badges community gets is a sneaky way to penetrate what has proven to be a challenging barrier to the wider adoption of digital badges: employers’ and human resource departments’ understanding of digital badges.

Here’s how I envision this working, at least in my case.

Competencies and Goals: I will find competencies to incorporate into my learning/development pathway. For instance, my instructional design department has documented competencies that it wants its instructional designers to possess. I will also find instructional design competency standards from other sources -perhaps from the Association for Talent Development’s Competency Model. I also have professional goals that I already defined for this year of 2018. Both competencies and goals will serve as nodes in my learning / development pathway.

Earn or Self-Issue Badges for Training: When I take part in training that relates to those competencies or goals, I will collect evidence from training and issue myself a badge. Of course, in cases where the organization offering training is willing to issue me badge, I won’t need to issue my own.

Earn or Self-Issue Badges for my Work: what’s more interesting (in my opinion), I will reflect and take stock of my regular work at UW-Extension. If specific instances of the work I do represent progress toward my goals, or evidence that I am acquiring (or have acquired) a particular competency, I will collect rich evidence and issue myself a badge.

Seek Endorsements:In both the case of training and regular work, I will seek endorsements from colleagues, collaborators, and/or supervisors. Endorsements is an awesome new feature of the Open Badges 2.0 standard. However, not all Open Badge 2.0 vendors have added endorsement features to their platforms yet. I believe Credly and Digitalme have added the ability to endorse others and/or their badges. Nonetheless, I’m curious to see if Badgr’s Pathways feature will be compatible with this notion of downward stackability. And I don’t imagine Badgr is too far away from offering the ability to endorse. In the meantime, my plan is to simply ask people to endorse the badges by writing a brief description of how they know me, how they know my work, and why they think my work demonstrates a competency. Then, once I finally settle on a badge platform, I’ll ask my endorsers to add the endorsements that they’ve already written.

Incorporate Badges into my Performance Review:At least twice per year, I will present my badges to my supervisor and HR department and seek their endorsements. If they’re willing, I might ask my supervisor and HR department to issue me badges. But… maybe not. Digital badges are still unfamiliar territory for my colleagues, and I must continually remind myself to take baby steps. Patience is a virtue!

That’s it, at least for the badging part. I’ll continue to blog about this experiment as it unfolds, and I plan to keep a reflective journal throughout the experience. If you are willing to try this with me, maybe you can do something similar? I’d be very curious to learn from you and from others. Cheers!

At the University of Wisconsin-Extension CEOEL, I work on a competency-based micro-credentialing program called the University Learning Store (ULS). It was conceived of as an online store for learning, where students can purchase mini-courses both to acquire and to be assessed on discreet competencies. I suspect there will be more programs like the ULS popping up as micro-credentialing and competency-based education continue to gain recognition and acceptance.

As an open education believer / dreamer working on such a program, I asked myself “Is there a place for open pedagogy in this CBE and micro-credentialing world?”

That question led to a pilot project: a mini-course, fully open and editable, that welcomes learner-generated instructional content from anyone who wishes to contribute. Associated with this course is an assessment and an opportunity for learners to earn a university-branded micro-credential (i.e., a digital badge). The subject of the course is Human Resource Management: Creating a Training Plan Using the ADDIE Design Model. An unfinished draft of the course is available for editing now on Wikiversity (a sibling of Wikipedia). Check it out!https://en.wikiversity.org/wiki/Create_a_training_and_development_plan

Why Combine OEPs and CBE Micro-Credentialing?

The majority of open pedagogy projects I’ve seen take place in the context of traditional courses offered by traditional institutions –and that’s wonderful and as it should be. A benefit of these courses is that they typically come equipped with visionary and inspiring teachers.

The CBE micro-credentialing courses I’ve worked on tend to lack teacher:learner interaction. The SME’s with whom I’ve collaborated to develop these mini-courses have done great design work; it’s just that teacher:learner interaction isn’t a big part of the CBE micro-credentialing model, generally speaking.

I wondered: to what extent do open pedagogy projects require the presence of visionary faculty to give them life and drive them forward? Could a CBE micro-credential course support open educational practices even without a strong faculty presence? Jim Gee, in his 2005 paper “Semiotic Social Spaces and Affinity Spaces,” gave us the term “affinity spaces” to refer to places (real or virtual) where learners self-organize to create their own teaching and learning around shared interests and endeavors. From what I understand of them, affinity spaces seem to have a good deal in common with open educational practices. According to Gee, an affinity space tends to have the following characteristics:

Common endeavour, not race, class, gender or disability, is
primary

Newbies and masters and everyone else share common space

Some portals are strong generators

Internal grammar is transformed by external grammar

Encourages intensive and extensive knowledge

Encourages individual and distributed knowledge

Encourages dispersed knowledge

Uses and honors tacit knowledge

Many different forms and routes to participation

Lots of different routes to status

Leadership is porous and leaders are resources

It seemed to me that, if a CBE micro-credential course were going to support open educational practices, it would likely fit Gee’s description of an affinity space. The next question was, what technological platforms are best for supporting affinity spaces?

Wikis appear to be one kind of technology that can support affinity spaces. Wikipedia is the most well known example, but it has a less well-known sibling called Wikiversity, which was meant to enable users of the web to create free, openly-licensed, online courses in the same way Wikipedia enabled users to create the world’s largest encyclopedia. Wikiversity is a platform that appears well-suited, perhaps even purpose built, for open pedagogy. And it was based on a platform (Wikipedia) that blazed a wildly successful trail in just a slightly different context. I decided to try to build the pilot using Wikiversity.

Unfortunately, interest in Wikiversity appears to have peaked prior to 2010 and languished since then (Wikimedia Statistics, n.d.). Why is that?

Allow me to suggest one possible factor for the waining interest in Wikiversity, and explain how the approach taken by this pilot may fill a learner need (with the caveat that this possible factor is not backed up by any systematic research and I really don’t know the answer). Early in Wikiversity’s history, users decided the platform should not attempt to issue credentials (Wikiversity, n.d.). Hence, users can learn on Wikiversity, but there is no obvious feature of the Wikiversity platform that enables users’ learning to be recognized by anyone but themselves. Absent recognition, Wikiversity users may not have seen enough value from the platform to continue using it.

This pilot from UW-Extension asks the question “What happens when we attach recognition in the form of a university-branded credential to instruction created by learners in an affinity space?”

The answer is… I have no idea what will happen! I can only tell you what I’d like to see happen.

I would like people to contribute! This whole experiment really hinges on contributors. All other hopes are contingent on this.

I hope the Wikiversity course could serve as an affinity space for those interested in training and development or human resources management. Users could ask questions and debate about the instructional content using the wiki’s talk pages. Novices could contribute content alongside experts. An aspect of social learning might be infused into a mini-course that would otherwise be a static competency-based resource intended for consumption by individuals.

If the resource were tied to an active group of users, I’d love to see instructional content that is up-to-date, validated, and improved by user testing.

I’ve heard for years that employers and industry don’t have a good way to communicate their needs to higher ed. I hope that training and HR professionals would be able to use this mini-course to voice their needs.

I hope the Wikiversity pages would be used as an open educational resource.

I’d be glad if this pilot could contribute in some small way to the larger conversation about the sustainability of open educational resources and practices. At least in the case of UW-Extension, the approach taken by this pilot may provide a financial incentive toward good OER stewardship. The incentive derives from the potential for learner- and user-generated instructional content to reduce costs on course development and maintenance. To my simple way of thinking, if an organization sees OERs as appealing not only to its mission and ideals, but also to its bottom line, then perhaps it will be more likely to act as a good OER steward (Petrides, Levin, and Watson, 2018).

If you’d like to learn more about the University Learning Store, have a look at the website. The assessment that accompanies the mini-course HRM: Create a Training Plan Using the ADDIE Design Model isn’t available yet, but it should be published within the next week or two. I will update this post with a hyperlink to the assessment as soon as it is published.

Finally, here’s a comic I did about the pilot project. Thanks for reading and feel free to comment.

References

Gee, J. P. (2005). Semiotic social spaces and affinity spaces: From The Age of Mythology to today’s schools. In D. Barton & K. Tusting (Eds.), Beyond communities of practice: Language, power and social context (pp. 214-232). Cambridge, UK: Cambridge University Press.

]]>https://stinjitsu.wordpress.com/2018/04/06/open-educational-practices-and-micro-credentialing-do-they-work-together/feed/0stinjitsuSFTPNLHE-blogCreative Commons LicenseAn upside-down way to stack open badgeshttps://stinjitsu.wordpress.com/2018/02/27/an-upside-down-way-to-stack-open-badges/
https://stinjitsu.wordpress.com/2018/02/27/an-upside-down-way-to-stack-open-badges/#commentsTue, 27 Feb 2018 16:05:53 +0000http://stinjitsu.wordpress.com/?p=38In the previous post, I suggested a way that learners may be able to reputably self-issue badges. I wrote it in response to the Open Recognition Alliance’s MIRVA project (Make Informal Learning Visible and Actionable). Here’s a link to a webinar in which Serge Ravet describes his goals for the MIRVA project: https://archive.org/details/ORA2018-January-17.

In this post, I would like to amend my original idea to replace “interest” badges with competency nodes on a pathway. You see, although I didn’t originally frame it as an idea about badge pathways, I’ve since realized that’s exactly what it is. But first, a recap.

Recap:

The idea went like this:

Step 1: A learner identifies a competency that she is interested in developing, or that she believes she already possesses.

Step 2: The learner acquires an “interest” badge, which simply represents her interest in the competency, or her claim about already possessing the competency. There’s no criteria or evidence required to acquire this badge. This interest badge is akin to a meta-badge (but as you’ll soon see, I now suggest using a competency node instead).

Step 3: The learner begins to earn badges that support the claim of the interest badge. Each supporting badge is a child of the parent interest badge. These child badges contain evidence, perhaps from artifacts she’s created or events she’s attended, that support the claim of the parent interest badge.

Step 4: The learner’s supporting badges are endorsed by others.

In most badge pathway schemes, learners must earn all the child badges before earning the parent/meta-badge. Hence, badges and meta-badges continually stack up as the learner progresses along the path toward mastery, or certification, or whatever. This “upward stackability” seems to be the norm when it comes to badge pathway progression.

In the idea I described, it’s reversed: a learner earns the child badges only after acquiring the interest badge (which acted similar to a meta-badge). Hence, I called it downward stackability.

Enter Pathways: A competency node instead of an interest badge

I was all set to publish a second blog post, alternative to this post, that delved into more details about how Open Badges 2.0 + a badge pathway standard might support downward stackability. Then I read Tim Cook’s 2015 blog post, “The Quantum Mechanics of Learning,” (https://medium.com/sprout-stories/quantum-mechanics-of-learning-742f9ba70cc2) and was obliged to start over. In his post, he suggested that nodes in pathways could represent more than just badges. Nodes could also represent competencies or resources. I was instantly intrigued (and only a little annoyed) because I saw how his idea could work much better for what I was proposing. Instead of issuing themselves “interest badges,” learners could insert competencies into a pathway and then begin issuing, or acquiring, badges in support of those competencies. Downward stackability is still facilitated, and the learner is still very much in control of the development of his or her own pathway. But it has the likely added benefits of greater standardization and probably a more streamlined and intuitive user experience.

One more possible benefit of using competencies: adding a competency node to a pathway may not require any extensions to the Open Badges 2.0 standard, whereas interest badges may have required an extension to articulate criteria for issuance of supporting child badges. Of course, competency nodes on a pathway would also need to articulate criteria for issuance of supporting child badges. But as luck would have it, the new IMS Global CASE standard (Competency and Academic Standards Exchange) seems to provide support for just that sort of use case, at least as I understand it (but I’m still reading up on CASE, so please correct me if I’m wrong).

Why should pathways support something like downward stackability?

Why bother with something like downward stackability? Carla Cassili, in her 2014 “Paraquel” blog post (https://carlacasilli.wordpress.com/2013/03/25/badge-pathways-part-1-the-paraquel/), wrote what I think is as good an argument as any: “Badge pathways can be and most likely will be entirely emergent. This, friends, is from whence all their magic derives. Badge pathways provide people with opportunities to make decisions based in personal agency, to define steps that may seem more like hops, and to think about ways to do things that aren’t sequential or even seemingly rational.”

And yet, when I look around at the sorts of badge pathways that are emerging, they all appear rather linear and “top-down.” There is still the need to pre-define the curricular scope of the set of child badges required to earn a meta-badge. It is still the job of a designer to define the learning path, and the job of learners who want the credential to follow that path. And don’t get me wrong, that’s not entirely a bad thing. I imagine the vast majority of badging use cases that consumers want would require that sort of top-down design.

But I don’t see how that kind of linear or tiered design facilitates individuals making their own informal learning visible and actionable. I also don’t see how that type of design could be flexible enough to truly map the learning and interests within an organization as effectively as the more emergent system.

I think we want both types of pathway, linear and clustered, at the same time. After all, even if a clustered pathway can capture an individual’s informal recognition, that recognition probably won’t be especially visible or actionable if it exists in a bubble. Learners and communities will likely want ways to connect that informal learning to more traditional learning, because doing so will make their informal learning more visible and more actionable.

I believe the value an idea like downward stackability is in letting learners define and actively generate evidence of their own competencies. The more evidence a learner is able to offer in support of his or her competencies, and the more that evidence is endorsed by others, the more compelling the learner’s competency claim becomes. And I see no reason something like downward stackability couldn’t exist side-by-side with regular ‘ol upward stackability.

So the next question is: could we have a hybrid?

As I said, IMS Global’s CASE standard (Competency and Academic Standards Exchange) looks promising for enabling the description/articulation of criteria needed to define what ought to count as evidence that learners have acquired or demonstrated some modicum of competency. As for badge pathways standards, Concentric Sky/Badgr says that it has submitted a proposal to IMS Global for a badge pathways standard called Open Pathways. Supposedly it will make use of not only the Open Badges standard, but also of the IMS CASE standard and the IMS Extended Transcripts standard. You can watch Concentric Sky CEO Wayne Skipper describe the proposed standard in this YouTube video (https://www.youtube.com/watch?v=JBCCVdrIaDQ). Concentric Sky is obviously interested in enabling the alignment of pathways to standards. Maybe they have something in mind that is similar to what I’ve described. I certainly hope so.

]]>https://stinjitsu.wordpress.com/2018/02/27/an-upside-down-way-to-stack-open-badges/feed/9stinjitsuA thought concerning MIRVA and Open Badges 2.0https://stinjitsu.wordpress.com/2018/01/30/a-thought-concerning-mirva-and-open-badges-2-0/
https://stinjitsu.wordpress.com/2018/01/30/a-thought-concerning-mirva-and-open-badges-2-0/#commentsTue, 30 Jan 2018 18:45:22 +0000http://stinjitsu.wordpress.com/?p=12Not long ago I sat in on an ORA (Open Recognition Alliance) conference call about their MIRVA project to “Make Informal Recognition Visible and Actionable.” Serge Ravet led the call, the specific purpose of which was to discuss a draft framework for MIRVA.

In the call, Serge challenged the participants to think about ways recognition that is visible and actionable might also be made more community-centered and dynamic. In this post (of a blog I created solely for the purpose of writing this post), I’m exploring one idea for how that challenge might be met by the existing Open Badges 2.0 standard (OB 2.0).

Serge characterized the ways in which recognition can be made visible and actionable as existing along a two dimensional plane. Looking at the x-axis, we see that recognition can range from traditional (i.e., static and past-oriented) to non-traditional (i.e., dynamic and future-oriented). Looking at the y-axis, we see that recognition can be formal (i.e., institution-centered) or informal (i.e., community-based).

Image created by Serge Ravet

According to Serge, the majority of our work with open badges up to this point has existed in the lower-left quadrant of the plane, that is, on the “Traditional” and “Formal” ends of the spectrum. He challenged us to think of ways recognition might also exist in the upper-right quadrant, that is, on the “Non-Traditional” and “Non-Formal” ends of the spectrum.

The Problems to Solve

I’ll describe the problems I’d like to address and then spell out the proposed solution as a series of questions. It might be an unusual approach, but hopefully you’ll get the idea*.

Problem #1: As badges are currently used, there doesn’t appear to be an elegant way for recipients to stack their own badges according to themes, e.g. competency or topic. A badge issuer can add metadata to a badge that identifies it as “related” to other badges in terms of topic or competency. However, badge recipients aren’t able to edit the metadata of badges they receive (for understandable reasons).

Yet the ability to make implicit themes or overarching competencies within a collection of badges explicit (visible and actionable) should matter to individual and community badge holders. That’s because their badges, taken as a collection, can paint an emergent picture of who that individual or community is and what they can do. But it’s a picture that may easily be overlooked by badge consumers.

Problem #2: In the current badging environment, there doesn’t yet seem to be a credible way for individuals and/or communities to issue badges to themselves.

Problem #3: For any given competency, an individual may be capable of demonstrating different “levels” of competence, ranging from novice to expert. One system of badges might define levels of competence, but can those levels be easily understood and rectified within other badging systems?

The Idea

Here’s the idea about how Open Badges 2.0, or something close to it, might be used to address those problems. I’ve parsed the idea as a series of questions.

Question 1: What if an individual, using Open Badges 2.0, issued a badge to himself or herself that represented either an interest in a topic or a claim about a competency that he or she possesses?

Question 2: Supposing OB 2.0 could support the action in Question 1, what if that individual then issued badges to himself or herself that represented evidence in support of the interest or competency claim? Such evidence might be an event, a relationship (e.g. professional mentor/mentee relationship), a demonstration, or something else. Each of these “evidence” badges would contain metadata that identified it as related to the “interest” badge described in Question 1 (if possible, as a child of that interest badge).

Question 3: Supposing OB 2.0 could support the actions in both Questions 1 and 2, what if those evidence badges where then endorsed by others, using OB 2.0’s endorsement features? Would those self-issued badges look any more credible to badge consumers?

Stackability

People familiar with open badges are likely familiar with the idea of stackability. Badges are stackable in the sense that a set of badges can sometimes comprise an overarching meta-badge. For example, if I earn a badge on writing a project charter, another on planning a project’s schedule, another on budgeting, and another on stakeholder communication, then that set of earned badges might amount to an overarching badge called project management.

You could think of this as a sort of upward stackability because the badge recipient first has to earn the smaller badges before earning the overarching meta-badge.

The idea described in the above questions would amount to something like downward stackability of badges. First, the recipient acquires the meta-badge, which represents that recipient’s interest in a topic / competency, or claim about already having some competency. Then, the recipient earns more badges -an unlimited number- in support of that overarching meta-badge.

Referring to Serge’s table, this idea might enable “upper-right quadrant” recognition, i.e., recognition that is both non-formal and non-traditional.

However, it might also have value for the lower-left “standardized” quadrant. For example, imagine an organization that publishes a badge class whose purpose is to represent an interest in a particular competency. In support of that badge, the organization publishes criteria for what it would consider credible evidence that an individual has attained some modicum of that competency.

An individual wishing to acquire the organization’s “interest” badge could do so easily -perhaps by simply providing his or her email address to the badge issuer. After that, the individual would set about issuing “evidence” badges to himself or herself, or earning evidence badges that others offer, in support of that overarching meta-badge. The more evidence badges the individual created, and the more others endorsed the credibility of the evidence, the more credible the individual’s overarching claim of competence would start to look.

Reflection

I’ll admit that this isn’t a fully baked idea, and I already see a few problems with it.

First, users would need to create entirely new badge classes every time they wanted to issue themselves evidence badges. That’s time-consuming. Second, there’s still plenty of room for self-issuers to fabricate evidence. Third, it’s hard to imagine users spending much time researching someone else’s interest and evidence badges in order to endorse them. But who knows, maybe there’s a value proposition to those who would be endorsing others that I haven’t considered.

I’m interested in hearing from others if Open Badges 2.0 could support something like this, and whether or not it’s an idea worth pursuing.

*Apologies to anyone else who may have already this idea or a similar idea. I didn’t cite you because I haven’t encountered your work yet.