The authors note that a psychological perspective on social networks is rarely taken and advocate for more research. They define 8 (psychological) roles that are thought to be played in these networks regarding goal achievement that they present as a framework to encourage more research. The roles are:

Goal Striving; directly attempting to achieve a specific goal.

System Supporting; supporting those in goal pursuit.

Goal Preventing; actively working to prevent goal achievement.

Supportive Resisting; supporting goal preventeurs.

System negating; responding with negative affect such as making fun of a person who is goal striving.

Interacting; People who can affect goals even though they do not intend to support or resist.

Observing; People who only observe network activity, but nonetheless ca be involved in unintended effects.

This framework could make for an interesting analysis of networks and may have practical relevance for a wide variety of practices. It may prove to be hard to disentangle the effects wrought by multiple or even conflicting goals in complex environments, or with fluid and changing alliances and more study is needed, however it may be interesting to follow.

We (have arrived) at a most surprising conclusion: . . . the things supposedly contained “in” (our inner lives) are not to be found “inside” us as individuals at all, but “in” the continuously unfolding relations occurring between ourselves and others (or an otherness), in our surroundings. (We cannot) hide the contents of our inner lives wholly inside ourselves, for, Like it or not, we “display” them in the unfolding movement of our living out our lives, responsively, amongst others. . . . we cannot but be immersed in it. (Quoting Wittgenstein) “Only in the Stream of thought and life do words [and our other activities] have meaning”. John Shotter, 1998, Social Construction as Social Poetics

Compare the activities of 2 students.

One studies a book, hears a lecture, and memorizes facts and theories of lead-base paint as an environmental hazard, before taking a test of recall. This is educating the latent mind of a student. But realistically, how long will this information be available? How well prepared is that student to be a productive part of society?

A second student also studies this book, but is not concerned with recall, confident that the content exists in digital resources that act as a scaffold to their understanding and can be located whenever needed. This students then participates in a peer discussion locating potential lead problems in their community and strategizing how this problem might be solved including additional research for resources through governmental and environmental organizations. The students defends their activities and strategies orally and they include a record of the resources they used in devising and supporting their strategies. They also documents their actions in a digitalized portfolio. How well prepared is this student to participate in society, to understand this topic in depth and over time, and to be responsible to their peers and their teacher for their engagement and their actions?

This is the educational relevance of Wittgenstein’s preference for finding meaning through practice. We have an idealized view of cognition, that our knowledge can be contextualized without contextualing cognitive skills. Knowing something is a cognitive skill. Being able to apply that knowledge within practice is also a cognitive skill, abet at a much higher functional level of cognition. This higher functional level represents the difference between project-based learning with performance assessment and lower level pedagogy with recall-based standardized assessment. Certainly the second student has emerged from this activity as a more capable, confident and engaged person. This doe not mean that facts and theories are not important. These types of things make up a significant portion of the discourse that students must have in order to engage each other, as well as the experts in this topic. But until they have engage responsively with others in authentic situations, this higher level of cognition will not be fully developed and even the lower level knowledge will not be significantly understood.

Interesting NY Times opinion piece today that extends my recent posts on validity and performance management. It is entitled: Why Your Boss Is Wrong About You, By Samuel Culbert. In that article he states:

In my years studying (performance) reviews, I’ve learned that they are subjective evaluations that measure how “comfortable” a boss is with an employee, not how much an employee contributes to overall results.

Samuel Culbert in this statement leads us to believe the problem with performance reviews is their subjective nature. From a measurement perspective I believe this is incorrectly stated. The problem is one of validity, that performance measurements typically measure the bosses level of comfort with an employee, not their performance. Greater objectivity will not help us if comfort is still the construct being measured. Instead we must look at validity.

Culbert proposes a performance preview process as an alternative.

It’s something I call the performance preview. Instead of top-down reviews, both boss and subordinate are held responsible for setting goals and achieving results. . . . bosses are taught how to truly manage, and learn that it’s in their interest to listen to their subordinates to get the results . . . “Tell me your problems as they happen; we’re in it together and it’s my job to ensure results.” . . . . It encouraged supervisors to act as coaches and mentors. . . . But understand that the performance review makes it nearly impossible to have the kind of trusting relationships in the workplace that make improvement possible.

This preview process may be a good idea in and of itself, but it does not logically get at the root problem. Measurement within this new process can have just as many validity problems as the old process. This is why validity is important.

Two additional things I’m thinking:

Culbert doesn’t quite get there, but I sense he is looking at something like Action Analytics, measurement tied to real-time feedback that can support performance while performance is still in formation. Instead of measurement in the service of performance review it is measurement in the service of performance support.

The second thought is this. The central point in Culbert’s process is trust between an employee and a boss, that is, a management relationship. This is the central construct in successful management and it may be important to measure.

This leads me to an interesting final conclusion and maybe a management axiom:

In managing people, we management relationships, but support their performance.

Introduction

We do believe that it is imperative that psychologists make a place for (construct validity) in their methodological thinking, so that its rationale, its scientific legitimacy, and its dangers may become explicit and familiar. This would be preferable to the widespread current tendency to engage in what actually amounts to construct validation research and use of constructs in practical testing, while talking an “operational” methodology which, if adopted, would force research into a mold it does not fit. (Emphasis added)

What was widespread in 1955 has not substantially changed today. Construct measures are routinely developed without regards to their construct or consequential validity, and it is in detriment to our practices. I will name this state, naive operationalism; measuring constructs with what amounts to an operational methodology. I will also show why it is a problem.

Operational Methodology: Its Origins as a Philosophical Concept

What do Cronbach & Meehl mean by an operational methodology? Early in my psychological studies I heard the definition of intelligence stated as “that which is measured by an intelligence test”. It was an example of operationalism (or operationism). Originally conceived by a physicist named Percy Bridgman, operationalism conceptually states that the meaning of a term is wholly defined by its method of measurement. It became popular as a way to replace metaphysical terms (eg. desire or anger) with a radical empirical definition. It was briefly adopted by the logical positivist school of philosophy because of its similarity to the verification theory of meaning. It also became popular for a longer time period in psychology and the social sciences. Neither use stood up to scrutiny as noted in Mark Bickhard’s paper.

Positivism failed, and it lies behind many of the reasons that operationalism is so pernicious: the radical empiricism of operationalism makes it difficult to understand how science does, in fact, involve theoretical and metaphysical assumptions, and must involve them, and thereby makes it difficult to think about and to critique those assumptions.

Not only does the creation of any measurement contains many underlying assumptions, the meaning of any measurement is also a by-product of the uses to which the measurement is put. The heart of validity theory in the work of Cronbach (and also in Samuel Messick), is in analyzing various measurement assumptions and measurement uses through the concepts of construct and consequential validity. Modern validity theory stands opposed to operationalism.

Operational Definition as a Pragmatic Psychometric Concept

Specifying an operational definition of a measure is operationalism backwards. Our measurements operationalizes how we are defining a term, not in the abstract, but in actual practice. When we implement a measurement in practice, that measurement effectively becomes the construct definition in any processes that involves that measure. If the process contains multiple measures, it is only a partial definition. If it is the sole measure, it also becomes the sole construction definition. Any measure serves as an operational definition of the measured construct in practice, but we don’t believe (as in operationalism) that the measures will subsume the full meaning of any construct. Our operational definition is no more than a partial definition and that is why consequential and construct validity are needed in our methodological thinking. Validity research tell us when our operational definitions are problematic and may give us indication as to how to make improvements to our measures. Validity research studies the difference between our operational definitions and the construct being measured.

Naive Operationalism

For most of us, operationalization outside the larger issue of a research question and conceptual framework is just not very interesting.

I could not disagree more! Not including validity in our methodological thinking will mean that our operationalized processes will result in what I will call naive operationalism. If we devise and implement measures in practice, without regard for their validity, we will also fail to understand any underlying assumptions and will be unable to address any validity problems. In effect, it is just like philosophical operationalism and sets us up for the same problems. Lets consider a concrete example to see how it can become a problem.

An Example of Naive Operationalism

Richard Nantel and Andy Porter both suggests that we do away with Performance Measurement, which is considered “a Complete Waste of Time”. These are the reasons given for scrapping performance measurement:

Short term or semiannual performance reviews preventing big picture thinking, long-term risk taking and innovation. We want employees to fail early and often.

This critique is advocating for a different set of constructs. True, the constructs they imply may not be common to most performance measurement systems, but there is no reason to stay with standard constructs if they are not a good fit.

There is no reason why formative assessments like action analytics and other more appropriate feedback structures could be a part of any performance improvement systems.

This is another instance where it appears that the wrong constructs, based on out of date motivational theories, are being measured. They are the wrong constructs and therefore the wrong measures.

The consequences of any measurement systems is the most important question to ask. Anyone who doesn’t ask this questions should not be managing measurement processes.

Conclusion

What is the bottom line? There is nothing Richard or Andy point out that would make the concept of performance measurement wrong. The measurement systems they describe are guilty of naive operationalism. The idea that any specific measure of performance is the sole operational definition needed and this is true even they are unaware of what they are doing. No! We should assess the validity of any measurement system and adjust according to an integrated view of validity within an appropriate theoretical and propositional network as advocated by Cronbach and Meehl. Measurement systems of any kind should be based on construct and consequential validity, not an operational methodology, whether it is philosophical or naive.

A new Model of Learning: from the Classroom to the Network

Learning has always been multifaceted, but where the old concrete model of learning activity was exemplified by the classroom, a new concrete model of learning activity will be exemplified by a network. It’s not a change in what learning is, but more of a change in the why, where, how, and when learning happens.

Why New Ideas for Learning are Needed.

The pace of market change and creative destruction is increasingly requiring innovation and adaptive responses just for business survival. John Hagel points beyond product and process innovation ot the need for institutional innovation is we are to counter the movement of innovation to Asia. The complex understanding and responses needed requires greater access not just to to innovative ideas, but also the social spaces that contain both knowledge flows and the diverse capabilities needed to actualize those ideas. Businesses need to move beyond the traditional boundaries of the firm.

Human development, once thought to be relative unchanging after age 25, now highlight the ability for all kinds of growth in mental complexity and ability throughout one’s active adult life. In response, new theories of performance are now available to support development and increase performance throughout one’s career.

In order to achieve complex adaptive change in activity, we must further our own development, improve the tools we have available, and make sure we are applying them and attending to the correct object or focus. This entails

Where will Learning Occur

Traditionally the classroom was led by an expert who was guided by a set curriculum and a transfer metaphor of learning. In contrast, the network contains a diverse array of individuals interacting with learning as an emergent phenomenon. This is not to say that experts, classrooms and the transfer metaphor will disappear, and learning, as a psychological and behavioral phenomenon, will not change. It’s just that the most valuable and ongoing form of learning will emerge through network participation and will emphasize it’s natural connection with relationships and activity instead of focusing exclusively on knowledge content. It will bypass the problem of learning transfer through learning in situ, in a just in time manner. Instead of teachers, we will depend on a variety of people who’s role will be more like a guide, facilitator or collaborator.

Network learning has a built in efficacy benefit in that it’s so closely tied to activity and action in which the learning subject is engaged. In a recent Charlie Rose episodeDaniel Wolpert mentioned that the only purpose for a brain is to enable complex adaptive behavior through the motor systems and that the motor cortex and muscle system is the end-path for all of our sensory systems. To think of content and knowledge as separated from activity is to ignore the way the brain is inherently organized. Just in time network learning is tied closely to enabling action, which is more in line with the natural organization of brain systems. If for no other reason, this type of learning is productive because it replaces the huge amount of knowledge that is committed to memory just in case it might be needed in the future with targeted knowledge that leads directly to action.

While learning is just in-time, building robust and diverse networks is the preparation we need. When you need resources is not the time for network building. The network building that taps us into vibrant engaging relationships and social spaces should be an ongoing activity. The support needed for this are learning institutions, but not like institutions of the past. Not the institutions that horde experts, but ones that foster these vibrant and engaging social spaces and excel at building business relevant social networks. This does not succeed by some network magic. Networks need to be filled with passionate and talented people. You need to be hooked in with the smartest people on the block, just as they need to be hooked in with talented and passionate you.

What will be the focus of Network Learning

I believe that learning as a psychological and social phenomena is not substantially changing, only the focus of learning will be on the activities and challenges we face. I will rely on an older model of Vygotsky and Leonte’v to explain a model of the architecture of human activity. Vygotsky gave three poles that combined to drive human activity: a subject, a mediator (tool) and an object, all leading to an outcome. This table gives examples for a carpenter and a loan officer.

Therefore the focus of learning is on:

the development of the subject’s identity and capabilities (achieving one’s developmental potential)

Opportunities for collaborative practice -based research (eg. localized unconferences) to maximize development and learning within or around one’s specialities.

Opportunities for creating and maintaining mentoring as well as other diverse types of relationships within one’s local environment.

Networks that are institutionalized to allow you to pursue and developmental goals and identities while conducting business. I say institutionalized to mean that the infrastructure may need to be created and supported. Like Hagel’s “Pull”, we rely on serendipity for opportunity, but we plan to make serendipity more likely to happen.

This is not the end of my “theorizing” but a good summation from which to begin a more active research process.

Interesting post from David Jones, documented here for future reference. The basic gist is that the task of developing support for performance and capability improvement in many contexts resembles a wicked problem in a complex adaptive system. Therefore, it is not adequately addressed by traditional analytic engineering approaches and should include the incorporation of learning throughout implementation.

. . . you need to include more of the growing/gardening approach into your engineering method. Rather than seeking to gather and analyze all knowledge separated from practice and prior to implementation. Implementation needs to be designed to pay close attention to knowledge that is generated during implementation and the ability to act upon that knowledge.

I’ve previously discussed cognitive mediation here, but today I want to consider the foundation or the roots (etiology) of this concept in my thinking.

Marx considered labor as a form of mediation to explain how humans interacted with their environment (This was guided by Hegel’s version of dialectic theory, usually stated as thesis-antithesis-synthesis). Marx did not delve much into the specifics of how mediation worked, except as he used the idea to focus on the way that labor became subservient to capital, thereby alienating laborers.(See note 1.)

Vygotsky extended the psychological aspects of this view of mediation by analyze how language and concepts acted like cognitive tools that enabled humans to give meaning to perception, (He spoke of translating lower psychological functions like perception into higher psychological functions like meaning). (See note 2.) Mediation then enabled humans to interact with and modify their environment (or to perform labor). Vygotsky also noted that mediators are not usually developed by individuals out of thin air, but already exist in the surrounding culture and people acquire these abilities by imitation, instruction or similar means.

Gal’perin noted that all cognitive mediation was not equal. Tools could be improved to make labor more effective or efficient.

Hence my idea that it is good to be aware of the mediation you’re using. If your goal is to improve the performance of people’s labor, understand what mediators are guiding performance. Consider developing better mediation and passing it on through learning new ways of mediating, by changing mediators in work processes or by both methods. This approach may be able to improve performance far better than through increasing individual efforts (like boot strapping). The bottom line – If we are to fully enter into a “knowledge age”, we must understand how knowledge mediates to improve our practices and labors.

Note – My own personal opinion is that Marxist analysis is frequently very enlightening. But, considering the general failure of communism and central planning, Marxism generally fails to offer any viable alternatives ways to organize human activity.

Note – Vygotsky began as an enthusiastic Marxist, hoping that it would lead to the end of Jewish persecution. He died young from TB, but lived to see his ideas attacked because he committed the Stalinist sin of referencing western ideas, like those from William James or Jean Piaget. His ideas, although explicitly Marxist in their original intent, have generally been taken up by social cultural educational psychology (cognitive psychology that sees cultural as the place where cognition originates and with enculturation as important to cognitive development). He is generally ignored by Marxist theorist today. I believe it is because he focused on the mediational side and not on the alienation side of the Marxist equation.

Vygotskian education psychology places a high priority on mediational artifacts or cognitive tools; things like knowledge, concepts, criteria, schemas, etc . . .. These tools act as cognitive mediation and are instrumental to activity as subjects work on an object to produce an outcome.

Activity as Vygotsky's Unit of Analysis

I spoke here about how unity of the 3 elements and the central unit of analysis is the activity. Lets consider an activity example relevant to evidence-based practice.

A clinician (the subject) uses the idea of evidence-based practice (the mediating artifact) to examine routine aspects of their practice (the object) with the goal of changing their practice to improve their patience’s health (the outcome). If you find that evidence-based changes are not being made in a field, where would you look for a problem? Many analysis have implied that there is a problem with the subjects, they’re just not using the available evidence or that their knowledge based is deficient. I would say that it is much more likely that the solution can be found by developing an appropriate mediating artifact that can support clinicians in examining their practice.

This was the focus of Gal’perin, a prominate follower of Vygotsky. He said that not all (cognitive tools (mediators) are of sufficient quality and that the quality of development (like the development of evidence-based practice) is most dependent on the quality of the cognitive tools. Specifically, he thought that cognitive tools should be organized around and support the psychological functioning of the subject. So, what are the psychological functions around which you might organize the concept of evidence-based practice?

First, don’t focus on the evidence, focus on the practice and use a tool that brings evidence to a practice focus. An example might be a checklist used by a surgical team as they prepare for surgery. The checklist reflects the available evidence and allows the team to bring that evidence to their practice focus, but still allows their cognitive load for addressing important aspect of their practice.

Second, use cognitive tools to organize information and to orient evidence toward action. A research finding may represent important evidential information, but they are seldom oriented to practice in a way that naturally leads to action. An example is a network security assessment I developed. It reflect HIPPA security requirements (the evidence) in a series of 46 questions. The questions were structured not only to assess security status, to clarify an action plan that would improve the security status. This again would reduce the cognitive load needed to include an enormous amount of information in a short time span.

Vygotsky developed this idea of mediational tools or cognitive artifacts during the 1920’s, but with the increasing importance of knowledge and other cognitive artifacts, it has never been as relevant or important. Vygotsky was thinking mainly of children’s development, but his theory is also relevant to adults and their cognitive functioning in their work life.

My last post was not advocating that we abstain from all representations in education and assessment; that is simply not possible. What I am advocating is for a carefully evaluation of the representations we choose and how we choose use them.

In example: it’s common to engage in classroom training, based on representations of work actions, with follow up assessments based on recall of those representations. We are not really interested in representation recall, we’re interested in performance. Many recall items have limited validity for actual performance or at least this correspondance has not been evaluated. It is easy to assess if these representations can be recalled, but what do we care if simple recall has a limited impact on performance. The actual actions needed in the work setting frequently have limited correspondence to the classroom representations and as a result have limited utility.

A better way is to provide multiple ways to support work actions that can include classroom activities, but can also include informal social media tools, communities of performance, searchable web-based tutorial and onsite performance support tools like coaching and collaborative teams. These tools will likely make use of representations, but those representations will be more closely related to the actual actions required. With learning assessments and with educational programs, it is always better to choose representations that are more closely related to and valid for actual performance.

Another example is Liz Coleman’s (President of Bennington College) TED Talk regarding the reinvention of liberal art education. She proposes a more active program of education. Instead of learning about what ever subject is at focus, it is about learning to actively do. Rhetoric, design, mediation and other ways of doing education gain prominence because of the ability to actively do intellectual work; to act on challenges, not to recall words (representations) that are useful only for talk about challenges. Her great takeaway line: “There is no such thing as a viable democracy made up of experts, zealots, politicians and spectators”. What is needed are participants.

Learning is doing as doing is learning. The primary biological purpose of neurological activity is found in how it is associated with motor activity in ways that allow an organism to couple with its environment through movement (Maturana, H.R. & Varela,1992). The problem with many educational activities is in how they over emphasize representation and under emphasize the activity that is the true target of learning. Most classroom activities focus on manipulating symbols and leads to the recall of these symbolic representations. Problems occur in the gap frequently seen between representations and an inability ability to act based on those representations. What is needed instead is a holistic approach that creates a new portfolio of actions to replace an old portfolio (Zeleny, 2008). The outcome of learning activities, the test of learning if you will, should be the performance of new activities not the recall of representations. Zeleny gives an example of this type of thinking through his assessment approach to strategy. Strategy is not found in plans and statements. It is found in the action a company takes. If you want to know what a company’s strategy is, look at the actions they take and the structure of those actions. Do not look at what they say. Talking does not convey their strategy, acting does.

Workplace learning is moving beyond the classroom to a holistic approach to performance improvement. Emphasizing learning transfer is not going to significantly help performance unless it can recognize that learning is about doing, not in grasping representations of doing. Real learning is not learning about doing, it is learn to do. Classroom training may occupy a small part of any program, but most training and performance support must focus on the location where actions are performed and focus what is actually needed to support doing.