I don’t know where this came from, but I think that the synthesis of transformational experience, JS Brown’s ‘questing disposition’ and especially Rappaport (God rest his soul) is shear genius. The following is from a review of Rappaport’s book by Mary Catherine Bateson:

Rappaport is . . .describing the kind of ecology of ideas and actions that might include and sustain religion (or secular rituals) as an integral part of life. . . What is needed is not new theology (though some tune-ups might be helpful) but new forms of practice and social engagement. We can talk until we are blue in the face, but that may do more harm than good, creating new polarities; what we need to do instead is to march or dance or sing, as in the great civil rights demonstrations of the sixties that forged new convictions and new unity.

I think these forms of practice and social engagement can come in many different forms and can be short lived or last for centuries, but they all must exist and tap into an ecology of ideas and actions that are at a scale that is much larger than any single individual or any single designed event.

My first thoughts on Browns ‘questing disposition’ reminds me of the institutional university. Many of the best are full of rituals and lore, and their ancient stature towers above all who are there (think: Go Crimson). But the way they do things and their ecology of ideas, just doesn’t fit this new world. We go to university, graduate, take a job and hear from them in the form of donation requests. Our educational rituals and our educational relationships need to be lifelong, just like our learning. There will come a time in the year 2020 when I need will need knowledge and practice from my university, except that the course I will need hasn’t been thought of yet. We need university rituals that go with us into the world, and extend beyond rooting for your team in March Madness.

A second thought. I have long thought that religion must have served some important purpose to be so widespread and this fits with Rappaport’s basic idea (at least as presented by Bateson) that ritual and religion co-evolved with language and that it did this in order to counter some of the destructive things that can be done with language (hence Bateson’s comment above that talking can “do more harm than good”). And Bateson’s prescription also seems real to me; that religion’s problems are rooted not primarily in theology, but in the need for new practices and new forms of social engagement. Now I admit, there are some theologies that I do have some problems with, but I believe that Bateson would say that a lot of these problems are just me letting my words getting in the way of practice and social engagement.

A third thought. Ritual, as Bateson talks of it, can be seen as a form of distributed cognition or maybe as a form of the distributed unconscious. Yes, I believe that just as there is much cognition that is below our conscious awareness, much of our rituals serve a similar unconscious cognitive role.

Tony Karrer has re-posed a question originally asked by Peter Drucker: how do you increase the productivity of knowledge (concept) workers? Since more jobs depend on knowledge today, we might just as well ask, how do we support productivity in the 21st Century. I suggest that significant organizational learning is the best measure of productivity in a knowledge intensive environment. The following 4 things could be measured as evidence of significant learning:

An increase in organizational capabilities, and

In innovation.

The development of a maturity framework with evidence-(standards)-based practices for improving the quality of repeatable or routine processes.

An increase in soft skills relating to non-routine and networking processes, which could be measured by the extent and the strength of an organization’s internal and external networks.

I believe that organizational learning is facilitated by individuals, but I would not consider it synonymous with individual learning. I’m not sure exactly how to measure individual learning, which might vary with different contexts. I do believe that employers can focus mostly on the measurement of organizational learning and maybe on individual contributions to organizational learning.

There is certainly enough here to keep me thinking for some time. Thanks for the question Tony!

Are your measures valid across a range of concerns? Improving validity will lead to improved actions, better frameworks for acting and ultimately improved performance. In example:

The turn of the century saw an increase in the expectations tied to measurement through such phenomena as “No Child Left Behind” or SAT test prep classes. This has begun to change as colleges put less emphasis on SAT scores and I believe we’ll soon see similar changes in high stakes graduation tests. Two observations:

While high expectations pose difficult challenges for assessment, most of the problems that resulted in less use of assessment are in the expectations placed in specific tests not in the capabilities of assessment in general. It’s a hermeneutic problem. The meaning of test scores was much narrower than were the expectations for assessment; a mismatch between the meaning that was required and the meaning that the test could supply. From a narrow psychometric perspective involving external validity, these test were valid, but from other perspective (structural or consequential validity – see the previous post) they are found wanting.

People will still act and those actions will still require assessment and those assessments will still be made. They will just be more casual, less observable, and even less valid that those made by high stakes tests.

Most actionable situations require a range of assessments that are valid across a range of validity concepts. Just because some are less empirical or more qualitative does not mean they should not be considered in an appropriate mix.

My PhD was in education psychology and most of my classes occurred in the mid to late 90s. The paradigm wars were winding down, but there was still a noticeable split between hermeneutic social constructionists* and the psychometrician. My nature is to want to synthesize, often leading one to walk in two worlds. Too what would I be drawn; a hermeneutic account of psychometrics of course.

I was investigating dissertation topics around disability. The split here was conveyed as between old psychometric ways of conceiving of disability and new socially constructed accounts. An advisor made a casual comment that my concerns seemed to be about validity and it seemed insightful. Yes! The problem was that existing measures were validated by psychometric models that did not account for the hermeneutics of identity construction or for the consequences of resulting identities.

I started my investigation by reading Samuel Messick’s chapter on Validity in Educational Measurement (3rd ed.). What I read was Messick’s attempt to address hermeneutic aspects of measurement from a psychometric perspective. What was important in measuring, is the meaning you derive from the data and the associated implication for action. First, there are only 2 ways to think about invalidity:

Construct Under-representation; The construct you are interested in is larger than what your assessment is able to measure.

Construct Irrelevance; you are measuring things that are irrelevant to the information you need to take action and lead to either false positives or false negatives.

Messick would later write about six categories of validity concerns. I take these categories to be a framework for how to think about or find meaning in measurement. They are 6 different way of looking for under-representation or irrelevance:

Content – Is there evidence that the scope of the content appropriate and representative of the construct.

Substantive – Is there a theory for the processes and tasks being performed and is there empirical support for the theory.

Structural – Is there evidence that the assessment faithfully reproduces the tasks or processes in contexts or in the natural settings to which you are trying to extrapolate.

Generalization – Has the assessment been shown to apply to many different groups, contexts and over time. While this may not reduce validity in specific situations, it would indicate to look much closer at the situation your in.

External – convergent or divergent criterion evidence.

Consequential – Is there evidence that your actions are improved by the assessment and that it is fair and free of bias.

* Note – I have no interest in most philosophical discussions of the beliefs of social constructionist or realists. For me, SC is mostly about the ways that things and people are thoroughly effected and affected by the pervasiveness of language and its accompanying hermeneutics. Not only is there no denial of reality, the current trend is to highlight the embodied nature of our living even as it is totally inhabited by hermeneutics. I fall back on pragmatics, not because it is defensible, but because it is a way to go on. Most other discussions are about drawing boundaries that are just too fluid to nail down in a convincing manner.

For the past several years the Carnegie Foundation has been working with a group of California community colleges . . . for different ways to think about and conduct professional development.

First, opportunities for teachers to grow and develop must be sustained over time.

A second principle is the importance of collaboration.

The third defining feature is a focus on evidence about student learning. . . . information is at the heart of powerful feedback loops. But an important lesson . . . is the power of viewing classroom data through the lens of larger institutional trends and patterns.

While reading dissertations from MIT’s DSpace (Thanks to Richard Hoeg for the link) I followed a reference from Seo & Creed (2002; Academy of Management Review, 27 #2). This is a follow-up on my previous post and shows some reasons why there are multiple ways that disfunctional behavior can be supported in organizations. They discuss four reasons why contradictory impulses can be seen in organizations:

Legitimacy that undermines functional inefficiency: There are certain practices and beliefs that become legitimate and require conformity within organizations or even entire industries. But, these standard practices can be confining to technical practices that need to remain fluid and change according to the context if they are to achieve maximum efficiency. Over time some crystalized organizational structures can hamper practices substantially.

Adaptation that undermines adaptability: This concern is very similar to the previous one except to recognize that many crystalized forms are originally adaptive. In one example they discuss the way individuals develop schemas to deal with complexity, but that they may resist changing that schema even when it is no longer functional.

Intra-institutional conformity that creates inter-institutional incompatibilities: “Organizations tend to incorporate all sorts of incompatible structural elements, practices, and procedures. . .” This can occur because of different competing ideals in society where capitalism, family values, government bureaucracies, liberal democratic ideals and Judeo-Christian traditions can have “contradictory “central logics”.

Isomorphism that conflicts with divergent interests: There are social and power arraignments with divergent interests within any organization. This means that all organizations are political. Politics can often get in the way of function and efficiency.

Ok; So what’s the suggested responce?

Decouple – Allow and explore how individual teams can explore diverse ways to self-organize and decouple from institutional structures and ways of acting.

Praxis – Politically recognize and support teams that are engaged in serving their customers over internal power struggles.

Geetha Krishnan asks us a foundational question that is central to education, management and any other field related to psychology and concerned with behavioral skills. “Can (behavioral skills) really be taught to an adult? More accurately, can an adult change behavior through training?” I would answer yes; with substitute behavior, with appropriate mediation, with changing the functional structures supporting change and with a long term view and plan. In answering, I will reference the examples presented by Geetha in his blog post

Before beginning, let’s take Geetha’s question one more step. Trainers, coaches, and others involved in leadership, professional development, training or performance support are responsible for facilitating behavioral change. The question is not if, it’s how? I think it starts with a deep understanding of the person: their motivations, beliefs and desires, as well as with the expectation of the culture and community in which they are embedded and it continues as a partnership. You may develop curricula materials and training programs based on the general needs of the population you’re targeting (like the inspirational program Geetha references), but I think facilitating behavioral change is person specific, long-term and personal. This may not be an exclusive list, just a start.

Substitute Behavior: One of BF Skinners insights is that people are active. You can’t stop behavior, people will still be active, you can only change behavior. In Geetha’s initial discussions the question is; how do you get people to talk less in meetings and to substitute active listening and recording behaviors instead. In addition to thinking about what you do not want people to do, give some thought to what else they could do instead.

Mediation: The psychological insight of Vygotsky and Leont’ev was that people can regulate their behavior through cultural mediation. That is, we can use language (inner speech) and ideas to regulate our behavior and these ‘mediators’ originate in the social world around us. In Geetha’s example, there was an original training session that provided the ideas and justification for changing the way a supervisor was interacting with the people under him. These ideas (mediator) were reinforced by the person’s peer group and likely by the corporate hierarchy. What we don’t know from Geetha’s example are the cultural community and social influences on this persons behavior. I think Geetha correctly identifies that there are tools that help us to change behavior, there are hows and methods for making this change and there are whys and reasons we are motivated. What I suggest is that these are all mediators and that they are formed in the social world, but the ones discussed are not the only ones. There are many others, sometimes contradictory ones that are out there and it takes a deep understanding of the social situations to understand what is supporting behavior in a person’s social milieu. It is not hierarchical either. Pressures from subordinates can be as important as those from bosses.

Structure: I believe that behavioral changes that are functional are the most likely to be maintained. What do I mean? Consider a person who listened more and recorded what he learned during client meetings. Suppose he would then follow up by regularly summarizing what he learned and would make that learning a part of his response to his clients. If that improved his performance, especially as it is seen in the social milieu, it would be a strong motivating factor in maintaining his new behavior. I believe people can do things out of habit. Making the new behavior a functional improvement is one why to break out of a habit.

The long view: Prochaska, DiClemente & Norcross (1992) speaks of a reoccurring cycle when trying to change behavior. (They were speaking of therapy for addictive behavior, but I think it is applicable to many types of behavioral change.) Behavioral relapse is common. It does not mean that behavior cannot change, just that the cycle of change must be repeated. For any change that is important, I think we must be committed to a long-term view with repeated efforts. Any change process should anticipate relapse and the need for repeated efforts to change should be considered normal.

This rather long and wonkish post should be considered the first draft of an idea in response to an important question. It is almost the history of psychology. I am sure that much addition thought and revision needs to occur here. I certainly welcome any comments.

The first one struck me as an illustration of distributed cognition; how we use external aids to add structure and extend our thinking.

Write to find out what you think. Your thoughts will be somewhat muddled until you get them in writing. Don’t go around and around in circles internally until you know what to write. Write before you know what you’re going to say.

Learn to tolerate some degree of confusion, and yes, complexity in your early writing. I’ve noticed that many academics get panicky when their first draft is a mess. It’s supposed to be a mess! Have faith in the revision process.

I really do need to get something down “on the page” before I really understand the implications of what I’m thinking. It supports the limitations of short-term and working memory, but more than that too! It’s also the back and forth / give and take revising process. I’m revising my thoughts and ideas while I get the first words down. This is one of the main reasons I blog, to workout ideas on the page and over time.

Good thinking, writing and communicating should go hand in hand. I also think that there are no principled breaks in the chiastic relationships between thought, writing and communicating. I think there is a common sense that academic writing is for scholars and not for the rest. Now in one sense, academic are writing for other academics and thus their writing serves an instrumental purpose, but good writing should also be able to serve a broader purpose. It should be able to communicate, and inspire good thinking for non-academics.

Why are non-academic not exposed to good thinking and why are academics not writing in ways that better influence practitioners:

I think there are low expectations for non-academics. To anyone familiar with the literature on teacher expectations, this should send up red flags! Its is easy to downplay potential via expectations.

I think academics get too caught-up in the need for complexity that serves only disciplinary vocabulary and categorization schemes, not the underlying thinking. Look at the current economy / finance mess and those complex derivatives. It’s looking more and more like pretty simple fraud that people perpetrated on themselves and on the rest of us by using complex vocabulary and mathematical formulas to cover-up what was basically simple. Thought can be complex, but there is a lot of complex writing that shouldn’t be.

I shouldn’t be ranting, not with MY dissertation, just working to try getting better.

George Siemen’s blog looks at the report Clickstream Data Yields High-Resolution Maps of Science. He notes; “The data is current provided as images. It would be useful to navigate the resulting “map of science” in an interactive application”. When this comes-to-pass, it would really represent a powerful web 2.0 app and a data tool for ideas.

“First, collective intelligence tends to be most effective in correcting individual biases in the overall task area of (idea) generation” and

‘Second, because most applications lack a strong feedback loop between generation and evaluation, “companies should consider deploying such feedback loops with greater frequency because the iterative process taps more fully into the power of a collective.”’

This could really be realized if there were two more developments like clickstream data.

If disciplinary research agendas would become more self-organized by being more connected to the collective 2.0 world, because research is just such a feedback loop that Bonabeau is calling for, and

We had a better aggregator for research results. There is too much research and knowledge being generated to use, at least in a way that taps into collective intelligence. This would make the leap from idea generation 2.0 to evaluation 2.0.

Finally Andrew Filev in the Project Management 2.0 blog (referencing Seth Godin) says that collective intelligence still needs leadership (as in a leader of the project tribe). It seems like this is a re-introduction of bias back into the system, but maybe some bias can be productive for getting things done. I’m not sure.

A recent HBR articletouts the benefits on ethnography at Intel. (Ethnographic Research: A Key to Strategy. By: Anderson, Ken, Harvard Business Review, 00178012, Mar2009, Vol. 87, Issue 3) There are many types of measures in the social sciences. Each has its own strengths and weaknesses and each has a place in your measurement repertoire. But, as this article points out, if you limit your view of measurement and science (or data collection and how you are able to deal with different kinds of datum) you will ultimately lose out.

Some people may not like this kind of viewpoint. It tends to broaden one’s field of vision and many people like to stay narrow and focused. There is a time and a place for narrow and focused, but there is also a time and place for broad. Reminds me of something Martin Buber wrote (paraphrasing)

Only a fool give someone three choices. The wise man gives only two choices, one that obviously good and one that is obviously evil.

I do hope I am correct in reading this sarcastically. If this is indeed the knowledge age, we need lots of people who can deal with 3 and more choices on a regular basis.