Posts tagged “methodology”

[from steve_portigal] Non-Jews Begin to Embrace Ketubah Wedding Tradition [NYTimes.com] – [Cultural appropriation of religious traditions as we continue to seek meaning through symbols] The decade of non-Jews discovering the ketubah coincides with three relevant social trends: the rise of Christian Zionism, the growth of interfaith marriage, and the mainstreaming of the New Age movement with its search for spirituality in multiple faith traditions. As a result, an increasing number of gentiles have taken up Judaic practices: holding a Passover Seder, eating kosher food and studying kabbalah, the Jewish mystical movement. “A lot of these things are grass-rootsy,” said Prof. Jenna Weissman Joselit, a historian at George Washington University, who has written extensively on Jewish popular culture. “They have to do with the growing popularity of intermarriage — openness, pluralism, cultural improvisation. And for those who are more religiously literate, they add another level of authenticity or legitimacy.”

[from steve_portigal] More Focus Groups for ‘Spider-Man: Turn Off the Dark’ [NYTimes.com] – [What is the meaning of using consumer research? Do we admire producers for being user-centered or do we decry them for being desperate?] The producers of the Broadway musical “Spider-Man: Turn Off the Dark” are offering $60 goodie bags to people who serve in focus groups that will respond to several performances. Two focus groups attended Thursday night’s performance, and four more are scheduled to be at Friday night’s show and the Saturday matinee. It’s not unheard of for Broadway producers to use focus groups, and the musical has used them before since preview performances began on Nov. 28. But these are the first since largely negative reviews of the show by theater critics across the country were published on Tuesday. OnTrack Research, a marketing and consulting firm, is coordinating the focus groups, and here’s the rub: participants only get to see Act I or Act II, not both. They are then asked to fill out surveys and join in discussions in a “V.I.P. room.”

[from julienorvaisas] Six scientists tell us about the most accurate science fiction in their fields [Mad Science io9] – [Brief interviews with scientists discussing where some of the real science resides in our science fiction. Great comments thread, including this one: "The other side of the coin is how has science benefited from science fiction stories."] Ronald Arkin, director of the Mobile Robotics Lab, Georgia Tech: "Realistic depictions of robots are pretty boring, so there's not much to say on what is accurate or not. No positronic brains, no running amok killing everyone and everything. I guess that's the fiction in science fiction. You watch enough videos of robots at real research conferences and it's hard to stay awake… Anyway, [one] comes to mind that is a bit more accurate than most: Hal 9000, in 2001, apart from his apparent psychotic episode, is a robotic system that people live inside. Current research agendas, in human-robot interaction, task planning, command and control, etc., could conceivably lead to such an intelligent system."

[from steve_portigal] Will You Try My Paper iPhone App? [Techcrunch] – [Stanford HCI student gets soundly criticized for seeking feedback on paper prototype with actual users! The drama – as often on the web – really takes off in the comments.] When I looked down at his hands, however, instead of an iPhone, he held a few pieces of paper with wireframe drawings in pencil. This was his app. I was supposed to pretend the paper was an iPhone screen and press the hand-drawn buttons as I shuffled through the flow. The idea is that you could point your camera at a magazine rack and get digital versions of the magazines, which you could preview on your iPhone and then purchase individual articles or the entire magazine. It made a lot more sense when he did it (see video). Now, there is nothing wrong with getting your ideas down on paper or paper prototypes to work out the kinks before you start coding. But you might want to wait until you have an actual working app on an iPhone before testing it out in the wild and asking for feedback from normal people.

Gathering insights by having people hand write their stories [DMI News & Views] – Asking people to tell us stories has repeatedly proven to be a rich and productive avenue for important insights. We ask people to tell us a story about a relevant event or experience. For instance, tell me about the last time you baked something from scratch. Or tell me about the last time you purchased a car. We try not to set too many rules or give too much guidance. We let them determine where the story will begin. This, after all, is what we are looking for. We ask for the story to be in writing—and ideally the story will be handwritten, if the logistics permit. We ask for the story to be as descriptive as possible—and we ask that the story be illustrated with pictures (hand drawn stick people or cuttings from magazines or from the Internet).

100 Records: Project turns on fictional jackets [SFgate.com] – Exhibiting as "100 Records", Sonny Smith, a San Francisco musician, artist and writer, commissioned nearly 100 artists from around the world to create the artwork for 100 45 rpm record jackets that represent more than 60 fictional bands and singers.

Storylistening for consumer insight – There are many ways of collecting stories but here are three that may be new to you:
* Anecdote circles
* Naive interviewers
* Mass narrative capture
Collecting stories is not about finding the one perfect story that describes a brand or a consumer experience. Rather it is about gathering a broad spread of qualitative data. Individually a story may be seen to be banal but their power lies in the cumulative effect of many stories.

Interpreting stories
* Experts
* Machines
* Participants

Story interpretation is best done by a range of groups (e.g. consumers themselves, a marketing department) that may have differing perspectives on the same situation. The most appropriate techniques often avoid direct analysis initially and allow different groups to immerse themselves in the stories to produce nuanced interpretations of the consumers' world.
(via DinaMehta.com)

Sony, B&N promise to rekindle rights for book owners – Boing Boing recently talked to Sony's Steve Haber, President of Digital Reading, about its flagship ebook reader, named the "Daily Edition." "Our commitment is that you bought it, you own it," Haber said. "Our hope is to see this as ubiquitous. Buy on any device, read on any device. … We're obligated to have DRM but we don't pull content back."

What design researchers can learn from hostage negotiators – Interesting to look at various collaboration and communication scenarios and unpack what's going on to define some principles that can be reused. Not sure how much new about design research is brought to light here, but the framing may make it more memorable or understandable. Always glad to see the emphasis on rapport, but I don't agree with their hostage-rapport approach as a one-size-fits-all method for design research rapport building. I also think they underplay the emotional levels that good design research can uncover. Beyond frustration with products, we hear stories about cancer, divorce, infertility, hopes, dreams, and beyond. All very charged stuff.

If you outlaw meep, only outlaws will say meep – Tthe nonsense word started with the 1980s Muppet character Beaker. Bob Thompson, a pop culture professor at Syracuse University, said he first heard students meep about a year ago during a class screening of a television show.
"Something happened and one of them said 'Meep,'" he said. "And then they all started doing it."

The meeps, he said, came from all of the students in the class in rapid-fire succession. When he asked them what that meant, they said it didn't really mean anything.

But meeping doesn't seem to be funny to Danvers High School Principal Thomas Murray, who threatened to suspend students caught meeping in school.

In an interview with the Salem News, Murray said automated calls were made to parents, warning them of the possible punishment after administrators learned that students were conspiring online to mass-meep in one part of the school building.

We’re very excited today to be posting our findings from the Reading Ahead research project.

Lots more in the deck below, but here’s the executive summary

Books are more than just pages with words and pictures; they are imbued with personal history, future aspirations, and signifiers of identity

The unabridged reading experience includes crucial events that take place before and after the elemental moments of eyes-looking-at-words

Digital reading privileges access to content while neglecting other essential aspects of this complete reading experience

There are opportunities to enhance digital reading by replicating, referencing, and replacing social (and other) aspects of traditional book reading

We sat down yesterday in the office and recorded ourselves delivering these findings, very much the way we would deliver them to one of our clients.

Usually, we deliver findings like these to a client team in a half day session, and there’s lots of dialogue, but we tried to keep it brief here to help you get through it. (The presentation lasts an hour and twenty minutes.)

It’s been a great project, and we’ve really appreciated hearing from people along the way. We welcome further comments and questions, and look forward to continuing the dialogue around this work.

We’ve been hard at work synthesizing the Reading Ahead data. There’s a great deal of writing involved in communicating the results, and sometimes it makes sense to develop a visual model that represents a key idea.

Here are several partial models evolving through paper and whiteboard sketches, and finally into digital form.

We’ll be finishing synthesis soon, and publishing our findings on Slideshare, with an audio commentary.

I started today by typing up all of the Post-it notes you saw in our recent blog post on Synthesis.

This activity created a 6-page Word document of bullet points.

The next part of the process is something I always find challenging: taking an incredibly detailed list of observations, particpant statements, hypotheses, and ideas; figuring out what the Big Ideas are (there’s a point in the process where many of them seem Big!), and putting those into a form that tells a cogent story.

First step: make a cup of tea.

Ok, then my next steps were:

Categorize all those bullet points

Synthesize those categories a bit further

Write down in as short a paragraph as possible what I would tell someone who asked me, “what did you find out?”

Then I went into PowerPoint, which is what we use when we present findings to our clients. I’ll continue bouncing back and forth between Word and PowerPoint; each piece of software supports a different way of thinking and writing.

I dropped my synthesized categories into a presentation file, sifted all of the bullet points from my Word doc into the new categories, and then started carving and shaping it all so that it started to follow the paragraph I had written. (I’m mixing cooking and sculpting metaphors here.)

I printed out the presentation draft, and laid it out so I could see the whole thing at once.

Steve came back from a meeting and I asked him to read over what I’d printed out. He started writing notes on my printouts, pulling out what he saw as the biggest of the Big Ideas.

We talked about what he’d written, which led to an energetic discussion in which we really started to breathe life into this. Tomorrow, I’ll start the day by iterating the presentation draft based on our conversation.

Synthesizing field data into well-articulated, data-driven patterns, themes, and opportunities is a big part of our work, but it’s an aspect that generally has less visibility than the fieldwork.

An essential early step in the synthesis process involves going back over the fieldwork sessions. An hour or two-hour interview creates an incredible amount of information. By going back into a record of the interview, we make sure not to leave anything significant behind.

We go through and make notes on interview transcripts (done by an outside service), watch videos of the sessions, and look over photographs, sketches, maps, and participatory design pieces.

Annotated interview transcript

We made a bulletin board of the people we met, so they’re ever-present while we’re working.

Yesterday we came together to share the points we’d each pulled out. We present each interview, like a case study, to the team. Sometimes it’s just us, and sometimes our clients join us for part of this process.

While one of us presented, the other captured the essence onto Post-its. We had a lot of discussion and debate while we did this, pulling together multiple viewpoints.

When we were done presenting the interviews, the board looked like this:

Our next step is to take these notes and start grouping them. We’ll look at different ways the information can be organized, and from there, will start refining our work and writing it up clearly and succinctly into a report.

In addition to our in-person fieldwork, we often ask research participants to do some kind of task on their own. In past projects, these assignments have included a range of activities, from workbooks and journals with specific daily prompts, to fairly open-ended photo diaries.

For Reading Ahead, we asked people to do a short photo diary, and send us five or so digital pictures (before the interview session) that would help us get a sense of how reading fits into their lives.

Diaries like this accomplish several things. They give us access to a person’s world beyond what we might be able to get in just a face-to-face meeting. We’re able to see what they do in more locations, at more different times, and in more situations.

We probably won’t be there to see someone actually reading in bed before they go to sleep at night, but they might well ask a family member to take a picture of it for their photo diary, as Tracy did for this project.

Diaries also help us build rapport more quickly with the people we’re meeting, by giving us a common set of stories to begin the conversation with. There’s often a bit of back and forth dialogue between us and the participants during the assignment as well, which helps us establish a relationship.

Having some shared knowledge and possibly interaction prior to the interview means that when we are face-to-face, we can jump right in with that person at a deeper level, which can free up time in the interview session for exploring more areas of the research topic.

When you look at the pictures people sent us for this project, you’ll see that they’ve responded in a variety of ways. Some focused on objects, some took pictures of other people, and some photographed themselves or had other people photograph them. It’s useful to see the different ways people interpret the topic area, and the connections they draw. It helps us understand how each person sees the world, and can point us to additional lines of inquiry.

Tracy and her younger son thinking about possibilities for books and reading devices

Our fieldwork sessions often include a piece in which we ask participants to brainstorm and fantasize about the future.

In an earlier post, we talked about the simple models we were building for the Reading Ahead interviews.

Book and device models for participatory design activity

We wanted to put something in people’s hands to help them show us what the “book of the future” and “reading device of the future” could be and do. (This fieldwork approach borrows from participatory design.)

We’ve had clients come out in the field with us and say after an interview, “That person didn’t give us any ideas,” so it’s important to clarify that we don’t expect this kind of activity to directly produce marketable ideas. Rather, it gives people another mode for expressing themselves, and it’s great for helping them communicate things which may not always be easy to verbalize, like:

Their desires

What they think should exist

What problems they are trying to solve

What seems acceptable and what seems outlandish to them

Preferences and in what ways they would like something to be different

Chris uses the device model to help express his thoughts about navigation

Often for us, the very act of making the props for an activity suggests new ways of using them. In this case, while making a blank cover for the “future book” model, we realized that we could also make a blank inner page spread.

Holding the “book of the future” model

As it turned out, this meant that when we were done with the sessions, people had created very nice book models for us, with a cover and inner spread.

Part of the preparation for each interview session was to get the models ready with new blank paper. Here I am on the trunk of my car, prepping the models before an interview in San Francisco.

Now that the fieldwork is done, we have a great collection of models made by the people we interviewed.

Artifacts from participants’ “future book” ideation

The last section (copied below) of our Topline Summary synthesizes some of what we gleaned from this part of the fieldwork. These are just quick hits; we’ll develop any themes and recommendations that come out of these activities much further in the analysis and synthesis phase of the project.

Excerpt from Topline Summary: Participant ideation about the “book of the future” and “reading device of the future”

NOTE: The first thing a number of the participants said when asked about what the “book of the future” could be and do was that it’s pretty hard to improve on the book-it works very well the way it is. In addition to all the qualities already mentioned, books are

Instant on-off

Durable

But people did have ideas. Here are some of them:

Interactive

Put yourself in the story

Leave the story for more information

Choose from alternate endings, versions

Size-shifting

Able to morph from bigger size for reading to smaller for transporting

Retain the book form while adding functionality

Book form with replaceable content: a merging of book and device, with a cover, and page-turning but content is not fixed-it can be many different books

As soon as possible after concluding fieldwork, we write a Topline Summary, in which we capture our first impressions and the ideas that are top-of-mind from being in the field.

We’re always careful to be clear about what the Topline is and isn’t. There’s synthesis that happens from the fieldwork experience itself (which the Topline captures), and synthesis that happens from working with the data (which we haven’t done yet).

In the Topline we go a step further than the field highlights and start to articulate some of the patterns we think are emerging, but these ideas may change once we do a detailed analysis and synthesis of the data we’ve gathered.

In a client project, we’ll have a discussion with the client team around the Topline Summary. We encourage members of the client team to come out in the field with us, and the Topline discussion is a great opportunity for everyone who did so to share their experiences and tell stories. The Topline discussion is also a good time for our clients to let us know if there are any specific directions they want us to pursue as we analyze and synthesize the data we’ve gathered.

We’ve now finished our fieldwork for Reading Ahead. We conducted six in-depth interviews, with photo diary and participatory design activities (more in our next few posts about these methods).

Here’s our Topline Summary:

Portigal Consulting: Reading Ahead Topline Summary

Reading is not just a solo activity; there are significant social/interpersonal aspects for many people

Recommendations, book clubs, lending

Books facilitate the interpersonal aspects of reading

Can be easily lent or given away

Given as gifts

People can use a book together: parents and kids, showing someone a passage or illustrations, etc.

Books engage the senses: they are tactile, visual objects, with specific characteristics like smell and weight

Become carriers of specific memories

Develop a patina that carries meaning

An inscribed book becomes a record of an event, interaction, relationship

There is an art/collector aspect to books (which is absent in the Kindle)

First editions

Signed copies

Galley proofs

Typography

Pictures and illustrations

Quality of paper, printing, etc.

Books say something about a person

Others can see what you’re reading; like clothes, etc., this carries meaning

“Looking at someone’s bookshelves when you go to their house” (Jeff)

When people give books as gifts they are deliberately communicating something about the relationship, the event, themselves, and the recipient

Books can create a physical record of someone’s reading activity

Chris used to line up all the books he had read to get a sense of accomplishment

Annotations, bookmarks, tags all convey the reader’s personal history with that book

Books are easily shared

Pass them along to others

Donate to library

Sell or buy at used book store

Borrow from the library rather than purchasing

How books are stored and organized carries meaning

Emotion, sense of pride, expression of personality, record of engagement

Erica organizes her books by how the content/type of book feels to her: “dusty” classics, modern classics, etc.

Julie’s extensive shelves are organized alphabetically to reinforce the idea of library

Libraries and bookstores provide specific experiences

As a little girl, Erica visited different libraries with her Mom. This was their daily activity, and Erica retains strong and specific memories

Julie and her housemate recreated a library atmosphere in their home

A quiet, comfortable space

Good lighting

Alphabetized bookshelves

A unified décor

For Jeff and others, spending time browsing in a bookstore represents having leisure time

The Kindle

For people whose love of reading is bound up in their love of books, the Kindle loses much of the reading experience; it is only a content carrier

Julie has a history of wanting to read on electronic devices as well as from printed books, so to her, the Kindle is a big evolutionary step from her old Palm, the iPhone, etc.

For Erica, the Kindle signifies “computer,” so it does not let her “unplug” from the fast-paced connected lifestyle that books provide a refuge from

Several people described the kinetics of page-turning as an important aspect of reading books that is absent in the Kindle

Books afford ways of navigating content that the Kindle does not: flipping, comparing non-sequential pages, looking at the recipes at the end of each chapter, etc.

Peter finds it frustrating that when he buys a Kindle book from Amazon, he can’t share it. When he started working in an environment where people were passing books around, he went back to reading printed books

Participant ideation about the “book of the future” and “reading device of the future”

NOTE: The first thing a number of the participants said when asked about what the “book of the future” could be and do was that it’s pretty hard to improve on the book-it works very well the way it is. In addition to all the qualities already mentioned, books are

Instant on-off

Durable

But people did have ideas. Here are some of them:

Interactive

Put yourself in the story

Leave the story for more information

Choose from alternate endings, versions

Size-shifting

Able to morph from bigger size for reading to smaller for transporting

Retain the book form while adding functionality

Book form with replaceable content: a merging of book and device, with a cover, and page-turning but content is not fixed-it can be many different books

As we get into actual fieldwork this week, we’ll be using (as is typical for us) a mashup of techniques. In addition to interviewing people, we’ll be watching how and where they read books or Kindles. But we’ll also want to get into a discussion of future possibilities. Reading Ahead is not about evaluating existing designs but instead getting inspiration and information that can drive future designs. So we don’t have anything to test, but that doesn’t mean we can’t use testing-like (“What do you think of this?”) activities. In this case, we’ve built extremely simple representations that suggest book and digital reader.

Dan building a thing-to-hold for this week’s interviews

The idea (and we’ll learn what happens once we do a couple of interviews) is to have something neutral to put into people’s hands and let them gesture, sketch, or otherwise perform, so the activity of discussing the future isn’t just a verbal one. We’re going to ask people to draw on these props, and then we’ll have an artifact created by the participants themselves. Those artifacts can be compelling, and can also be a much more impactful symbol of what took place in that part of the interview.

We’ve never used this exact prop before, but in just about every project we’ll come up with a range of tools to use in the field. Our interviews are very open-ended so we could use either of the props to explore any emergent themes, depending on what we use them to talk about (From “How would you hold this?” to “Well, if it did come with your Happy Meal, draw what you’d expect to see on the main screen.”

Before we go out in the field we write an interview guide (or field guide), a list of all the topics we want to cover.

Interview guides end up being very linear and structured, but the interactions we have in the field are looser and more conversational. We’ll let the way we pose our questions flow much more from what the person we’re interviewing is saying than from the sequence and phrasings of the interview guide.

Even though we know this will happen, we’ve still been working hard to hash out our questions ahead of time–even the basic ones. It’s like John McLaughlin said about jazz improvisation: you have to learn all the chords so that you can forget about them.

The interview guide is also something for everyone (including our clients) to look at, to make sure we’re all on the same page as we head into fieldwork.

Design Research Methods for Experience Design – Triading is a method that allows a researcher to uncover dimensions of a design space that are pertinent to its target audience. In triading, researchers present three different concepts or ideas to participants and ask them to identify how two of them are different from the third. Participants describe, in their own terms, the dimensions or attributes that differentiate the concepts. Participants follow this process iteratively—identifying additional attributes they feel distinguish two of the concepts from the third until they can’t think of any other distinguishing factors.

The benefit of this process is that it uncovers dimensions of a particular domain that are important to the target audience rather than the researcher or designer. For example, participants may describe differences in groups as “warm” versus cold” or business-like” versus fun.” Designers can then use the most relevant or common dimensions as inspiration for further design and exploration.

Mapping Oakland – Mapping Oakland is a research project aimed at mapping people’s perceptions of neighborhoods and urban space within the City of Oakland. Mental maps have been used in geography to understand individual perceptions of space and place for sometime. The method has proven useful in helping geographers understand how people perceive elements within the landscape for navigational purposes and to understand the cultural value of spaces. This web site provides citizens throughout Oakland access to a survey that measures how people perceive and use public open space in the City of Oakland.

How ethnic groups change Oakland neighborhoods – When Robert Lemon, a UC Berkeley landscape architecture grad student, was a community planner in Columbus, Ohio, he noticed that despite the car-oriented landscape, residents of the city's Latino community, for the most part, liked to get around on foot and bicycle and, as a result, were bending the neighborhood to their collective will. Taco trucks and open-air produce markets popped up in vacant parking lots on one of the city's main shopping thoroughfares. The bicycle was a key mode of transportation even though there weren't dedicated bike lanes, and colorful murals appeared on the walls of large buildings. The neighborhood had the feel of small-town Oaxaca, the Mexican state from which many of the city's Latinos hailed.

In California, he found similar changes occurring in Oakland's Fruitvale and Chinatown neighborhoods. He is conducting a formal survey as part of a fellowship & has gone through Oakland's diverse neighborhoods, walking up and down the streets asking questions.

Is ethnography more suited to shedding light and depth on peoples relationships with some products or services over others? Are there times when it’s completely unnecessary?

That’s a two-parter! I’ll take a shot at this all, but I hope that others will jump in to clarify, correct, disagree, or whatever.

Are there times when ethnography is completely unnecessary?! Absolutely. But your second part is about time and your first part is about category. In terms of time, in terms of the process of developing a product or service, there are different types of research that may be more or less appropriate at different points along that process. There’s no overall answer, for me, it always depends.

For example, who is the organization? What do they know about the category, or the customer, or the channel, or the market, at this point? Is it a new product in a new category, or a redesign of something they’ve been doing before that they are improving? These are all facets of “what do you need to know more about?” I guess.

The best times to do any sort of customer research (and sorry if this is obvious) is when you can take some action with the results. If there are decisions to be made and they need informing by a user/customer perspective, then try to do that research ahead of time.

Depending on how you define ethnography, you may wish to see it only as a discovery process, as something that happens early on to define needs so that you can create solutions to match those needs. But putting solutions – prototypes – back into that conversation as a way to further validate the needs or dive deeper into them is a great way to go. You can read a case study of ours here that relates how an early prototype was taken into homes in order to have a more meaningful discussion about what the needs were and how this product could address those needs. You can always hold back your artifact for some portion of your session; that’s a standard technique we use.

As far as categories of products and services go, I don’t see that as a big dividing line in terms of the usefulness of research (and I’m speaking about research in general, whatever methodology you want to consider, some way of bringing customer perspectives into your design and decision process). I’d point again to the organization and its process as the crucial gating factors.

And

Also, are there precedents for ethnographic research in Asia?

Wow – two opportunities to cite DUX conference papers in one blog post. This PDF uses research in Japan (conducted by us, and by Adobe, over several years) as a template for considering research in global locations.

I think there’s two flavors here: Global companies coming to Asia to understand these markets, and Asian companies doing for their own domestic markets something like ethnography. For the first, absolutely. Japan, China, and India have all been high-profile destinations for user research (and in case anyone wonders, we are very keen to go back and do some more). For the second, I hear more about India, specifically, with an increasing drive by Indian companies to understand how their products and services will be received. A colleague in India does a regular audit of teenager attitudes for MTV India. I don’t know in much more detail the market for this sort of thing in any part of Asia, only the different anecdotes I’ve collected from various Asian colleagues.

Should I also re-plug CultureVenture at this point? For companies looking to get a handle on these other markets, this is our service to drive inspiration through immersion. Facilitated hanging out, if you will?

How’s that Charles? I hope you’ll add some follow up questions in the comments.