Guild Purpose Coding: Attempts and Thoughts

I have been working recently on importing data into SPSS from the first part of my April survey on World of Warcraft motivations. This has been a fairly straightforward process for the most part. The exception is the last question about the respondent’s guild type and purpose. The question was presented as the following:

In a short sentence (140 characters), describe the primary purpose of the guild in which you spend most of your time, or enter “no guild”.

Example: I’m in a social guild that believes in random acts of kindness. We love to dance but we also raid end-game content with other casual guilds.

I am not happy with how the coding for the type and purpose is going. When I created the SPSS codebook for that part of the survey, I initially broke it down into two parts: a type and a primary purpose. The type represented a breakdown primarily between “social” and “hardcore”. The purpose could be a key activity or a purpose. The divisions were chosen based on an an initial review of the received responses (see Table 1).

The “No guild” choice is self-evident. “Not codable” was intended for responses like “Forks” or “I am an herbalist in my guild”. This was usually where the respondent had not answered the question correctly, so no type or purpose could be determined. “No identifiable activity” was intended for responses like “Social”, where no obvious activity was ascertainable.

Andrew and I independently coded the 51 responses according to type and activity. We were allowed to use general knowledge about World of Warcraft but not specific knowledge about any guilds or people if identifiable somehow from the response. The response itself had to dictate the type and purpose.

We almost completely agreed on “type”, but we disagreed by about 28% on the “activity”. If an answer mentioned raiding and that seemed more key than anything else included (or nothing else was included), I coded it as “raiding”. Andrew often coded it as “fun” because he felt the raiding was deprecated or a lesser part even if it was the only thing mentioned. So, for example, if someone said something like “I’m in a social guild and we also do some raiding”, Andrew coded that as “fun”, because the “also raid” was a lesser thing, whereas I would have said “raiding”.

To fix that, he suggested that we add a category that indicated the amount of raiding. I would then have “raiding” and “some raiding”. That seemed sensible, but then left the question of what the “primary” purpose of the guild was if “some raiding” was ancillary. While the survey specifically asked for the primary purpose, but many of the responses, likely because of the poorly chosen example, included one or more activities. When coding, that meant the coder had to pick the one activity that seemed most prominent.

A colleague, in discussion about the first attempt at coding, suggested going for as wide a range of activities as possible before cutting back. I had, for example, initially omitted “performing random acts of kindness”, which occurred fairly often, as an activity. She thought that should be added as well as “social.” Social” as an activity was omitted because I was using the “social” designator appearing in responses to code for the guild type. However, in retrospect, it occurred to me it would be even more appropriate if I changed the guild type to be as interpreted by me based on my assessment of the respondent’s answer; the question did not ask the respondent to specify their guild type but to describe the purpose. I need to stay true to the question’s intent.

That does mean that I need more formal definitions of “social”, “hardcore”, and “casual” then to use for the coding guild type. It also means there is a question about how many activities to code, given that very few responses only mentioned one purpose or activity. While the question did ask for a primary activity, I do not have a problem per se with coding for more activities. The questions are then: how many and should order be significant?

Response

The guild I am part of is primarily a social guild but we also get involved in raiding.

It a social guild with occasional raids

Roleplaying, adventuring, helping each other and gernally [sic] having a good time

[guild name] As Crafter

I’m in a social guild. We have a great sense of humor

I’m in a social guild that believes in random acts of kindness. We love to dance but we also raid end-game content with other casual guilds.

Table 3: Some sample responses for guild purpose.

Open Issues for Comment and Resolution

Is the second draft draft more sane in terms of how type and activity are determined in general?

What are some existing formal definitions of guild types that can be used?

How many purposes or activities should be coded?

If multiple activities or purposes are coded, is the order significant? That is, should I assume that the first thing mentioned is the most important, the second, the next, and so on?

Any opinions or help are appreciated, as this type of analysis is new to me.

6 comments on “Guild Purpose Coding: Attempts and Thoughts”

Is it important that you identify a single purpose for each guild in order to answer your research questions?

Could you not add a coding category for 'Multiple Purposes'? This would avoid shoehorning lots of Guilds into categories where they don't quite fit – and it might help you to identify significant differences between single-minded Guilds and those that are more multifaceted.

I could add a category for "multiple purposes", but that wouldn't then say very much, would it, other than to differentiate between, as you say, multifaceted and single-purposed guilds. I'm not set on identifying a single purpose. The only reason I took that approach originally when developing the coding was because the question specifically asked for a primary purpose. As you can see from the responses, though, that's not what I got. That's partially my own fault because of the sample answer given. Mea culpa!

I think I can accomplish your distinction if I allow multiple codes for a given respondent. If there is only one code, then, in the mind of that respondent, their guild for them was primarily for one thing. If there is more than one, then it's more multi-faceted. That would still leave the questions of how many activities to code for (all of them?) and how to decide the order or importance (order they appear)? It has a more practical issue, too: how do I code and analyze multiple ones in SPSS? SPSS for this sort of thing isn't exactly my forté, so I'm not sure how to go about that or if it's possible.

Looking at your examples given above I'd query coding a reply that just mentioned raiding as "fun", because to me that seems that you are interpreting what raiding is. It may be fun to some, but to others it is a way to grind to get gear – not necessarily fun but a necessary evil!

Using the categories provided by the respondents, and using multiple catgories would give a richer picture of the guild dynamics and purpose. Setting this in context with your research into what motivates people, you could then look at whether those multifaceted guilds had more motivated players. Also what types of guild foster motivation maybe?

As to order, when you list something there is an inherent priority – i.e. you usually list the most prominant things first. Would you want to weight categories according to order, unless the respondent suggests that it is an equal balance? And if you weighted it would certain categories come out as dominant, and how then does that link to motivation and learning?

Are you doing follow up interviews? If so one of the things might be interesting is that "philanthropy" or "helping" category. When people mention this does this mean contributing some gold to buy armour, crafting items for guild use, or is it the mentor/mentee relationship? Certainly a lot of initial help I recieved in game play, and still get is from more experienced "hard core" players who can advise on tactics, advise on best places to farm materials or even take you around instances/areas for first time in a hand holding manner whilst you become familiar with it.

Thanks for commenting, Juanita. A large part of my problem here is the tool I'm trying to use to do the analysis. Having previously used SPSS, I was attempting to do the coding in that, but it's fairly limited. I'm switching to using NVivo (http://www.qsrinternational.com/ formerly NUD*IST), which will make it a lot easier to add as many tags to qualitative data like this as I want.

I agree that philanthropy and helping is really interesting, because I think in here we'll find many examples of informal learning occurring, through people mentoring, giving ad-hoc advice, acting as templates, etc. I'm so excited about this!

This was a small-scale survey to work out the kinks, explore the process of doing the analysis, and to generate some motivations for why people play World of Warcraft that could be used for a second, larger scale survey. I therefore do not envision doing follow-up interviews from this survey, but it is not completely out of the question either. I started off by playing with this much smaller question to get my feet wet before tackling the much longer and more articulate responses people gave in their "essays" as to why they played World of Warcraft.

[...] way by coding each 140-character response into one of a number of categories, I found that approach unsatisfying. Even in such short responses, there was more nuance than I could easily accommodate in a simple, [...]

1 Trackbacks / Pingbacks

[...] way by coding each 140-character response into one of a number of categories, I found that approach unsatisfying. Even in such short responses, there was more nuance than I could easily accommodate in a simple, [...]