Q-Assessor Newshttp://q-assessor.com/entries
Q-Assessor NewsDoing Q "the R Way"http://q-assessor.com/entries/3016
<p>One unfortunate result of online Q systems like Q-Assessor &#8212; and one that is frequently criticized by traditional Q practitioners on the <a href="http://listserv.kent.edu/archives/q-method.html">Q Listserv</a> &#8212; is that investigators can rapidly accumulate huge data sets. Instead of collecting perhaps a few dozen qsorts via in-person, on-paper administration, online data collection can produce hundreds and thousands of qsorts.</p>
<p>This is usually dismissed as &#8220;doing R and not Q.&#8221; The conceptual problem is that Q seeks to find relevant <em>types</em> of perspective in a population, not the <em>prevalence</em> of such types. All that is required for this, according to Q experts, is a relatively small number of responses, whereas a large number introduces noise and reduces the meaning of any analysis.</p>
<p>The practical problem with this is that the factor analysis algorithms that Q uses &#8212; notably those implemented in the canonical Fortran program called <span class="caps">QMETHOD</span> and its subsequent PC wrapper PQMethod &#8212; are not designed for efficiency since they don&#8217;t expect such huge data sets. Q-Assessor runs a direct implementation, within its development environment, of this original code. Trying to crunch through huge data sets with these inefficient implementations on a shared server basically exhausts available resources and causes processes to hang and eventually crash. Not acceptable.</p>
<p>Hence Q-Assessor now tests a given study&#8217;s data set &#8212; looking at the number of statements being sorted and the number of collected sorts of those statements &#8212; and removes the online analysis option when the data are too massive to handle comfortably. Instead, Q-Assessor will advise the investigator to download the raw data or <a href="http://q-assessor.com/entries/3013">export it in the PQMethod format</a> and then perform any analysis offline using desktop tools like PQMethod. It is likely that sufficiently large data sets will also choke PQMethod, but they will be more easily handled by such an application running alone on a desktop computer.</p>
<p>Note that this issue pertains only to the <strong><em>analysis</em></strong> aspect of Q-Assessor&#8217;s functions. Investigator can still happily use Q-Assessor to collect as many sorts of as many statements as they want; they just can&#8217;t do these massive analyses online. But please note: the analysis function has never been Q-Assessor&#8217;s principal purpose. Rather, Q-Assessor&#8217;s key strength is carrying out the full range of functions in the Q study lifecycle &#8212; from study creation and configuration to enrollment management and administration to data collection. Q-Assessor&#8217;s analysis capabilities are principally a convenience operation for regular Q studies.</p>
<p>If an investigator is trying to do &#8220;weird Q&#8221; with huge numbers of participants, serial administrations, and some of the other decidedly non-canonical protocols that recent Q-Assessor subscribers have been doing, then the investigator likely wants to fiddle with alternate analysis techniques anyway &#8212; such as the <a href="http://q-assessor.com/entries/3013"><span class="caps">PCA</span> factor extraction method and other tools recently added to PQMethod</a> &#8212; using desktop statistical tools like PQMethod or the <a href="https://cran.r-project.org/web/packages/qmethod/index.html">qmethod package for R</a>. To do these unusual or cutting-edge analyses, such investigators need to download/export the data from Q-Assessor in any case.</p>
<p>The more general conceptual issue about whether and how to stop investigators from trying to &#8220;do Q the R way&#8221; is well beyond the agenda of Q-Assessor. The best we can do is try to ensure that one investigator&#8217;s study does not prevent all the other Q-Assessor investigators from being able to use the system.</p>http://q-assessor.com/entries/3016New Payment Optionshttp://q-assessor.com/entries/3015
<p>In response to subscriber requests, we&#8217;ve increased the payment options for subscriptions to Q-Assessor. These options are described in greater detail on <a href="http://q-assessor.com/pages/3010">the subscriptions page</a>. The two new options are these:</p>
<ul>
<li><strong><a href="http://www.paymode-x.com">Paymode-X</a></strong><strong>:</strong> This is a payment processor used by many larger institutions like universities and corporations. It imposes no fees on us either, so we welcome this form of payment. If your institution is picking up your subscription cost, they likely will insist on this mechanism. The information we provide on the invoice is sufficient for your finance people to make the payment to us via Paymode-X.</li>
<li><strong><a href="https://www.paypal.com/home">PayPal</a></strong><strong>:</strong> This payment processor has a global reach and allows subscribers to pay via their PayPal accounts, if they have one, or via credit cards, if they don&#8217;t. Due to the low volume of subscriptions that want to pay this way, we have not implemented any automated PayPal functions like a &#8220;Pay With PayPal&#8221; button. Instead, if you want to pay this way, we will send you an invoice via PayPal that has links by which you can then log into PayPal and send the payment. This works well, though PayPal extracts a usurious percentage rate for &#8220;service fees&#8221; particularly if there is a currency exchange. If you are in a hurry and are unwilling or unable to pay via one of the above alternatives, then we will work with you to use PayPal.</li>
</ul>
<p>You select your preferred payment option when requesting a subscription. If there are questions or issues regarding handling payments, we will of course discuss with you via email.</p>
<p>Note: because of problems we have had with bank wire transfers, this payment option is no longer offered.</p>http://q-assessor.com/entries/3015Making Sense of Your Resultshttp://q-assessor.com/entries/3014
<p>So you&#8217;ve successfully completed your study &#8212; all your participants have submitted their sorts and completed the post-sort interviews. You&#8217;ve munged your raw data using Q-Assessor&#8217;s online tools or you&#8217;ve downloaded your data and churned it with PQMethod or R. You&#8217;re now staring at some number of factors with reported eigenvalues and variances and whatnot. What does this all mean and how do you use it?</p>
<p>Well of course this is where the majesty or genius or mystery or bollocks (depending on your disposition) of Q stands forth. You can (and should) resort to the many obsessively detailed books and <a href="http://listserv.kent.edu/archives/q-method.html">discussions</a> out there where the vagaries of Q analysis and interpretation are scrutinized in densely-packed jargon. Since you&#8217;re doing a Q study, you will presumably want to report your results in similar fashion.</p>
<p>However en route, you will probably benefit if you spend a bit of time stepping back and thinking about your study from the simple first principles of Q. Recall that Q identifies <em><strong>types of people and their perspectives</strong></em> from the cross-correlations amongst their sorts. Each factor in your results thus represents a type of person and what they think. The task is to understand the nature of that person type by looking at two things: 1) the meaning of the statements that most define that type; and 2) the self-expressed features of people of that type as revealed in their answers to the post-sort interview. Once you understand the type, you can then provide it with an explanatory label that provides a useful shorthand as you compare that type to the others you discover and then interpret them all within the semantic arena of your study.</p>
<p>So for instance, if you were looking at &#8220;attitudes of college students to gender,&#8221; you might find that one factor represents &#8220;obnoxious fraternity &#8216;bros&#8217; with mommy issues,&#8221; another factor &#8220;brilliant arts students who couldn&#8217;t get dates in high school,&#8221; and so on. Then you would use these discovered types to tell you something about the college and gender in the context of your research question. Note that one interesting result would be if you found individual respondents whose demographics would suggest they should belong to one type but their sorts place them in quite another type &#8212; e.g. a female violin major whose attitudes match best with the &#8220;bros&#8221; (analogous to the white supremacists whose <span class="caps">DNA</span> analysis shows large degrees of African <span class="caps">DNA</span> for instance). That&#8217;s the kind of revelation that Q supposedly can uniquely provide. What that might mean would be up to you to explain, but Q can at least discover such things for you.</p>
<p>Anyway, once you &#8220;explain and label&#8221; each of your factors so you&#8217;re no longer thinking about eigenvalues and loadings but instead are thinking of types of people and their perspectives, you are then in a better position to determine which provide meaningful insights into whatever fundamental question you had that led you to conduct the study. That would also provide a cogent rationale for why you choose the factors (viz, which types of people/perspective) that you report as significant answers to your question.</p>
<p>Once you personally understand something new and important about the world your study examined, you can either report it out clearly and understandably, or else you can follow conventional Q practice and wrap it in layers of verbose, opaque Q jargon in the fashion that Q traditionalists expect. That choice depends on your audience. But at the very least, <em>you</em> should understand what your study means.</p>http://q-assessor.com/entries/3014Q-Assessor Now Provides Export to PQMethodhttp://q-assessor.com/entries/3013
<p>From its beginning, Q-Assessor has utilized the centroid factor extraction method used in PQMethod&#8217;s <span class="caps">FORTRAN</span> public domain source code. Recent Q-Assessor users have reported the need to use some of PQMethod&#8217;s other tools &#8212; notably the <span class="caps">PCA</span> method. Although Q-Assessor was intended to provide an end-to-end solution where this wouldn&#8217;t be necessary, we have realized the value in providing this option.</p>
<p>Now you will find in the &#8220;General Configuration&#8221; section of your study two links to export your statements and sort data in PQMethod-compatible files. (Note: these links only appear when you actually have something to export.) These links also are on the Statements and Responses pages, respectively.</p>
<p>Clicking these links will bring up a dialog in your browser allowing you to save these files to your local computer. The files will be named thus: SIDxxx.sta and SIDxxx.dat, where xxx is your study&#8217;s ID number used by Q-Assessor, and .sta is the PQMethod suffix for the statements file and .dat is the PQMethod suffix for the sorts file. (Note: if you&#8217;re using Safari on a Mac, the browser will forceably add a .txt suffix &#8212; producing SIDxxx.sta.txt and SIDxxx.dat.txt &#8212; in a Microsoftian display of unwanted &#8220;helpfulness.&#8221; You will have to manually remove the .txt suffix from the file names before PQMethod will be able to use these files.)</p>
<p>Once these files are on your hard drive, drag them to your projects folder for PQMethod. Launch PQMethod and provide the path to the files along with the name SIDxxx &#8212; without any suffixes. This will tell PQMethod that it is working on your new project. You should then ignore the first two PQMethod menu options &#8212; because you don&#8217;t want to overwrite the files that already contain your statements and sort data. Instead, you will proceed directly to option 3 (centroid) or option 4 (PCA) factor extraction. You then perform subsequent steps just as per PQMethod procedures.</p>
<p>We think that for most Q-Assessor users, the basic centroid method and other analytic steps/reports will be quite satisfactory. For those who want to explore/manipulate their data further, though, this new Q-Assessor feature will make life much simpler and allow you to use Q-Assessor for all the study management features that it uniquely delivers while offloading alternative analytic steps to PQMethod.</p>
<p>One other related feature that we&#8217;ve added to Q-Assessor is display of the correlation matrix &#8212; on the analysis page as well as in the downloadable report. This is the same matrix that the PQMethod report includes, but those Q-Assessor users who want to continue to use Q-Assessor&#8217;s analytic features but also need the correlation matrix now can have both.</p>
<p>We hope that these developments make Q-Assessor an even more attractive solution to anyone wanting to do Q efficiently and effectively.</p>http://q-assessor.com/entries/3013Q-Assessor Now Supports Images In Statementshttp://q-assessor.com/entries/3012
<p>A recurring if infrequent request has been for Q-Assessor to allow investigators to place images within statements so that participants can sort graphical elements, not just text. Because of the variable and always too-limited screen real estate, this is a challenge. To be visible, images need to be large, but to fit numerous ones in a single view, they need to be small.</p>
<p>Q-Assessor now has what we think is a useful solution for this. Thumbnail versions of each image are placed in the statements when they&#8217;re being dragged around the screen, but large versions of each image show in a popup window when the participant hovers her cursor over the thumbnail &#8212; or taps the thumbnail on a tablet computer or smartphone. This allows the participant to quickly and intuitively switch between minimum and maximum views of each image.</p>
<p>During construction of the statements, an investigator uploads into Q-Assessor images for the statements. Q-Assessor automatically creates the necessary sized versions of the image and handles the popup magic when the study is deployed.</p>
<p>You can see all of this in action in this <a href="/studies/830/responses/new">demo study</a>.</p>http://q-assessor.com/entries/3012Q-Assessor Now Supports Specifying Number of Extracted Factorshttp://q-assessor.com/entries/3011
<p>One feature some Q-Assessor users have requested has been the ability to specify how many centroid factors are extracted from the initial correlation matrix. We resisted adding this step, as there is no support for it in <a href="http://qmethod.org/papers/Brown-1980-PoliticalSubjectivity.pdf">Brown&#8217;s book</a> (cf the Factor Analysis section of chapter 4) and from a mathematical viewpoint, we can&#8217;t understand why this &#8220;feature&#8221; was added to PQMethod. Nevertheless, because this is a request that doesn&#8217;t seem to go away, we have added this capability.</p>
<p>The default number of extracted factors when you first analyze your data is the nominal seven, which is the &#8220;magic number&#8221; (per Brown) that is the maximum number of plausibly significant factors in a data set. A single popup menu option then lets you recalculate your results restricting the analysis to 1-7 factors. Thus you can restrict the search to a smaller number of factors and then rotate them as you like. Some users find significant value in this flexibility.</p>
<p>We remain skeptical of the value of this step, however, for this reason. The basic idea behind Q is that the cross-correlations amongst the participants&#8217; sorts encode patterns of opinion/belief/understanding that the factor analysis process reveals. The number of statistically-significant factors thus relies within the data &#8212; not the sieve used to examine the data. If there are five significant factors, looking for seven will identify the five and show that there are two additional factors that are insignificant and can be ignored. If however the algorithms are restricted arbitrarily to look only for, say, three factors, then important information is lost. This is not part of how Brown explains this should be done, so we can only speculate why the <span class="caps">FORTRAN</span> code underlying PQMethod included it.</p>
<p>Regardless, Q-Assessor now supports both approaches to analysis.</p>http://q-assessor.com/entries/3011Free "Test Drive" Use of Q-Assessor Now Available In Addition to Subscriptions http://q-assessor.com/entries/3010
<p>Because of the gratifying interest in Q-Assessor over the past year, we have reworked our policies regarding access to the system:</p>
<ul>
<li>We have resumed open registration to the site.</li>
<li>New registered users can configure and conduct one test study using all of Q-Assessor&#8217;s features, limited only by the number of concourse statements, interview questions, and allowed responses.</li>
<li>We have improved the subscription request system, so that a simple form &#8212; inquiring the number of months desired, the projected start date, and a billing address &#8212; generates an invoice and signals to us of your interest.</li>
<li>A paid subscriber can create and conduct any number of studies during the subscription period, plus gain access to older, completed studies.</li>
<li>All users who tried out Q-Assessor during the prior beta-test phase can access (when logged in) their studies again, albeit in &#8220;read-only&#8221; mode. Thus they can retrieve their data and analyses if they like, just not restart those studies &#8212; unless they subscribe.</li>
<li>Subscriptions are denominated in one-month intervals. The cost is currently $500 <span class="caps">USD</span> per investigator per month.</li>
<li>Q-Assessor can accommodate teams collaborating on a study once one team member is the designated subscriber for the group. All group members can develop and deploy group studies, though they cannot run their own personal studies without a personal subscription.</li>
<li>Payment options at this point are bank checks drawn on US funds and bank wire transfers. We do not as yet accept other payment processors like PayPal and credit cards, but at whatever time subscription volume is significant to offset the substantial fees of those processors, we will reconsider.</li>
</ul>
<p>We think that these policies will simplify and improve use of Q-Assessor. We&#8217;re always glad for <a href="mailto:webmaster@q-assessor.com">your feedback</a>.</p>
<p>You can read more about <a href="/pages/3010">the subscription system and policies here</a>,</p>http://q-assessor.com/entries/3010Q-Asssessor Now Fully Internationalizedhttp://q-assessor.com/entries/3009
<p>Developed primarily for use in the U.S., Q-Assessor previously hard-wired the English language in operational texts like sort instructions, button labels, dialog window texts, email links, and the like. Investigators could author their statements, categorical poles, and interview questions in their language of choice, but all these other elements persisted in English. Clearly this was an unsatisfactory situation.</p>
<p>Q-Assessor now supports <a href="/pages/3015">full internationalization</a> of a study in any of &#8212; at present &#8212; 22 other languages besides English. With a single configuration, investigators can match all of Q-Assessor&#8217;s own texts to the language they use in the content of the study.</p>
<p>We will continue to add more languages as demand for them arises. We also welcome any suggested edits to our translated texts from native speakers who find issues with our initial translations &#8212; which were generated by Google&#8217;s translation service.</p>http://q-assessor.com/entries/3009Limited Availability to Serious Investigatorshttp://q-assessor.com/entries/3008
<p>Because of recurring inquiries, we&#8217;ve decided to make Q-Assessor accessible to serious investigators doing projects with real budgets. The cost per month per user is $500 <span class="caps">USD</span>. If you are interested, <a href="mailto:info@q-assessor.com">please email us with details about your project and desired use</a>. Others should investigate the various free tools out there.</p>http://q-assessor.com/entries/3008Extended Beta Test Phase Drawing to Closehttp://q-assessor.com/entries/3007
<p>After a couple years experimenting with Q-Assessor, listening to feedback from users, and evaluating the opportunities for Q in various applications, we&#8217;re closing down the beta test phase. Thus new sign-ups are no longer being accepted.</p>
<p>It appears to us that Q-Assessor&#8217;s technologies have most potential in bespoke vertical applications and not in a general-purpose &#8220;online Q system&#8221; as Q-Assessor is currently configured. There just isn&#8217;t a sustainable market amongst the small, primarily academic Q practitioners out there. Q simply appears to be a too-little used methodology to support a web system like Q-Assessor. We have identified several specific applications where the Q technique might be quite valuable without invoking all the baggage that a specific identification with &#8220;Q&#8221; involves. We hope that Q-Assessor&#8217;s <span class="caps">DNA</span> will live on in other forms.</p>
<p>We will provide current users until 1 June 2013 to wind their current studies to conclusion and download their data. We are not supporting any new studies.</p>
<p>We appreciate the interest and feedback we&#8217;ve received about Q-Assessor.</p>http://q-assessor.com/entries/3007