The 2009 ABA Legal Technology Survey Report contains a wealth of valuable and current information about U.S. lawyers’ use of technology. Among the topics the report addresses is lawyers’ use of mobile devices for legal research. Two questions on the 2009 survey addressed this topic, but appear to yield inconsistent results.

In Volume V of the report, entitled “Online Research,” at page V-51, question 31b asked: “How often do you use each of the following computer hardware to conduct legal research while away from the office: Smartphone/BlackBerry?” 19.1% of respondents answered: “Regularly.”

By contrast, in Volume VI of the report, entitled “Mobile Lawyers,” at page VI-53, question 36n asked: “How often do you use each of the following features on your PDA/smartphone/BlackBerry while away from the office: Online legal research (e.g., Wireless Westlaw)?” 2.3% of respondents answered: “Regularly.”

At first glance, these questions seem to be addressing the same behavior: the extent to which lawyers use their mobile devices to perform legal research. Yet the use of multiple, differently worded questions indicates that ABA actually intended each of the questions to measure a different behavior. I asked Stephen Stine, Research Specialist at ABA’s Legal Technology Resource Center, which administers the survey, to identify the behaviors that ABA wished to measure by means of these two questions, and to interpret the responses to the questions. Here is his response:

“Having examined the questions, my conclusions are that any difference in the results for the two questions is due to the fact that the two questions ask respondents about two different types of research behavior, and that the respondents understood the intent of the questions and answered them accordingly based on the differences in the questions’ wording and their context within the questionnaires.

“Question 31b asks about any type of legal research performed using a Smartphone/BlackBerry, which can include online legal research requiring Internet access as well as offline legal research using materials such as briefs and cases stored on such a device (whether previously downloaded to the device from the Internet or transferred to the device in some other manner, e.g. USB cable, Bluetooth, removable media). The section of the questionnaire that question 31b appeared in was entitled ‘Electronic Legal Research Away from the Office’ (not limited to online legal research). This section included other questions regarding use of online as well as offline legal research materials—q.30, for example, asked about the use of ‘Removable Media (e.g., CD-ROM disk)’ for legal research while away from the office. It would thus be clear to respondents that the question section inquired about the use of both online and offline legal research materials.

“Question 36n on the other hand asks exclusively about ‘online legal research,’ and was designed to inquire about respondents’ use of Internet connection-dependent mobile device-specific websites and applications such as ‘Wireless Westlaw.’ We chose the parenthetical ‘(e.g., Wireless Westlaw)’ to convey an example of a mobile device-specific online legal research website or application, as opposed to the general purpose LexisNexis.com and Westlaw.com online legal research websites—presumably fewer attorneys will have used or even heard of such mobile device-specific online legal research websites and applications than have used or heard of their general purpose counterparts, and this is reflected in the results to this question. We chose the language ‘feature’ to denote that we were asking about mobile device-specific websites and applications—other ‘features’ we inquired about were telephone, GPS, camera, and text messaging, which are more commonly features of mobile devices such as PDAs/Smartphones/BlackBerrys, and less commonly of desktop and laptop computers. This context, asking about ‘features’ on PDAs/Smartphones/BlackBerrys and the ‘Wireless Westlaw’ parenthetical were designed to make it apparent to respondents that we were inquiring about mobile device-specific, as opposed to general purpose, online legal research websites and applications.

“The results of the questions bear out the conclusion that the respondents understood the differences in the intent and thrust of the questions based on their wording and positioning in the questionnaires: 19% reported that they ‘Regularly’ use a Smartphone/BlackBerry to conduct legal research (online and offline) while away from the office, whereas only 2% reported that they ‘Regularly’ use online legal research (not including offline legal research) features such as Wireless Westlaw on their PDAs/Smartphones/BlackBerrys.

“Next year we may further clarify the intent of question 36n by changing the language to read something similar to, ‘How often do you use each of the following features on your PDA/Smartphone/BlackBerry while away from the office: Mobile device-specific online legal research websites and applications (e.g., Wireless Westlaw, as opposed to the general purpose LexisNexis.com or Westlaw.com)?”

I’m grateful to Mr. Stine and the LTRC for this detailed response. I invite readers to share their thoughts respecting these survey questions, in the comments to this post.