Usability Testing of a Facebook Brand Page

Abstract. Social networks, as a part of Web 2.0 technology, provide the technological platform for users to connect, produce and share content online. Availability of user generated content is encouraging companies to fully engage with their customers in order to enhance and enrich social media users’ experiences. However, the usage of the social media platforms differs from the more traditional forms of a company’s web presence, resulting in challenges of applying the traditional usability testing methods. In this paper we analyze this problem for the context of a Facebook brand page and propose applying an iterative usability testing approach over automatically collected data. From the discussion, we derive implications for usability and social media marketing. Keywords: social networks, usability testing

Document transcript

Abstract. Social networks, as a part of Web 2.0 technology, provide thetechnological platform for users to connect, produce and share content online.Availability of user generated content is encouraging companies to fully engagewith their customers in order to enhance and enrich social media users’experiences. However, the usage of the social media platforms differs from themore traditional forms of a company’s web presence, resulting in challenges ofapplying the traditional usability testing methods. In this paper we analyze thisproblem for the context of a Facebook brand page and propose applying aniterative usability testing approach over automatically collected data. From thediscussion, we derive implications for usability and social media marketing.Keywords: social networks, usability testing1 IntroductionSocial networks (SN) have a mediating effect between individuals and society in thevirtual world. As such, they represent a natural technological platform for marketing,providing access to a large number of users. Companies, across all industries arestarting to understand the possibilities of social media (SM) marketing. They haveevolved their approach to their customers, offering contact or assistance on a personallevel at any time through social network sites such as Facebook, Twitter, etc.(Gordhamer, 2009). However, how these platforms are being used, what theirpotentials are and how consumers interact, remains largely unknown and has yet to beaddressed from different perspectives (Richter et al., 2011).Availability of the user generated content of SM platforms is encouragingcompanies to fully engage with their customers in order to enhance and enrich SMusers’ experiences. However, the usage of the SM platforms differs from the moretraditional forms of companies’ web presence resulting in challenges of applying thetraditional usability guidelines and methods to this new communication channel.Classic usability testing methods, involving few participants using a prototype, arelimited when applied to SM. To contribute in this direction we try to answer thequestion: How to perform usability testing on a Facebook brand page?2 Related WorkUsability can be defined as “the extent to which a product can be used by specifiedusers to achieve specified goals in a specified context of use.”1The main usabilityprinciples, as described by Gould and Lewis (1985) are: (1) focus on users and tasks,(2) empirical measurement, and (3) iterative design. Furthermore, Dumas and Redish(1993) describe five characteristics of usability testing: (1) improving the usability ofa product/service, (2) the participants represent real users, (3) the participants do realtasks, (4) observing and recording of participants activities, and (5) analysis of thedata. Following the described steps should lead to early problem identification andgeneration of recommendations on how to solve these problems and ultimatelyimprove the product/system.The problem of usability testing has been studied in depth over different categoriesof websites. However, content on the social media platforms used for marketingpurposes differs from the traditional web content in terms of representing a dialogbetween two involved parties, companies and individuals (Boyd and Ellison, 2008).Furthermore, social media platforms such as Facebook and Twitter leave little spacefor interface customization. For that reason, practitioners are beginning to addresssocial media usability from a perspective of enhancing the content and activities.Based on the traditional usability study, Estes et al. (2009) have identified 85 usabilityguidelines grouped as (1) content, (2) frequency and timing, (3) voice and tone, (4)engaging followers and facilitating discussion, (5) profile information and design, and(6) promoting social network presence. Similarly, Fidgeon (2011) advises: (1)focusing on the users, (2) selecting the appropriate timing, and (3) creating contentsuitable for the medium. At the same time, he points out that due to the great diversityof different communities there are no guidelines that could guarantee the success inthis new form of communication channel.To overcome the described challenge, we propose an iterative usability testingapproach based on the automatic data collection that could potentially lead to greateruser satisfaction.3 Problem DefinitionUsability testing usually involves usage scenarios in order to determine how peopleinteract with the product. However, when applied to a Facebook brand page, theinteraction assumes dialog between the members of the community and the companywhich could not be described in a form of a task. Furthermore, due to diversity of theonline communities, social media guidelines embraced at the planning phase mightnot lead to the expected results. To overcome these challenges, we propose iterativeusability testing that would enable strategy refinement in compliance with the actualcharacteristics and actions of the Facebook brand page fans. In the continuation, wedescribe the proposed method by answering the questions: (1) which data sources touse, (2) how to prepare the data for analysis, and (3) how often to analyze the data.

There are two available sources of data describing the activities on aFacebook brand page: (1) Facebook Insights2, and (2) Facebook Graph API3.Facebook Insights is a platform provided to Facebook page administrators to enablemonitoring of the activities on the page. It provides information on users, such asdemographics (gender, age, language, etc.) and number of fans, as well as details onthe interaction with the page. Data collected through the Facebook Insights offerspossibilities for analysis of the target audience characteristics. While thedemographics data is useful, since it can’t be obtained from another source, theinteraction data provided by the Facebook Insights platform might not be accurate; forexample, number of post views is defined as “the number of times people haveviewed a News Feed story”. This however can only be counted as the number oftimes the post appeared on someone’s “wall” and there is no guarantee that the postwas seen or read. For that reason, for the interaction evaluation we recommendimplementing a tool that would gather the content shared on the Facebook brand pagebased on the Facebook Graph API.The Facebook Graph API provides access to the Facebook social graph via auniform representation of the objects in the graph (e.g., people, pages, etc.) and theconnections between them. Upon a query, the data can be returned as a Page objectcontaining connections such as Feed, Posts, Photos, etc. A Feed connection representsa list of all Post objects shared on the “wall” containing the following relevantinformation: (1) post content, (2) post type, (3) posting user, (4) likes, (5) comments,(6) application used for posting, (7) creation time and (8) time of last interaction. Datagathered through the Graph API offers possibilities for quantitative and qualitativeanalysis with relevance to the usability guidelines referring to the content shared onthe social media platform (Pletikosa and Michahelles, 2011, 2011a). Furthermore, itprovides insights into the users’ reaction in terms of posting time (Momentus Media,2011).Normalization of the Data.

Both of the data sources provide numbers as an absolutemeasure of the interaction represented through the number of comments, likes, clicks,views, etc. However, these numbers are related to the total number of users at theobserved moment. For that reason, when analyzing the interaction data werecommend normalization of the data with the number of fans at the moment, asdescribed by Pletikosa and Michahelles (2011a).Iteration Frequency.

In terms of the iteration frequency there are no many existingguidelines on what should be the optimal frequency of testing and refinements of theusability. Dubach Spiegler (2011) proposes two levels of iteration (1) tactical and (2)strategic control. High-level metrics related to moderator and user activities as a partof the tactical control should be assessed daily in the initial phase of the Facebookbrand page, and not less than weekly once the page is in its operationally maturephase. Deeper analysis should also be performed on a regular interval to supportstrategic control. Since Facebook Insights offers monthly aggregated data, for easier

2Facebook Insights, http://www.facebook.com/help/?page=10303Facebook Graph API, http://developers.facebook.com/docs/reference/api/integration of the proposed data sources we suggest the iteration frequency of onemonth for data gathered for strategic control.5 Discussion and ConclusionsIn this paper we have presented a method for usability testing of a Facebook brandpage. We have addressed two relevant questions: (1) how to overcome the problem ofdata collection in the social media context where the traditional controlled usabilitytesting might not lead to the expected results, and (2) how often should the usabilitytesting be performed in order to enable timely reaction to the possibly changing rulesof communication in this noisy medium. The described approach contributes in twodirections: usability testing and social media marketing. From the usabilityperspective of usability testing it provides a methodology suitable for Facebook as asocial media platform which overcomes the discussed challenges. From the socialmedia marketing perspective, this paper provides insights into how usability conceptsand guidelines could be integrated with a company’s social media marketing strategyfor a Facebook brand page. Issues identified by Estes et al. (2009) overlap with thoseelaborated in the work of Dubach Spiegler (2011). This indicates that social mediamarketing practitioners could learn from existing knowledge in the field of usabilityand user-centered design.References1. Gordhamer, S.: 4 ways social media is changing business. Mashable.com,http://mashable.com/2009/09/22/social-media-business (2009)2. Richter, D. Riemer, K., vom Brocke, J.: Internet social networking: Research state of the artand implications for Enterprise 2.0. Business & Information Systems Engineering (2011)3. Gould, J.D., Lewis, C.: Designing for usability: key principles and what designers think.Commun. ACM 28 (3), 300-311 (1985)4. Dumas, J.S., Redish, J.C.: A practical guide to usability testing. Norwood, NJ: AblexPublishing Corporation (1993)6. Fidgeon, T.: Social media usability: content & activities. Spotless Interactive,http://www.spotlessinteractive.com/articles/usability-research/usability-testing/social-media-content-and-activity.php (2011)7. Estes, J., Schade, A., Nielsen, J.: Streams, Walls, and Feeds: 109 Design Guidelines forImproving Notifications, Messages, and Alerts Sent Through Social Networks and RSS.Nielsen Norman Group Report, http://www.nngroup.com/reports/streams (2009)8. Pletikosa Cvijikj, I., Michahelles, F.: Understanding Social Media Marketing: A Case Studyon Topics, Categories and Sentiment on a Facebook Brand Page. 15th MindTrek Conferenceand the International Academic Conference (2011)9. Pletikosa Cvijikj, I., Michahelles, F.: A Case Study of the Effects of Moderator Posts withina Facebook Brand Page. 3rd International Conference on Social Informatics (2011a)10. Momentus Media: Engagement & Interaction - A scientific approach to Facebookmarketing. Momentus Media Whitepaper, http://momentusmedia.com/blog/?p=899 (2011)11. Dubach Spiegler E.: Applications and Implications of User-Generated Content in Retail,PhD Thesis, ETH Zurich (2011)