Best Practices for Load Testing Mobile Applications | Part I

Mobile applications and mobile websites have become a major channel for conducting business, improving employee efficiency, communicating, and reaching consumers. In the past, mobile played a smaller role in business applications, so performance issues and outages were less of a concern. This is no longer the case. Today, performance problems with mobile applications lead directly to revenue loss, brand damage, and diminished employee productivity. Part I of this article discusses the differences between testing traditional web and mobile applications, specific challenges associated with mobile load testing, mobile testing basics and best practices for recording mobile load test scenarios. In Part II, we will look at how to conduct realistic tests and how to best analyze the results.

Application developers have long understood the need for load testing conventional desktop web applications to ensure that they will behave properly under load with the expected number of users. With the advent of mobile apps and mobile websites the principles of load testing have not changed. There are, however, challenges specific to mobile load testing that must be addressed by your load testing solution.

Since mobile apps and applications for desktop web browsers use the same underlying technologies, the good news is that most load testing tasks and challenges are the same. This means that you don't necessarily need a brand new, mobile-specific load testing tool, but you do need a quality web load testing tool capable of handling the nuances of load testing mobile apps. Using a tool that enables testing of traditional and mobile web applications enables you to leverage existing in-house skills for designing and parameterizing your scripts, running your tests, and analyzing the results.

Aside from the similarities between traditional and mobile load testing, there are three key differences:

Simulating network and bandwidth for wireless protocols: With 3G wireless protocols, mobile devices typically connect to the Internet using a slower, lower quality connection than desktops and laptops. This has an effect on response times on the client side and on the server, which you'll need to account for as you define your tests and analyze your results. Additionally, latency and packet loss becomes more of a factor with mobile applications and needs to be considered.

Recording on mobile devices: Obviously, mobile apps run on mobile devices, and this can make it difficult to record test scenarios, particularly for secured applications that use HTTPS.

Supporting a wide range of devices: The many different kinds of mobile devices on the market have led web application designers to tailor content based on the capabilities of the client's platform. This presents challenges for recording and playing back test scenarios.

Mobile Load Testing BasicsAs you may know, a typical automated functional test for a mobile application emulates user actions (including tap, swipe, zoom, and entering text) on a real device or an emulator. The objective of load testing, however, is not to test the functionality of the application for just a single user. Rather, the goal is to see how the server infrastructure performs when handling requests from a large number of users, and to understand how response times are affected by other users interacting with the application.

An effective load test simulates a high volume of simultaneous users accessing your server via your application. Using real devices or emulators for this task is impractical because it demands acquiring, configuring, and synchronizing hundreds or thousands of real devices or machines running emulators.

The solution, of course, is to use a load testing approach that is designed to scale as needed. With a client-based approach, user actions in the browser or the native application are recorded and played back. In contrast, a protocol-based approach involves recording and reproducing the network traffic between the device and the server. To verify performance under large loads, tools that enable protocol-based testing are superior to those that support only client-based testing because they can scale up to millions of users while checking errors and response times for each user.

The basic process for protocol-based mobile load testing is:

Record the network traffic between the device and the server

Replay the network requests for a large number of virtual users

Analyze the results

It appears straightforward, but there are challenges at every step. The good news is that these challenges can be addressed with an effective load testing approach.

Recording Mobile Load Testing ScenariosTo generate a mobile test scenario, you first need to identify the type of mobile application under test. Challenges associated with capturing the data exchanges between a mobile application and the server depend on the design of the application:

Native apps - These apps are coded using a programming language (Objective-C for iOS, Java for Android) and API that is specific to the device. As such, they are tied to a mobile platform and are installed from an online store or market.

Web apps - Built with web technologies (such as HTML and JavaScript), these applications can be accessed from any mobile browser. More sophisticated web apps may use advanced features like geolocation or web storage for data or include customizations to better match the browser used. Two popular web apps are http://touch.linkedin.com and http://m.gmail.com.

Hybrid Apps - A web app embedded in a native app is known as a hybrid app. The native part of the application is limited to a few user interface elements like the menu or navigation buttons, and functions such as automatic login. The main content is displayed in an embedded web browser component. The Facebook application, installed from an online store or a market is a typical sample.

CIO, CTO & Developer Resources

Recording Tests for Native AppsBecause native apps run on your device or within an emulator, to record a test you need to intercept the network traffic coming from the real device or the emulator.

To intercept this traffic, the equipment that records the traffic must be connected to the same network as the device. When the recording computer is on the intranet behind a firewall, it is not possible to record a mobile device connected via a 3G or 4G wireless network. The device and the computer running the recorder must be connected to the same Wi-Fi network.

Most load testing tools provide a proxy based recorder, which is the easiest way to record an application's network traffic. To use this approach, you need to configure the mobile device's Wi-Fi settings so that the traffic goes through the recording proxy. Some mobile operating systems, such as iOS and Android 4, support making this change, but older versions of Android may not. Moreover, some applications connect directly to the server regardless of the proxy settings of the operating system. In either of these cases, you need a tool that provides an alternative to proxy-based recording methods based on network capture or tunneling.

Note: You can use the following simple test to check if the application can be recorded using a proxy. First, configure the proxy settings on the device and record your interactions with any website in a mobile browser. Then, try to record interactions in the native application. If your testing tool successfully records the browser generated traffic, but does not record traffic generated by the native application then you can conclude that the native application is bypassing the proxy settings and that an alternative recording method is required.

Recording Tests for Web Apps and Mobile Version of WebsitesWeb apps use the same web technologies as modern desktop browsers. As a result, you can record the application or the mobile version of a website from a modern browser on your regular desktop computer, which is an easier and faster alternative to recording from the device.

Many web applications check the browser and platform used to access them. This enables the application, when accessed from a mobile device, to redirect to a mobile version of the content that may contain less text or fewer images. To test such an app from the desktop, you need to modify the requests to make them appear to the server to be coming from a mobile device. Otherwise, you will not be testing the mobile version of the application as the server may redirect to a desktop version. Some browser plugins provide the ability to alter the identity of the browser (by modifying the User-Agent header of requests). Support for this feature is also directly integrated in the recorder of advanced load testing tools.

Modifying the browser's identity is not always enough. You obviously cannot use this approach to transform Internet Explorer 6 into an HTML5 compatible browser. The browser you use on the desktop must be able to parse and render content created for mobile browsers, so it's best to record with a modern browser like Internet Explorer 9, Firefox 5, Chrome 15, or Safari 5 (or a more recent version of any of these if available). If the application includes WebKit specific features, you should use a WebKit based desktop browser, preferably either Chrome or Safari.

Recording Tests for Hybrid AppsObviously, tests for native apps cannot be recorded using a desktop browser. However, tests for many hybrid apps can. You may be able to directly access the URL used for the application, for example http://m.facebook.com for the Facebook application, and record your tests as you would for a classic web app.

Recording Tests for Secured Native ApplicationsThere are additional challenges to consider when recording tests for a secured native application, that is, an application that uses HTTPS for the login procedure or any other processing.

By default, all HTTPS recording methods, whether proxy or tunnel based, are seen as man-in-the-middle attacks by the device. This raises a non-blocking alert in a desktop or mobile browser but it leads to an outright connection refusal in native applications, making it impossible to record the secured traffic.

The only way to record tests for secured native applications is to provide a root certificate that authorizes the connection with the proxy or tunnel. While this feature is currently supported by relatively few load testing solutions, it is essential for load testing any native application that relies on HTTPS.

Note: The root certificate must be installed on the device. This operation is simple for iOS devices; you can simply send the certificate via email and open the attachment on the device. For other platforms, including Android, the procedure is not as straightforward and may depend on the version of the operating system and the manufacturer of the device.

Running Realistic TestsOnce you've recorded a test scenario, you need to be parameterize it so that it can emulate users with different identities and behaviors as it is played back to produce a realistic load on the server. This step is required for traditional and mobile web applications, and the tools used to complete it are the same. When playing back the test scenarios, however, there are several challenges specific to mobile load testing and we will discuss this more in Part II of this article on "Best Practices for Load Testing Mobile Applications."

Steve Weisfeldt is a Senior Performance Engineer at Neotys, a provider of load testing software for Web applications. Previously, he has worked as the President of Engine 1 Consulting, a services firm specializing in all facets of test automation. Prior to his involvement at Engine 1 Consulting, he was a Senior Systems Engineer at Aternity. Prior to that, Steve spent seven years at automated testing vendor Segue Software (acquired by Borland). While spending most of his time at Segue delivering professional services and training, he was also involved in pre-sales and product marketing efforts.

Being in the load and performance testing space since 1999, Steve has been involved in load and performance testing projects of all sizes, in industries that span the retail, financial services, insurance and manufacturing sectors. His expertise lies in enabling organizations to optimize their ability to develop, test and launch high-quality applications efficiently, on-time and on-budget. Steve graduated from the University of Massachusetts-Lowell with a BS in Electrical Engineering and an MS in Computer Engineering.

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

SYS-CON Events announced today that HPM Networks will exhibit at the 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
For 20 years, HPM Networks has been integrating technology solutions that solve complex business challenges. HPM Networks has designed solutions for both SMB and enterprise customers throughout the San Francisco Bay Area.

For IoT to grow as quickly as analyst firms’ project, a lot is going to fall on developers to quickly bring applications to market. But the lack of a standard development platform threatens to slow growth and make application development more time consuming and costly, much like we’ve seen in the mobile space.
In his session at @ThingsExpo, Mike Weiner, Product Manager of the Omega DevCloud with KORE Telematics Inc., discussed the evolving requirements for developers as IoT matures and conducted a live demonstration of how quickly application development can happen when the need to comply wit...

The Internet of Everything (IoE) brings together people, process, data and things to make networked connections more relevant and valuable than ever before – transforming information into knowledge and knowledge into wisdom. IoE creates new capabilities, richer experiences, and unprecedented opportunities to improve business and government operations, decision making and mission support capabilities.

Explosive growth in connected devices. Enormous amounts of data for collection and analysis. Critical use of data for split-second decision making and actionable information. All three are factors in making the Internet of Things a reality. Yet, any one factor would have an IT organization pondering its infrastructure strategy.
How should your organization enhance its IT framework to enable an Internet of Things implementation? In his session at @ThingsExpo, James Kirkland, Red Hat's Chief Architect for the Internet of Things and Intelligent Systems, described how to revolutionize your archit...

MuleSoft has announced the findings of its 2015 Connectivity Benchmark Report on the adoption and business impact of APIs.
The findings suggest traditional businesses are quickly evolving into "composable enterprises" built out of hundreds of connected software services, applications and devices. Most are embracing the Internet of Things (IoT) and microservices technologies like Docker. A majority are integrating wearables, like smart watches, and more than half plan to generate revenue with APIs within the next year.

Growth hacking is common for startups to make unheard-of progress in building their business. Career Hacks can help Geek Girls and those who support them (yes, that's you too, Dad!) to excel in this typically male-dominated world.
Get ready to learn the facts:
Is there a bias against women in the tech / developer communities?
Why are women 50% of the workforce, but hold only 24% of the STEM or IT positions?
Some beginnings of what to do about it!
In her Opening Keynote at 16th Cloud Expo, Sandy Carter, IBM General Manager Cloud Ecosystem and Developers, and a Social Business Evangelist, d...

In his keynote at 16th Cloud Expo, Rodney Rogers, CEO of Virtustream, discussed the evolution of the company from inception to its recent acquisition by EMC – including personal insights, lessons learned (and some WTF moments) along the way. Learn how Virtustream’s unique approach of combining the economics and elasticity of the consumer cloud model with proper performance, application automation and security into a platform became a breakout success with enterprise customers and a natural fit for the EMC Federation.

The Internet of Things is not only adding billions of sensors and billions of terabytes to the Internet. It is also forcing a fundamental change in the way we envision Information Technology. For the first time, more data is being created by devices at the edge of the Internet rather than from centralized systems. What does this mean for today's IT professional?
In this Power Panel at @ThingsExpo, moderated by Conference Chair Roger Strukhoff, panelists addressed this very serious issue of profound change in the industry.

Discussions about cloud computing are evolving into discussions about enterprise IT in general. As enterprises increasingly migrate toward their own unique clouds, new issues such as the use of containers and microservices emerge to keep things interesting.
In this Power Panel at 16th Cloud Expo, moderated by Conference Chair Roger Strukhoff, panelists addressed the state of cloud computing today, and what enterprise IT professionals need to know about how the latest topics and trends affect their organization.

It is one thing to build single industrial IoT applications, but what will it take to build the Smart Cities and truly society-changing applications of the future? The technology won’t be the problem, it will be the number of parties that need to work together and be aligned in their motivation to succeed.
In his session at @ThingsExpo, Jason Mondanaro, Director, Product Management at Metanga, discussed how you can plan to cooperate, partner, and form lasting all-star teams to change the world and it starts with business models and monetization strategies.

Converging digital disruptions is creating a major sea change - Cisco calls this the Internet of Everything (IoE). IoE is the network connection of People, Process, Data and Things, fueled by Cloud, Mobile, Social, Analytics and Security, and it represents a $19Trillion value-at-stake over the next 10 years.
In her keynote at @ThingsExpo, Manjula Talreja, VP of Cisco Consulting Services, discussed IoE and the enormous opportunities it provides to public and private firms alike. She will share what businesses must do to thrive in the IoE economy, citing examples from several industry sectors.

There will be 150 billion connected devices by 2020. New digital businesses have already disrupted value chains across every industry. APIs are at the center of the digital business. You need to understand what assets you have that can be exposed digitally, what their digital value chain is, and how to create an effective business model around that value chain to compete in this economy. No enterprise can be complacent and not engage in the digital economy. Learn how to be the disruptor and not the disruptee.

Akana has released Envision, an enhanced API analytics platform that helps enterprises mine critical insights across their digital eco-systems, understand their customers and partners and offer value-added personalized services.
“In today’s digital economy, data-driven insights are proving to be a key differentiator for businesses. Understanding the data that is being tunneled through their APIs and how it can be used to optimize their business and operations is of paramount importance,” said Alistair Farquharson, CTO of Akana.

Business as usual for IT is evolving into a "Make or Buy" decision on a service-by-service conversation with input from the LOBs. How does your organization move forward with cloud? In his general session at 16th Cloud Expo, Paul Maravei, Regional Sales Manager, Hybrid Cloud and Managed Services at Cisco, discusses how Cisco and its partners offer a market-leading portfolio and ecosystem of cloud infrastructure and application services that allow you to uniquely and securely combine cloud business applications and services across multiple cloud delivery models.

The enterprise market will drive IoT device adoption over the next five years.
In his session at @ThingsExpo, John Greenough, an analyst at BI Intelligence, division of Business Insider, analyzed how companies will adopt IoT products and the associated cost of adopting those products.
John Greenough is the lead analyst covering the Internet of Things for BI Intelligence- Business Insider’s paid research service. Numerous IoT companies have cited his analysis of the IoT. Prior to joining BI Intelligence, he worked analyzing bank technology for Corporate Insight and The Clearing House Payment...

"Optimal Design is a technology integration and product development firm that specializes in connecting devices to the cloud," stated Joe Wascow, Co-Founder & CMO of Optimal Design, in this SYS-CON.tv interview at @ThingsExpo, held June 9-11, 2015, at the Javits Center in New York City.

SYS-CON Events announced today that CommVault has been named “Bronze Sponsor” of SYS-CON's 17th International Cloud Expo®, which will take place on November 3–5, 2015, at the Santa Clara Convention Center in Santa Clara, CA. A singular vision – a belief in a better way to address current and future data management needs – guides CommVault in the development of Singular Information Management® solutions for high-performance data protection, universal availability and simplified management of data on complex storage networks. CommVault's exclusive single-platform architecture gives companies unp...

Electric Cloud and Arynga have announced a product integration partnership that will bring Continuous Delivery solutions to the automotive Internet-of-Things (IoT) market. The joint solution will help automotive manufacturers, OEMs and system integrators adopt DevOps automation and Continuous Delivery practices that reduce software build and release cycle times within the complex and specific parameters of embedded and IoT software systems.

"ciqada is a combined platform of hardware modules and server products that lets people take their existing devices or new devices and lets them be accessible over the Internet for their users," noted Geoff Engelstein of ciqada, a division of Mars International, in this SYS-CON.tv interview at @ThingsExpo, held June 9-11, 2015, at the Javits Center in New York City.

Internet of Things is moving from being a hype to a reality. Experts estimate that internet connected cars will grow to 152 million, while over 100 million internet connected wireless light bulbs and lamps will be operational by 2020. These and many other intriguing statistics highlight the importance of Internet powered devices and how market penetration is going to multiply many times over in the next few years.

As organizations realize the scope of the Internet of Things, gaining key insights from Big Data, through the use of advanced analytics, becomes crucial. However, IoT also creates the need for petabyte scale storage of data from millions of devices. A new type of Storage is required which seamlessly integrates robust data analytics with massive scale. These storage systems will act as “smart systems” provide in-place analytics that speed discovery and enable businesses to quickly derive meaningful and actionable insights.
In his session at @ThingsExpo, Paul Turner, Chief Marketing Officer at...

Digital Transformation is the ultimate goal of cloud computing and related initiatives. The phrase is certainly not a precise one, and as subject to hand-waving and distortion as any high-falutin' terminology in the world of information technology.
Yet it is an excellent choice of words to describe what enterprise IT—and by extension, organizations in general—should be working to achieve.
Digital Transformation means:
handling all the data types being found and created in the organization
understanding that through mobility, data is being generated and analyzed on the edges of the e...

“Dad, if my character dies in the game, would I die in the real world?”
What a beautifully naive question that my son, Trevor, asked me during a son-dad conversation about how games might change over the years.
Earlier last year, Mattel’s CEO, Bryan Stockton, was fired. After three years, it was clear that Mattel was continuing to be challenged with sales weakness, and lower gross margins, which drove down shareholder value.
As parents, we ALL know that it’s a very competitive toy aisle, and our kids are much different than we were at their age.

The concept behind the Internet of Things has been around for a while now, ATMs being some of the first enterprise, hardened, network-connected, managed devices for mainstream consumer use. So too with our mobile phones, these are not new concepts to network technicians or hardware geeks. But for the rest of us, we simply never imagined the extents that the "ubiquity of connectedness" would take all other industries, from biotech to automotive, personal care to agriculture, entertainment to custom manufacturing. The list is as long as our imaginations.

Business and IT leaders today need better application delivery capabilities to support critical new innovation. But how often do you hear objections to improving application delivery like, "I can harden it against attack, but not on this timeline"; "I can make it better, but it will cost more"; "I can deliver faster, but not with these specs"; or "I can stay strong on cost control, but quality will suffer"? In the new application economy, these tradeoffs are no longer acceptable. Customers will abandon your brand forever for a slow response or a privacy breach; competitors will steal critical ...

Learn how the IoT Cloud will power the world of tomorrow and why managing IoT through the cloud is as important as cloud computing itself. Learn how the devices of tomorrow will work on business models that reflect a new business strategy and a way to consume services.
In his session at @ThingsExpo, Ian Khan, Manager, Innovation & Marketing at Solgenia, will discuss how powered by the cloud and made possible by high tech manufacturing, sensors and devices with one way and even two way ability of control will devise a new IoT Cloud enabled world.

JavaScript is primarily a client-based dynamic scripting language most commonly used within web browsers as client-side scripts to interact with the user, browser, and communicate asynchronously to servers.
If you have been part of any web-based development, odds are you have worked with JavaScript in one form or another. In this article, I'll focus on the aspects of JavaScript that are relevant within the Node.js environment.

The Internet of Things (IoT) has quickly become the next “be all to end all” in information technology. Touted as how cloud computing will connect everyday things together, it is also feared as the real- life instantiation of The Terminator’s Skynet, where sentient robot team with an omnipresent and all-knowing entity that uses technology to control, and ultimately destroy, all of humanity.

We Need a Holistic Network Infrastructure: Why Controllers Are Not Cutting It
For years, we've relied too heavily on individual network functions or simplistic cloud controllers. However, they are no longer enough for today's modern cloud data center. Businesses need a comprehensive platform architecture in order to deliver a complete networking suite for IoT environment based on OpenStack.
In his session at @ThingsExpo, Dhiraj Sehgal from PLUMgrid discussed what a holistic networking solution should really entail, and how to build a complete platform that is scalable, secure, agile and auto...

Digital Transformation is the process of updating your business and IT infrastructure to align with today's and tomorrow's consumers. Today that is important, but hard to do. Mobile consumer behaviors are changing far faster than most IT budgets and initiatives and that can cause problems. If your customers are adopting technologies and changing their path-to-purchase journeys at a pace that is faster than you can deliver, then you are opening up an opportunity gap for a more nimble competitor.

The multi-trillion economic opportunity around the "Internet of Things" (IoT) is emerging as the hottest topic for investors in 2015. As we connect the physical world with information technology, data from actions, processes and the environment can increase sales, improve efficiencies, automate daily activities and minimize risk.
In his session at @ThingsExpo, Ed Maguire, Senior Analyst at CLSA Americas, will describe what is new and different about IoT, explore financial, technological and real-world impact across consumer and business use cases. Why now?
Significant corporate and venture...

Sensor-enabled things are becoming more commonplace, precursors to a larger and more complex framework that most consider the ultimate promise of the IoT: things connecting, interacting, sharing, storing, and over time perhaps learning and predicting based on habits, behaviors, location, preferences, purchases and more.
In his session at @ThingsExpo, Tom Wesselman, Director of Communications Ecosystem Architecture at Plantronics, examineed the still nascent IoT as it is coalescing, including what it is today, what it might ultimately be, the role of wearable tech, and technology gaps still in...

What if, during a snow emergency, an on-the-ground sensor could automatically trigger a relevant emergency notification related to snowfall and road impact. And then, after it’s triggered, that notification is delivered intelligently to individuals based on an extensive set of rules designed to alert the most available and capable responders.
This “what if” question about “smart highways” is short-sighted. We are already there, and we are only getting started. While mainstream attention is paid to machine-to-machine communications, new technologies are being developed to make these communica...

WebRTC: together these advances have created a perfect storm of technologies that are disrupting and transforming classic communications models and ecosystems.
In his session at WebRTC Summit, Cary Bran, VP of Innovation and New Ventures at Plantronics and PLT Labs, will provide an overview of this technological shift, including associated business and consumer communications impacts, and opportunities it may enable, complement or entirely transform.

A recent purchase of mine was a 2015 Jeep. Until now, I thought it was a safe and reliable car, but that is not the case anymore. As technology advances and cars become smarter and more technologically savvy, they become another target for hackers.
Imagine speeding down the highway, zooming past 18-wheelers, SUVs and coupes, when all of a sudden, your brakes give out and your speed increases. Instantly, you think what could be wrong with the car, you pump your breaks and check your dash and there is no sign of stopping. Instant panic washes over you and a fear of dread drowns out the honkin...

Cloud computing budgets worldwide are reaching into the hundreds of billions of dollars, and no organization can survive long without some sort of cloud migration strategy. Each month brings new announcements, use cases, and success stories.