Protractor: lessons learned

Dec 13, 2017

I’ve been working with Protractor a lot in recent monts,
writing automated tests for the web UI of our company’s customer portal.
In this post, I thought I’d share some of the lessons I’ve learned along the way,
and give some practical advice.

Here’s an outline to help you decide which parts of this article will be of use you:

browser.wait and ExpectedConditions:
if you’re not sure your UI will update fast enough for your test to pass, use
these Protractor features to delay the test until the update is done.

Marker attributes: relying on template HTML structure or
CSS class names to find elements in your test couples your test to those
structures, making them more likely to fail any time you change something in
your UI styling. Instead, use special markers that don’t change when your
styling or layout does, like classes with a predefined prefix, or a custom
attribute.

Protractor and the Promise Manager:
Protractor ships with the Promise Manager system enabled by default, but it’s
scheduled to be deprecated. If you start writing Protractor code now, disable
this system manually so you won’t have to migrate in the future. Use async /
await instead.

Mock backend: to test your app’s behaviour with unexpected
HTTP responses, you need to control those responses from the tests. Building
a mock implementation of your backend may be a more realistic and flexible way
to do this than overriding parts of the app directly and never sending the
request in the first place. Fortunately, building a fake backend is pretty
easy to do.

PO layer: it’s a good idea to build an abstraction layer between
the structure of your UI and your Protractor tests. The example given in the
Protractor doc is outdated though: here I show the structure we came up with.

Conclusion: your UI tests are a first-class citizen of your
code base. Put thought into its structure and design before writing your first
tests, or you’ll incur technical debt, like I did.

browser.wait and ExpectedConditions

One of the most common problems I run into when using Protractor is its
sensitivity to timing issues. It’s easy to ask Protractor to click a button
and then check that a certain element has appeared. But if some complex logic is
involved, or even a lengthy HTTP request, then it may take your UI some time to
update. If Protractor looks for the element to appear before it does, then it
rightly throws an error. And this error is unfortunately rather non-deterministic:
in some test runs your UI may update just in time, and no error is thrown.

Preventing such false positives requires you to put a lot of thought into your
tests. If you’re not sure the UI will update in time for your tests to pass,
you may need to delay your tests until the update is complete. Fortunately,
Protractor gives you browser.wait and ExpectedConditions to do just that.
For example:

This will test the same behavior as expect(element(by.id("my-text")).isPresent()).toBe(true)
would, but it gives your UI a 500ms window in which to update. If it doesn’t,
a timeout error will be thrown from browser.wait.

Marker attributes

Protractor gives you a lot of options to select elements in your web UI based
on tag names, CSS classes, or complicated selectors. However, by using these
methods of finding elements in your UI, your coupling your Protractor tests to
the structure of your HTML, which isn’t completely stable. So every time you go
in and add a <div> or change a class name, you may inadvertently break your
Protractor tests. That’s far from ideal.

One solution is to agree to a naming-convention for classes used to match UI
elements in your tests. Something like <button class="e2e-submit"> for example.
The class prefix shows that this element is used in your end-to-end tests.
So you know not to use that class for styling purposes, or to change the button
without updating your tests.

Protractor and the Promise Manager

Protractor is built for asynchronous execution. So if you issue a command, like
element(by.tagName('button')).click(), the action will not necessarily be done
by the time the execution of that statement is complete. However, if you’ve
written any Protractor code before you may not have been aware of this.
The reason for that is that WebDriver, which Protractor is built on, ships with
a system called the PromiseManager. Roughly speaking, this system keeps track
of all the asynchronous commands issues through the WebDriver API, and makes
sure the last command has been completed before the next starts.

The Promise Manager is a convenient system, but it is
scheduled to be deprecated.
This system was added in a time before Promises and async / await were added
to the ECMA Script standard. The Promise Manager adds a lot of complexity,
making code harder to debug. With these new language features it becomes simple
enough to just manage asynchronous operations yourself. A Protractor test
built with async / await might look something like this:

describe('The contact form',()=>{it('ensures the subject field is filled out before submitting',async()=>{awaitbrowser.get('/contact');constsubjectfield=element(by.css('input[name="subject"]'));consterror=element(by.id('subject-error'));awaitexpect(subjectfield.getAttribute('value')).toBe('');awaitexpect(error.isPresent()).toBe(false);awaitelement(by.css('input[type="submit"]')).click();awaitexpect(error.isPresent()).toBe(true);awaitexpect(error.getText()).toBe('Please enter a subject for your message');});});

If you decide to start adding Protractor code to your project, I advice you to
start writing async / await code right from the start so you won’t have to
migrate later. Async / await should work natively starting with Node 8, but if
that doesn’t work for you, you can always have it transpiled using e.g. Babel or
TypeScript. If you do decide to manage asynchronous timing yourself, don’t
forget to disable the Promise Manager system (it is still enabled by default for
now, though that will change as part of the deprecation process). Check the
instructions for using Protractor with async / await.

Mock backend

To completely test your UI’s behaviour, you have to check what it does if an
AJAX resolves slowly, or fails altogether. To do that, you have to control the
responses to your UI’s requests. There are several ways to do this. For instance,
you can route all requests in your application through one component, and then
override that component during tests.

However, I much prefer a different approach: overriding only the URL of requests,
and pointing them to a mock-implementation of the backend. By really doing the
HTTP request and processing the response, your tests will be far closer to actual
production behaviour.

Building a mock-implementation of your backend is not all that difficult. For my
last two projects I’ve built mock backends using Express.
A simple mock backend resource in Express might look something like this:

As you can see, this backend supports testing of a couple of scenarios,
triggered by adding certain GET parameters. By default, the predefined dummy
invoice data will be sent, with a delay of 200ms to more realistically simulate
a network request (this backend runs on the same system that runs the tests,
and without this delay the response would be far too fast). However, by adding
load-failure=true to the URL in some way, the app will be served a
500 Internal Server Error after 500ms. As you can see, this method gives you
a lot of flexibility to test all kinds of possible states of your production
backend and see how your app responds to them.

PO layer

The Protractor documentation advises that you build an abstraction layer between
the structure of your UI and your tests. They call it a
Page Object.
Here’s an example from the Protractor doc:

Apart from the outdated ES5 syntax, this is a fine start to your abstraction
layer. As my colleagues and I found out, though, just because Protractor calls
this a Page Object, that doesn’t mean it’s a smart move to write exactly one
such class per page. I suggest writing your PO classes at roughly the same level
as your UI components.

After some experimenting, we came up with a more hierarchical structure that
groups properties and methods by UI element, using nested plain objects.
Here’s an example (see the Marker attributes section
to see what by.marker does).

describe('The Addresses component',()=>{consttest=newAddressesTest();constoverview=test.addresses.overview;beforeEach(()=>test.load());it('shows the title and description from the CMS',async()=>{awaitexpect(test.addresses.title).toBe("Your addresses");awaitexpect(test.addresses.description).toBe("Manage your addresses.");});it('loads and shows an overview of addresses',async()=>{awaitexpect(overview.present).toBe(true);awaitexpect(overview.addressCount).toBeGreaterThan(0);awaitexpect(overview.getAddress(1).name).toBe("Address 1");});describe('if there are no addresses',()=>{beforeEach(()=>test.load({noAddressData:true}));it('doesn\'t show the overview',async()=>{awaitexpect(overview.present).toBe(false);});});});

Nice and simple, no? My colleagues and I have been using TypeScript for a while
now, and it makes importing these support classes and writing the actual test a
breeze.

Of course, you’re free to organize the PO layer as you see fit. I do recommend
that you take some time, before you start writing tests, to think about how
you want this abstraction layer to be structured. We started out just
following the example from the Protractor documentation. It took us weeks
to reorganize the abstraction layer to what I’ve shown here.

Conclusion

The moral of this story is simple: Test code is just as important as production code.
It is not a second class citizen. It requires thought, design and care.
It must be kept as clean as production code.

– Clean Code, Robert C. Martin

This is basically the lesson I learned by writing an extensive Protractor test
suite. Your testing code is a first-class part of the project. Adding tests
without thinking about your code design or architecture will land you in no less
of a mess than doing the same thing while writing production code.
By the time I realized this, it took me a few weeks to get rid of the technical
debt I myself had incurred. So next time, I will be putting a lot more thought
into the design of the entire code base, including the tests.
And I advise you to do the same. 😉