Archive

Regular skill testing can help you deliver consistently great and reliable voice experiences to customers. But as your skill becomes more complex, testing might become a time-consuming task if you have to test hundreds of possible dialogs manually. The right tools can help you simulate complex interactions in an automated way, which reduces the level of effort spent on repetitive tasks.

Test automation can also help you find bugs early in the cycle of the software development process – and the earlier you find a bug, the more time you have to fix it Furthermore, when multiple people work on the same code base, having test automation before checking in the code can prevent regressions from happening. As a result, your customers will benefit from a consistent and reliable voice experience.

We previously shared a high-level overview of automated testing and some best practices here and here, and introduced a cmd-line simulator called TestFlow. Today’s post shares how to create unit tests to validate a skill’s output against predefined assertions.

Types of Automated Tests for Alexa Skills

The classic test pyramid (referenced in the diagram below) outlines the different types of testing you can leverage to create an effective and balanced test automation suite. You should have many more unit tests than tests that validate the voice interaction with Alexa. In this way, you can address the major functional test cases with unit tests and you can treat the voice user interface (VUI) interaction as a sanity test to ensure the voice experience works as expected.

Unit testing involves breaking Alexa skill backend code into pieces and subjecting each piece to a series of tests. We declare the correct outputs and conditions we expect to see from each unit test. The testing tool will report on the number of passing and failing tests.

These frameworks are easy to set up and they support different types of skill features like cards, dialog models, Amazon DynamoDB, and session attributes. Under the covers they rely on standard test packages like Jest or Mocha.

In this post, we’ll use BrainMacInosh’s Alexa-Skill-Test-Framework to test a sample skill. If you’d like to follow along on your own device, make sure you have Node.JS and GitHub installed.

Sample Skill Unit Testing

Step 1: Clone the sample skill to your local desktop and install the node_modules dependency

Navigate to “test” folder, run “mocha helloworld-tests.js”. You will be able to see the test result for your Hello World sample Skill.

Write Your Own Tests Under Test Folder

Example 1: Unit Test for Hello World Skill

Now that we are setup, we will continue to use the Hello World Skill Test example to learn how to write custom unit tests. This test case test snippet demonstrates how to test the launch request in the Hello World Skill. The script runs “alexaTest.getLaunchRequest()” to simulate a launch request, and then defines the expected speech output under “says”. “repromptsNothing”: true means no reprompt is included in the launch request.

After you run the test, errors are displayed. From the screenshot you should see two errors, and we will be focusing on the first one:

The reason we see this error is that we expected speech output of “Welcome to the Alexa Skill Kit…” does not match the actual value of “Hello World”.

Example 2: Unit Test for Checking Session Attributes

Replace the code in index.js with the code from this helloworld.js file. This skill uses session attributes to store values inside a session.

Here is the portion of the code that sets the session attributes in this sample skill; it calls an attributeManager from handlerInput to get all the session attributes and assigns them to a variable. We can customize the session attributes value for this sample, change “attributes.foo = ‘bar’” to “attributes.activity = ‘eating’”.

To test, in the test file under the “describe” block, we simply need to put the expected pair of values after hasAttributes. If you are using version 1 of the ASK SDK, the test method implementation is the same. Here is the sample test code:

Build Skills, Earn Developer Perks

Bring your big idea to life with Alexa and earn perks through our milestone-based developer promotion. US developers, publish your first Alexa skill and earn a custom Alexa developer t-shirt. Publish a skill for Alexa-enabled devices with screens and earn an Echo Spot. Publish a skill using the Gadgets Skill API and earn a 2-pack of Echo Buttons. If you're not in the US, check out our promotions in Canada, the UK, Germany, Japan, France, Australia, and India. Learn more about our promotion and start building today.