Introduction

What are scopes

Introduction

One of Unity’s core features on the desktop is the Dash. The Dash allows users to search for and discover virtually anything from local files and applications, to web content and other online data. The Dash achieves this by interfacing with one or more search plug-ins called “scopes” (e.g. “Apps”, “Music”, “Videos”, or “Amazon”, “Wikipedia”, “Youtube”).

On the phone and tablet, scopes make up the central user interface, as they provide everything a user needs from an operating system. Scopes enable users to locate and launch applications, access local files, play music and videos, search the web, manage their favourite social network, keep up with the latest news, and much more.

Each scope is a dedicated search engine for the category / data source it represents. The data source could be a local database, a web service, or even an aggregation of other scopes (e.g. the “Music” scope aggregates “Local Music” and “Online Music” scopes). A scope is primarily responsible for performing the actual search logic and returning the best possible results for each query it receives.

This document describes how to implement, test and package your own scope using the Unity Scopes C++ API (unity-scopes-api).

Developing scopes

Getting started

A simple C++ scope template with cmake build system is currently available as part of the Ubuntu SDK IDE. To use it install the packages required for scope development:

sudo apt-get install libunity-scopes-dev

In the Ubuntu SDK you will have to decide whether your scope will either access the network, or access the local filesystem.

Now you're ready to explore and modify the sample code in the src/ directory.

Click packaging

To register your scope, you must use the "scope" click hook, and point it to a directory containing your .ini file and .so file. In the template, a manifest like the follow is used:

{

"description": "Net scope description",

"framework": "ubuntu-sdk-14.04-clibs",

"hooks": {

"myscope": {

"scope": "myscope", <-- Point to directory in build tree with .ini and .so

"apparmor": "scope-security.json" <-- Point to AppArmor manifest in build tree

}

}

"maintainer": "Some Guy <some.guy@ubuntu.com>",

"name": "com.ubuntu.developer.username.net-scope",

"title": "Some scope",

"version": "0.1"

}

Apparmor manifest

Scopes that are packaged using click are inherently untrusted and must be confined. At present, there are two different choices that can be made:

Network scope - can access the network / internet, but is not allowed to use APIs that could provide access to the user's data.

Local-content scope - can access local APIs that could provide access to user's data, but cannot access the network / internet.

For a scope confined to access local content only:

{

"template": "ubuntu-scope-local-content",

"policy_version": 1.1

}

For a scope confined to only be able to access the internet:

{

"template": "ubuntu-scope-network",

"policy_groups": [

"networking"

],

"policy_version": 1.1

}

Implementing scope

This short tutorial covers the basic steps and building blocks needed for implementing your own scope with unity-scopes-api, using C++. For complete examples of various scopes see demo/scopes subdirectory of the unity-scopes-api source project.

A typical scope implementation needs to implement interfaces of the following classes from the Scopes API:

The stop method should release any resources, such as network connections where applicable. See the documentation of ScopeBase for an explanation of when ScopeBase::run; is useful; for typical and simple cases the implementation of run can be an empty function.

The search() method receives two arguments: a unity::scopes::CannedQuery query object that carries actual query string (among other information) and additional parameters of the search request, stored in unity::scopes::SearchMetadata - such as locale string, form factor string and cardinality. Cardinality is the maximum number of results expected from the scope (the value of 0 should be treated as if no limit was set). For optimal performance scopes should provide no more results than requested; if they however fail to handle cardinality constraint, any excessive results will be ignored by scopes API.

Create a query class that implements SearchQueryBase interface.

The central and most important method that needs to be implemented in this interface is unity::scopes::SearchQueryBase::run(). This is where actual processing of current search query takes place, and this is the spot where you may want to query local or remote data source for results matching the query.

The register_category method is a factory method for creating new categories (see unity::scopes::Category). Categories can be created at any point during query processing inside run method, but it's recommended to create them as soon as possible (ideally as soon as they are known to the scope).

Every unity::scopes::PreviewWidget has a unique identifier, a type name and a set of attributes determined by its type. For example, a widget of "image" type expects two attributes: "source", which should point to an image (an uri) and "zoomable" boolean flag, which determines if the image should be zoomable. Values of such attributes can either be specified directly, or they can reference values present already in the unity::scopes::Result instance, or pushed spearately during the execution of unity::scopes::PreviewQueryBase::run().

Preview actions

Previews can have actions (i.e. buttons) that user can activate - they are supported by unity::scopes::PreviewWidget of "actions" type. This type of widget takes one or more action button definitions, where every button is constituted by an unique identifier, a label and an optional icon. For example, a widget with two buttons: "Open" and "Download" can be defined as follows (using unity::scopes::VariantBuilder helper class):

Handling result activation

In many cases search results can be activated (i.e. when user taps or clicks them) directly by the shell - as long as a desktop schema (such as "http://") of result's uri has a handler in the system. If this is the case, then there is nothing to do in terms of activation handling in the scope code. If however a scope relies on a schema handler that's not present in the system, the offending result will be ignored by Unity shell and nothing will happen on activation.

In cases where scope wants to intercept and handle activation request (e.g. when no handler for specifc type of uri exists, or to do some extra work on activation), it has to reimplement unity::scopes::ScopeBase::activate() method:

Exporting the scope

The scope needs to be compiled into a .so shared library and to be succesfully loaded at runtime it must provide two C functions to create and destroy it - a typical code snippet to do this looks as follows:

Case 2: A simple aggregator scope.

Aggregator scope is not much different from regular scopes, except for its data sources can include any other scope(s). The main difference is in the implementation of run method of unity::scopes::SearchQueryBase and in the new class that has to implement SearchListenerBase interface, which receives result from other scope(s).

Create a class that implements SearchListenerBase interface

The SearchListenerBase is an abstract class to receive the results of a query sent to a scope. Its virtual push methods let the implementation receive result items and categories returned by that query. A simple implementation of an aggregator scope may just register all categories it receives and push all received results upstream to the query originator, e.g.

void push(Category::SCPtr category)

{

upstream_->register_category(category);

}

void MyReceiver::push(CategorisedResult result)

{

upstream_->push(std::move(result));

}

A more sophisticated aggregator scope can rearrange results it receives into a different set of categories, alter or enrich the results before pushing them upstream etc.

Activation and previews of results processed by aggregator scopes

If an aggregator scope just forwards results it receives from other scopes, possibly only changing their category assignment, then there is nothing to do in terms of handling previews, preview actions and result activation: preview and perform_action requests will trigger respective methods of unity::scopes::ScopeBase for the scope that created results. Result activation will trigger unity::scopes::ScopeBase::activate() method for the scope that produced the result as long as it set interception flag for it. In other words, when aggreagor scope just forwards results (and makes only minor adjustements to them, such as category assignment), it is not involved in preview or activation handling at all.

If, however, aggregator scope changes attributes of results (or creates completely new results that "replace" received results), then some extra care needs to be taken:

if original scope should still handle preview (and activation) requests, then aggregator has to store a copy of original result in the modified (or brand new) result. This can be done with unity::scopes::Result::store method. Preview request for such result will automatically trigger a scope that created the most inner stored result, and that scope will receive the stored result. It will also do the same for activation as long as the original scope set interception flag on that result.

Note

Making substantial changes to received results and failing to store original results with them may result in unexpected behavior: a scope will suddenly receive a modified version of it and depending on the level of changes, it may or may not be able to correctly handle it.

if aggregator scope creates a completly new result that replaces original one, but doesn't store a copy of the original result, it is expected to handle preview (and potentially activation requests - if interception activation flag is set) - this is no different than for normal scopes, see Handling previews and Handling result activation .

Consider the following example of implementation of unity::scopes::SearchListenerBase interface that modifies results and stores their copies, so that original scope can handle previews and activation for them:

void MyReceiver::push(CategorisedResult original_result)

{

CategorisedResult result(agg_category); // agg_category is a category that aggregates all results from other scopes

result.set_uri(original_result.uri());

result.set_title(original_result.title() + "(aggregated)");

result.set_art(original_result.art());

result.store(original_result);

upstream_->push(std::move(result));

}

Testing

Unity Scopes API provides testing helpers based on well-known and established testing frameworks: googletest and googlemock. Please see respective documentation of those projects for general information about how to use Google C++ Testing Framework.

All the helper classes provided by Scopes API are located in unity::scopes::testing namespace. The most important ones are:

unity::scopes::testing::TypedScopeFixture - template class that takes your scope class name as a template argument and creates a test fixture that can be used in tests.

unity::scopes::testing::MockSearchReply - a mock of unity::scopes::SearchReply that makes it possible to intercept responses to search request sent from the scope to a client, making it easy to test if your scope returns all expected data.

unity::scopes::testing::MockPreviewReply - a mock of unity::scopes::PreviewReply that makes is possible to intercept and test responses to preview request sent from the scope to a client.

With the above classes a test case that checks if MyScope calls appropriate methods of unity::scopes::SearchReply may look like this (note that it just checks if proper methods get called and uses _ matchers that match any values; put actual values in there for stricts checks):

In addition to allowing the registry to make the scope available, this information controls how the scope appears in the "Scopes" scope.

Previewing scope

To help with the development of a scope and to be able to see how will the dash render the dynamically-specified categories (see unity::scopes::CategoryRenderer), a specialized tool to preview a scope is provided - the "Unity Scope Tool".

You can install it from the Ubuntu archive using:

sudo apt-get install unity-scope-tool

After installation, you can run the scope-tool with a parameter specifying path to your scope configuration file (for example unity-scope-tool ~/dev/myscope/build/myscope.ini). If a binary for your scope can be found in the same directory (ie there's ~/dev/myscope/build/libmyscope.so), the scope-tool will display surfacing and search results provided by your scope, and allow you to perform searches, invoke previews and actions within previews.

Note that the scope-tool is using the same rendering mechanism as Unity itself, and therefore what you see in the scope-tool is what you get in Unity. It can also be used to fine-tune the category definitions, as it allows you to manipulate the definitions on the fly, and once you're happy with the result you can just copy the JSON definition back into your scope (see unity::scopes::CategoryRenderer::CategoryRenderer()).

The scope-tool supports a few command line arguments:

by default (without any arguments) it will communicate with all scopes installed on the system and available on the smart scopes server.

When a path to a scope configuration file is provided, only that scope is initialized, but you can either pass multiple configuration files or the --include-system-scopes / --include-server-scopes option to allow development of aggregating scopes.