/*!
\mainpage notitle

\section overview What are scopes

\subsection intro Introduction

One of Unity’s core features on the desktop is the Dash. The Dash allows users to search for and discover virtually anything
from local files and applications, to web content and other online data. The Dash achieves this by interfacing with one or more
search plug-ins called “scopes” (e.g. “Apps”, “Music”, “Videos”, or “Amazon”, “Wikipedia”, “Youtube”).

On the phone and tablet, scopes make up the central user interface, as they provide everything a user needs from an operating
system. Scopes enable users to locate and launch applications, access local files, play music and videos, search the web,
manage their favourite social network, keep up with the latest news, and much more.

Each scope is a dedicated search engine for the category / data source it represents. The data source could be a local database,
a web service, or even an aggregation of other scopes (e.g. the “Music” scope aggregates “Local Music” and “Online Music”
scopes). A scope is primarily responsible for performing the actual search logic and returning the best possible results for each
query it receives.

This document describes how to implement, test and package your own scope using the Unity Scopes C++ API (unity-scopes-api).

\subsection LvsR Local vs remote scopes

Local scopes are scopes that are located and run on the user’s device, while remote scopes (or “Smart Scopes”) are scopes that
are located and run remotely on the Ubuntu Smart Scopes Server (or “SSS”). (Note: Although local scopes execute as local
processes, they may still query online services in order to retrieve search results). A local scope usually requires local data,
and therefore, can only be run locally, while a remote scope can effectively be run both locally and remotely.

When deciding on whether to write a local or remote scope, keep the user’s privacy in mind. For security reasons, a scope should
not access the user’s personal data unless absolutely necessary (i.e. The scope requires account information or local data to
perform searches). It is only in these situations that a scope should be written to run locally. <b>By default, a scope should
be written with the intention of running remotely on the Smart Scopes Server.</b>

(For more information on how to deploy your scope to the Smart Scopes Server, or how to implement a native remote scope using
the SSS REST API see: <i>link_not_yet_available</i>)

\section develop Developing scopes

\subsection starting Getting started

A simple C++ scope template with cmake build system is currently available
on Launchpad, to use it install the packages required for scope development:

\verbatim
sudo apt-get install libunity-scopes-dev
\endverbatim

Clone the bazaar branch with the scope template and build the scope:

\verbatim
bzr branch lp:~jpakkane/+junk/scopetemplate
cd scopetemplate
mkdir build
cd build
cmake ..
make
\endverbatim

Now you're ready to explore and modify the sample code in the src/ directory.

\subsection impl Implementing scope

This short tutorial covers the basic steps and building blocks needed for implementing your own scope with unity-scopes-api, using C++.
For complete examples of various scopes see demo/scopes subdirectory of the unity-scopes-api source project.

A typical scope implementation needs to implement interfaces of the following classes from the Scopes API:
<ul>
<li>unity::scopes::ScopeBase - the main scope class and an entry point for all incoming requests.
<li>unity::scopes::SearchQueryBase - the handler of search request.
<li>unity::scopes::PreviewQueryBase - the handler of preview requests (only if handling previews).
<li>unity::scopes::ActivationQueryBase - the handler of activation and preview action requests (only if handling previews and activation)
<li>unity::scopes::SearchListenerBase - the handler of search replies (only in aggreagator scopes, to pull results from other scopes)
</ul>

The following sections show them in more detail.

\subsubsection simplescope Case 1: A simple scope which doesn't query other scopes.

This is the typical case: a scope that connects to a remote or local backend, database etc. and provides results in response to search queries coming from a
client (i.e. Unity Dash or another scope).

\paragraph scopebase Create a scope class that implements ScopeBase inteface.

There are a few pure virtual methods that need to be implemented; at the very minimum you need to provide a non-empty implementation of
\link unity::scopes::ScopeBase::start() start\endlink and unity::scopes::ScopeBase::search()  and
unity::scopes::ScopeBase::preview() methods. 

\code{.cpp}
class MyScope: public unity::scopes::ScopeBase
{
public:
    virtual int start(std::string const&, unity::scopes::RegistryProxy const&) override;
    virtual void stop() override;
    virtual void run() override;
    virtual unity::scopes::SearchQueryBase::UPtr search(CannedQuery const& query, SearchMetadata const& metadata) override;
    virtual unity::scopes::PreviewQueryBase::UPtr preview(unity::scopes::Result const& result, unity::scopes::ActionMetadata const& metadata) override;
}
\endcode

The \link unity::scopes::ScopeBase::start() start\endlink method must, at the very least return unity::scopes::ScopeBase::VERSION, e.g.

\code{.cpp}
int MyScope::start(string const&, unity::scopes::RegistryProxy const&)
{
    return ScopeBase::VERSION;
}
\endcode

The \link unity::scopes::ScopeBase::stop() stop\endlink method should release any resources, such as network connections where applicable. See the
documentation of \link unity::scopes::ScopeBase ScopeBase\endlink for an explanation of when \link unity::scopes::ScopeBase::run() ScopeBase::run\endlink;
is useful; for typical and simple cases the implementation of run can be an empty function.

\paragraph handlingsearch Handling search

The unity::scopes::ScopeBase::search() method of scope implementation is the entry point of every search - it receives
search queries from the Dash or other scopes. This method must return an instance of an object that implements
unity::scopes::SearchQueryBase interface, e.g:

\code{.cpp}
SearchQueryBase::UPtr MyScope::search(CannedQuery const& query, SearchMetadata const& metadata)
{
    SearchQueryBase::UPtr q(new MyQuery(query));
    return q;
}
\endcode

The search() method receives two arguments: a unity::scopes::CannedQuery query object that carries actual query string (among other information) and
additional parameters of the search request, stored in unity::scopes::SearchMetadata - such as locale string, form factor string and cardinality.
Cardinality is the maximum number of results expected from the scope (the value of 0 should be treated as if no limit was set). For optimal performance scopes
should provide no more results than requested; if they however fail to handle cardinality constraint, any excessive results will be ignored by scopes API.


\paragraph querybase Create a query class that implements SearchQueryBase interface.

The central and most important method that needs to be implemented in this interface is unity::scopes::SearchQueryBase::run().
This is where actual processing of current search query takes place, and this is the spot where you may want to query local or remote data source for results
matching the query.

The unity::scopes::SearchQueryBase::run() method gets passed an instance of SearchReplyProxy, which represents a receiver of query results. Please note
that SearchReplyProxy is just a shared pointer for \link unity::scopes::SearchReply SearchReply\endlink object. The two most important
methods of SearchReply object that every scope have to use are \link unity::scopes::SearchReply::register_category register_category\endlink
and \link unity::scopes::SearchReply::push push\endlink.

The \link unity::scopes::SearchReply::register_category register_category\endlink method is a factory method for creating new categories (see
\link unity::scopes::Category\endlink). Categories can be created at any point during query processing inside run method, but it's recommended to create
them as soon as possible (ideally as soon as they are known to the scope).

When creating a category, one of its parameters is a \link unity::scopes::CategoryRenderer \endlink instance,
which specifies how will a particular category be rendered. See the unity::scopes::CategoryRenderer documentation for more on that subject.

The actual search results have to be wrapped inside \link unity::scopes::CategorisedResult CategorisedResult\endlink objects and
passed to \link unity::scopes::SearchReply::push push\endlink.

A typical implementation of run may look like this:

\code{.cpp}
void MyQuery::run(SearchReplyProxy const& reply)
{
    auto category = reply->register_category("recommended", "Recommended", icon);
    //... query a local or remote backend

    for (auto res: backend.get_results(search_query)) // for every result returned by a backend
    {
        ...
        CategorisedResult result(category); // create a result item in "recommended" category
        result.set_uri(...);
        result.set_title(...);
        result.set_art(...);
        result.set_dnd_uri(...);
        result["my-custom-attribute"] = Variant(...); // add arbitrary data as needed

        if (!reply->push(result)) // send result to the client
        {
            break; // false from push() means search was cancelled
        }
    }
}
\endcode

\paragraph handlingpreview Handling previews

Scopes are responsible for handling preview requests for results they created; this needs to be implemented by overriding unity::scopes::ScopeBase::preview()
method:

\code{.cpp}
class MyScope: public unity::scopes::ScopeBase
{
public:
    ...
    virtual unity::scopes::PreviewQueryBase::UPtr preview(unitu::scopes::Result const& result, unity::scopes::ActionMetadata const& metadata) override;
    ...
}
\endcode

This method must return an instance derived from unity::scopes::PreviewQueryBase. The implementation of unity::scopes::PreviewQueryBase interface is similar to
unity::scopes::SearchQueryBase in that its central method is unity::scopes::PreviewQueryBase::run(). This method is responsible for gathering preview data (from
local or remote sources) and passing it along with the definition of preview look to unity::scopes::PreviewReplyProxy (this is a pointer to
unity::scopes::PreviewReplyBasel; the run() method receives a pointer to an instance of unity::scopes::PreviewReply).

A preview consists of one or more preview widgets - these are the basic building blocks for previews, such as a header with a title and subtitle, an image, a
gallery with multiple images, a list of audio tracks etc.; see unity::scopes::PreviewWidget for a detailed documentation and a list of supported widget types.
So, the implementation of unity::scopes::PreviewQueryBase::run() needs to create and populate one or more instances of unity::scopes::PreviewWidget and push them to the
client with unity::scopes::PreviewReply::push().

Every unity::scopes::PreviewWidget has a unique identifier, a type name and a set of attributes determined by its type. For example, a widget of "image" type expects
two attributes: "source", which should point to an image (an uri) and "zoomable" boolean flag, which determines if the image should be zoomable. Values of such
attributes can either be specified directly, or they can reference values present already in the unity::scopes::Result instance, or pushed spearately during the
execution of unity::scopes::PreviewQueryBase::run().

Attributes can be specified directly with unity::scopes::PreviewWidget::add_attribute_value() method, e.g:

\code{.cpp}
    PreviewWidget image_widget("myimage", "image");

    image_widget.add_attribute_value("source", Variant("file:///tmp/image.jpg"));
    image_widget.add_attribute_value("zoomable", Variant(false));
\endcode

To reference values from results or arbitrary values pushed separately, use unity::scopes::PreviewWidget::add_attribute_mapping() method:

\code{.cpp}
    PreviewWidget image_widget("myimage", "image");
    image_widget.add_attribute_mapping("source", "art"); // use 'art' attribute from the result
    image_widget.add_attribute_mapping("zoomable", "myzoomable"); // 'myzoomable' not specified, but pushed below
    reply->push("myzoomable", Variant(true));
\endcode

To push preview widgets to the client, use unity::scopes::PreviewReply::push():

\code{.cpp}
    PreviewWidget image_widget("myimage", "image");
    PreviewWidget header_widget("myheader", "header");
    // fill in widget attributes
    ...
    PreviewWidgetList widgets { image_widget, header_widget };
    reply->push(widgets);
\endcode

\paragraph previewactions Preview actions

Previews can have actions (i.e. buttons) that user can activate - they are supported by unity::scopes::PreviewWidget of "actions" type. This type of widget
takes one or more action button definitions, where every button is constituted by an unique identifier, a label and an optional icon. For example, a widget with
two buttons: "Open" and "Download" can be defined as follows (using unity::scopes::VariantBuilder helper class):

\code{.cpp}
    PreviewWidget buttons("mybuttons", "actions");

    VariantBuilder builder;
    builder.add_tuple({
        {"id", Variant("open")},
        {"label", Variant("Open")}
    });
    builder.add_tuple({
        {"id", Variant("download")},
        {"label", Variant("Download")}
    });

    buttons.add_attribute_value("actions", builder.end());
\endcode

To handle activation of preview actions, scope needs to implement the following method of unity::scopes::ScopeBase:

\code{.cpp}
class MyScope: public unity::scopes::ScopeBase
{
    virtual ActivationQueryBase::UPtr perform_action(Result const& result, ActionMetadata const& metadata, std::string const& widget_id, std::string const& action_id) override
    ...
}
\endcode

This method receives a widget identifier and action identifier that was activated. 
This method needs to return an instance derived from unity::scopes::ActivationQueryBase. The derived class needs to reimplement
unity::scopes::ActivationQueryBase::activate() method and put any activation logic in there. This method needs to respond with an instance of
unity::scopes::ActivationResponse, which informs the shell about status of activation and the expected behaviour of the UI. For example, activate() may request
a new search query to be executed as follows:

\code{.cpp}
class MyActivation : public unity::scopes::ActivationQueryBase
{
    virtual unity::scopes::ActivationResponse activate() override
    {
        ...
        if (action_id == "search-grooveshark")
        {
            CannedQuery query("com.canonical.scopes.grooveshark");
            query.set_query_string("metal");
            return unity::scopes::ActivationResponse(query);
        }
        ...
    }
}
\endcode

\paragraph handlingactivation Handling result activation

In many cases search results can be activated (i.e. when user taps or clicks them) directly by the shell - as long as a desktop schema (such as "http://")
of result's uri has a handler in the system. If this is the case, then there is nothing to do in terms of activation handling in the scope code. If however a scope relies
on a schema handler that's not present in the system, the offending result will be ignored by Unity shell and nothing will happen on activation.

In cases where scope wants to intercept and handle activation request (e.g. when no handler for specifc type of uri exists, or to do some extra work on activation),
it has to reimplement unity::scopes::ScopeBase::activate() method:

\code{.cpp}
class MyScope : public ScopeBase
{
    virtual ActivationQueryBase::UPtr activate(Result const& result, ActionMetadata const& metadata) override;
    ...
}
\endcode

and also call Result::set_intercept_activation() for all results that should trigger unity::scopes::ScopeBase::activate() on activation. The implementation of
unity::scopes::ScopeBase::activate() should follow the same guidelines as unity::scopes::ScopeBase::perform_action(), the only difference with result activation
being the lack of widget or action identifiers, as those are specific to preview widgets.

\paragraph export Exporting the scope

The scope needs to be compiled into a .so shared library and to be succesfully
loaded at runtime it must provide two C functions to create and destroy it - a
typical code snippet to do this looks as follows:

\code{.cpp}
extern "C" {
    EXPORT unity::scopes::ScopeBase* UNITY_SCOPE_CREATE_FUNCTION()
    {
        return new MyScope();
    }

    EXPORT void UNITY_SCOPE_DESTROY_FUNCTION(unity::scopes::ScopeBase* scope_base)
    {
        delete scope_base;
    }
}
\endcode

\subsubsection aggscope Case 2: A simple aggregator scope.

Aggregator scope is not much different from regular scopes, except for its data sources can include any other scope(s). The main difference is in the
implementation of run method of unity::scopes::SearchQueryBase and in the new class that has to implement
\link unity::scopes::SearchListenerBase SearchListenerBase\endlink interface, which receives result from other scope(s).

\paragraph subquery Query another scopes via SearchQueryBase::subsearch()

To send search query to another scope, use one of the `subsearch()` overloads of unity::scopes::SearchQueryBase inside your implementation of unity::scopes::SearchQueryBase.
This method requires - among search query string - an instance of \link unity::scopes::ScopeProxy ScopeProxy\endlink that points to the target scope and an instance of class that implements
\link unity::scopes::SearchListenerBase SearchListenerBase\endlink interface. ScopeProxy can be obtained from unity::scopes::RegistryProxy and the right place to
do this is in the implementation of \link unity::scopes::ScopeBase::start() start()\endlink method of \link unity::scopes::ScopeBase ScopeBase\endlink interface.

\code{.cpp}

int MyScope::start(std::string const&, unity::scopes::RegistryProxy const& registry)
{
    try
    {
        auto meta = registry->get_metadata("scope-A");
        scope_to_query_ = meta.proxy(); // store the proxy for passing it further in search 
    }
    catch (NotFoundException const& e)
    {
        ...
    }
    return VERSION;
}

unity::scopes::QueryBase::UPtr MyScope::search(CannedQuery const& query, unity::scopes::SearchMetadata const&)
{
    SearchQueryBase::UPtr q(new MyQuery(query, scope_to_query_));
    return q;
}

...

void MyQuery::run(unity::scopes::SearchReplyProxy const& upstream_reply) 
{
    auto category = reply->register_category("recommended", "Recommended", icon, "");
    SearchListenerBase::SPtr reply(new MyReceiver(upstream_reply, category));
    subsearch(scope_to_query_, query_, reply);
    ...
}
\endcode

\paragraph receiver Create a class that implements SearchListenerBase interface

The \link unity::scopes::SearchListenerBase SearchListenerBase\endlink is an abstract class to receive the results of a query sent to a scope. Its virtual push methods let the
implementation receive \link unity::scopes::SearchListenerBase::push result items and categories\endlink returned by that query. A simple implementation of
an aggregator scope may just register all categories it receives and push all received results upstream to the query originator, e.g.

\code{.cpp}

void push(Category::SCPtr category)
{
    upstream_->register_category(category);
}

void MyReceiver::push(CategorisedResult result)
{
    upstream_->push(std::move(result));
}
\endcode

A more sophisticated aggregator scope can rearrange results it receives into a different set of categories, alter or enrich the results before pushing them
upstream etc.

\paragraph aggactiv Activation and previews of results processed by aggregator scopes

If an aggregator scope just forwards results it receives from other scopes, possibly only changing their category assignment, then there is nothing
to do in terms of handling previews, preview actions and result activation: preview and perform_action requests will trigger respective methods of
unity::scopes::ScopeBase for the scope that created results. Result activation will trigger unity::scopes::ScopeBase::activate() method for the scope that
produced the result as long as it set interception flag for it. In other words, when aggreagor scope just forwards results (and makes only minor
adjustements to them, such as category assignment), it is not involved in preview or activation handling at all.

If, however, aggregator scope changes attributes of results (or creates completely new results that "replace" received results), then some extra care needs to
be taken:
<ul>
<li>if original scope should still handle preview (and activation) requests, then <b>aggregator has to store a copy of original result</b> in the modified (or brand
new) result. This can be done with unity::scopes::Result::store method. Preview request for such result will automatically trigger a scope that created the most
inner stored result, and that scope will receive the stored result. It will also do the same for activation as long as the original scope set interception flag
on that result.

\note Making substantial changes to received results and failing to store original results with them may result in unexpected behavior: a scope will
suddenly receive a modified version of it and depending on the level of changes, it may or may not be able to correctly handle it.

<li>if aggregator scope creates a completly new result that replaces original one, but doesn't store a copy of the original result, it is expected to handle
preview (and potentially activation requests - if interception activation flag is set) - this is no different than for normal scopes, see \ref handlingpreview and \ref handlingactivation .
</ul>

Consider the following example of implementation of unity::scopes::SearchListenerBase interface that modifies results and stores their copies, so that original
scope can handle previews and activation for them:

\code{.cpp}
void MyReceiver::push(CategorisedResult original_result)
{
    CategorisedResult result(agg_category); // agg_category is a category that aggregates all results from other scopes
    result.set_uri(original_result.uri());
    result.set_title(original_result.title() + "(aggregated)");
    result.set_art(original_result.art());
    result.store(original_result);

    upstream_->push(std::move(result));
}
\endcode

\subsection scopetesting Testing

Unity Scopes API provides testing helpers based on well-known and established testing frameworks:
<a href="https://code.google.com/p/googletest/">googletest</a> and <a href="https://code.google.com/p/googlemock/">googlemock</a>.
Please see respective documentation of those projects for general information about how to use Google C++ Testing Framework.

All the helper classes provided by Scopes API are located in unity::scopes::testing namespace. The most important ones are:
<ul>
<li>unity::scopes::testing::TypedScopeFixture - template class that takes your scope class name as a template argument and creates a test fixture that can be
used in tests.
<li>unity::scopes::testing::MockSearchReply - a mock of unity::scopes::SearchReply that makes it possible to intercept responses to search request sent from
the scope to a client, making it easy to test if your scope returns all expected data.
<li>unity::scopes::testing::MockPreviewReply - a mock of unity::scopes::PreviewReply that makes is possible to intercept and test responses to preview request
sent from the scope to a client.
<li>unity::scopes::testing::Result - a simple class defined on top of unity::scopes::Result that provides a default constructor, making it possible to create
dummy results (with no attributes) for testing purposes.
<li>unity::scopes::testing::category - a simple class defined on top of unity::scopes::Category that makes it possible to create dummy categories (which would
otherwise require an instance of unity::scopes::SearchReply and a call to unity::scopes::SearchReply::register_category()).
</ul>

With the above classes a test case that checks if MyScope calls appropriate methods of unity::scopes::SearchReply may look like this (note that it just checks
if proper methods get called and uses _ matchers that match any values; put actual values in there for stricts checks):
\code{.cpp}
typedef unity::scopes::testing::TypedScopeFixture<MyScope> TestScopeFixutre;
using namespace ::testing;

TEST_F(TestScopeFixutre, search_results)
{
    const unity::scopes::CategoryRenderer renderer;

    NiceMock<unity::scopes::testing::MockSearchReply> reply;
    EXPECT_CALL(reply, register_departments(_, _)).Times(1);
    EXPECT_CALL(reply, register_category(_, _, _, _))
            .Times(1)
            .WillOnce(
                Return(
                    unity::scopes::Category::SCPtr(new unity::scopes::testing::Category("id", "title", "icon", renderer))
                )
            );
    EXPECT_CALL(reply, register_annotation(_))
            .Times(1)
            .WillOnce(Return(true));
    EXPECT_CALL(reply, push(_))
            .Times(1)
            .WillOnce(Return(true));

    unity::scopes::SearchReplyProxy reply_proxy(&reply, [](unity::scopes::SearchReplyBase*) {}); // note: this is a std::shared_ptr with empty deleter

    unity::scopes::CannedQuery query(scope_id, "", "");
    unity::scopes::SearchMetadata meta_data("en_EN", "phone");

    auto search_query = scope->search(query, meta_data);
    ASSERT_NE(nullptr, search_query);
    search_query->run(reply_proxy);
}
\endcode

\subsection deployment Deployment

Installing a scope is as simple as running `make install` when using the scope
template. You might need to restart the global scope registry when a new scope
is installed by running:

\verbatim
restart scope-registry
\endverbatim

The scope will be installed under one of the "scopes directories"
scanned by the scope registry.  Currently these default to:

<ul>
<li>/usr/lib/${arch}/unity-scopes</li>
<li>/custom/lib/${arch}/unity-scopes</li>
<!-- <li>??? Click packages?</li> -->
</ul>

Individual scopes are installed into a subdirectory matching the
scope's name.  At a minimum, the directory structure should contain
the following:

    -+- ${scopesdir}
     `-+- scopename
       |--- scopename.ini
       `--- libscopename.so

That is, a scope metadata file and a shared library containing the
scope code.  The scope author is free to ship additional data in this
directory (e.g. icons and screenshots).

The scope metadata file uses the standard ini file format, with the
following keys:

    [ScopeConfig]
    DisplayName = human readable name of scope
    Description = description of scope
    Author = Author
    Icon = path to icon representing the scope
    Art = path to screenshot of the scope
    SearchHint = hint text displayed to user when viewing scope
    HotKey =

In addition to allowing the registry to make the scope available, this information controls how the scope appears in the "Scopes" scope.

\subsection scopetool Previewing scope

To help with the development of a scope and to be able to see how will the dash
render the dynamically-specified categories (see unity::scopes::CategoryRenderer),
a specialized tool to preview a scope is provided - the "Unity Scope Tool".

You can install it from the Ubuntu archive using:
\verbatim
sudo apt-get install unity-scope-tool
\endverbatim

After installation, you can run the scope-tool with a parameter specifying
path to your scope configuration file (for example `unity-scope-tool ~/dev/myscope/build/myscope.ini`). If a binary for your scope can be found in the same
directory (ie there's `~/dev/myscope/build/libmyscope.so`), the scope-tool will
display surfacing and search results provided by your scope, and allow you to
perform searches, invoke previews and actions within previews.

Note that the scope-tool is using the same rendering mechanism as Unity
itself, and therefore what you see in the scope-tool is what you get in Unity.
It can also be used to fine-tune the category definitions, as it allows you
to manipulate the definitions on the fly, and once you're happy with the result
you can just copy the JSON definition back into your scope
(see unity::scopes::CategoryRenderer::CategoryRenderer()).

The scope-tool supports a few command line arguments:
\arg by default (without any arguments) it will communicate with all scopes
installed on the system and available on the smart scopes server.
\arg When a path to a scope configuration file is provided, only that scope is
initialized, but you can either pass multiple configuration files or
the `--include-system-scopes` / `--include-server-scopes` option to allow
development of aggregating scopes.

*/
