Table of contents
Last Updated: 12/13/2016
Important

You must apply to join the Cortana Developer Preview Program to create and use a Cortana skill.
Enroll in the Cortana developer preview today.

Register an app and associate it with a predefined Cortana skill

This tutorial demonstrates how to register a Universal Windows app called Contoso Café as a provider for the Order Food Cortana skill.

Overview

This app is designed to demonstrate the following basic features common to most food ordering apps:

  • A menu of food items
  • A shopping cart
  • A check out process
Note

Payment functionality is not relevant for this tutorial.

The OrderFood skill is presented to the user when:

These provide the following info to Cortana:

  • Food preferences (from the Cortana Notebook)
  • Meeting or appointment location (from the Calendar)
  • Meeting or appointment time (from the Calendar)

This info is appended to a deep link URI and passed to the Contoso Café app when the user selects the link, or automatically if the user has enabled Cortana delegation.

Remember, your app is the end point for the data provided by Cortana. Be creative with how you use it.

Important

A user must grant Cortana permission to use insights.

Prerequisites

Experience with C# and XAML

Windows 10 (Anniversary Update or Windows Insider Preview)

Visual Studio 2015 (Community Edition or greater)

The Contoso Café app

Registered account with the Cortana Developer Preview program

Step by step

This tutorial uses both the Contoso Café app and the Cortana dashboard to demonstrate Cortana's OrderFood skill.

Contoso Café

Do not close Visual Studio, we'll return to the Contoso Café app in a later step.

Cortana dashboard

The Cortana dashboard is where you create and configure all Cortana skills.

  1. Log in

    Go to the Cortana dashboard and log in with your registered Microsoft account. (The dashboard is where you create and configure your skills.)

    Note

    To create Cortana skills, you must register for the Cortana Developer Preview program. If you are already registered, you are redirected to the dashboard.

    For details and instructions on joining the developer preview program, see Applying to the Developer Preview.

    If you have any issues, please send an email to cortanaap@microsoft.com.

  2. Welcome

    Select the "Create new skill" button.

    Welcome screen for the Cortana developer preview program

    Welcome screen for the Cortana developer preview program

  3. Create new Cortana skill

    You can choose to create a skill from scratch, or you can choose from a set of predefined skills related to functionality provided by your app or website.

    The predefined skills are:

    • Order Food: Supports ordering food for pickup or delivery.
    • Send Message: Supports sending messages.

    Predefined skills are more deeply integrated into the Cortana experience. They can be presented on both the Cortana canvas and in a Cortana notification, they can be activated through both speech and text input, and they can be activated without direct user interaction through delegation.

    If the predefined skills are not suited to your app or web site, you can define your own skill. These custom skills can appear in the Cortana canvas and in Cortana notifications, but cannot be activated through speech and text input, and cannot take advantage of Cortana delegation.

    For this example, we are using the Order Food skill to integrate the Contoso Café app with Cortana.

    Select the *OrderFood" radio button, and click Next.

    Add a new OrderFood skill

    Add a new OrderFood skill

  4. Configure your skill

    This screen includes three sections:

    Context

    Here, you specify the contextual info that is provided through either the insight or direct user interaction.

    You cannot change the core info provided, but you can choose to include some addtional info.

    For this example, leave the default insights.

    Deep links

    This is where you specify the deep link URIs for the supported end points (Windows, Android, or website).

    The contextual info is appended to the deep link URI in the form of query string parameters (name-value pairs).

    The query string name is whatever is recognized by your app and the value is the corresponding slot names specified in the Context section.

    1. Select Windows from the list of platforms.
    2. Enter the following in the Deep link field: contosocafe://deeplink?cuisine=@@CuisinePreference@@&location=@@DeliveryLocation@@&time=@@DeliveryTime@@
    3. In the App service name field, enter "contosocafe".
    4. In the Package family name field, enter "contoso".
    5. Select Done to save the deep link.

    Add the Windows app deep link
    Add the Windows app deep link

    About

    Provide the metadata details for the skill, including name, display text, and provider name.

    For this example, enter the following info:

    • Action name (required): OrderFood
    • Display text (required): Order food
    • Provider name (required): Contoso
    • Description: Order food
    • Market availability: US
    • Website: No need to specify a website for this exercise, the website is used to rank Cortana skills
    • Icon (required): Find the icon in the assets folder of the app
  5. Submit your skill.

    On the Configure your skill screen, select Save to submit your skill to Cortana.

    Once the dashboard and Cortana are synchronized, your OrderFood skill is triggered for each calendar event that meets the specified insight criteria.

    Order Food skill presented on the Cortana canvas
    Order Food skill presented on the Cortana canvas

  6. Test your skill.

    As mentioned earlier, the OrderFood skill requires at least one of the following insights to be true before it is shown to the user:

    Alternatively, the skill can be invoked by a user when they instruct Cortana (through text or speech using Cortana's support for natural language) to order food.

    In addition, the user needs to configure a few things on their machine before the OrderFood skill can be presented by Cortana:

    • Cortana Notebook

      A user must specify the following in the Cortana Notebook:

      • About me
        • Favorite places -> Home and Work locations
      • Permissions
        • Location
        • Contacts, email, calendar, and communication history
      • Eat & drink
        • Eat & drink cards: On
        • Lunch Conflict Experience: On
        • Cuisine preference: One or more preferences specified
        • Lunch schedule: A typical lunch time specified
      • On the go
        • All on the go cards: On
    • Calendar

      The user must also have an event (meeting or appointment) that conflicts with the period of time specified in Eat & drink -> Lunch schedule.

      This info is routed through Cortana to your app using the deep link parameters you specified previously.

  7. Debug

    In the Cortana dashboard, select the Debugging link at the top of the page to open the Debugging your Cortana skills page.

    On this page, select the Enable debug mode for my Cortana checkbox in the Cortana canvas.

    Enable debug mode for my Cortana
    Enable Debug Mode in Cortana

    When selected, special cards are displayed on the Cortana home canvas to help with debugging.

    Debug mode enabled for my Cortana
    Debug cards displayed in Cortana when Debug Mode is enabled

  8. Enable support for the OrderFood skill in the Contoso Café app.

    Return to the Contoso Café app in Visual Studio.

    1. Create a new

    In Visual Studio, create a new Contoso Skills class.

    In Solution Explorer right click on the ContosoCafe project and select Add | New Folder from the context menu displayed. Name the new folder Cortana

    Figure 7 Add Cortana folder to solution
    Figure 7 Add Cortana folder to solution

  9. Right click on the new folder and select Add | Class… from the context menu. Name the class CortanaActions.cs

    Figure 8 Add CortanaSkills.cs class Figure 8 Add CortanaSkills.cs class

  10. Edit the new CortanaSkills.cs file to make the class public and add a method named Activated which takes IActivatedEventArgs as a parameter. This code will not compile as you need to add two new methods; CreateRootFrame and HandleAction. The Activated method will be called from the App code when Cortana triggers the deep link to be activated.

    using ContosoCafe.Models;
    using ContosoCafe.Common;
    using ContosoCafe.ViewModels;
    using ContosoCafe.Views;
    
    using Windows.ApplicationModel.Activation;
    using Windows.UI.Xaml;
    using Windows.UI.Xaml.Controls;
    
    namespace ContosoCafe.Cortana
    {
    public class CortanaSkills
    {
        public void Activated(IActivatedEventArgs args)
        {
            Frame rootFrame = CreateRootFrame();
            if (args.Kind == ActivationKind.Protocol)
            {
                HandleSkill(args, rootFrame);
            }
        }
    }
    }
    
  11. Add the CreateRootFrame method in the CortanaSkills class. This method gets the current frame hosted in the Window. If none exists a new one is created. It will be used to navigate to another frame once the context of the action is determined.

       private Frame CreateRootFrame()
       {
           Frame rootFrame = Window.Current.Content as Frame;
    
           if (rootFrame == null)
           {
               rootFrame = new Frame();
               Window.Current.Content = rootFrame;
           }
           return rootFrame;
       }
    
  12. In order to extract the parameter value from the deep link when Cortana invokes the Skill you will need some code. Add a method to the CortanaSkills class named QueryParamterValue

    private string QueryParameterValue(string query, string parameter)
    {
        string value = string.Empty;
    
        string[] parts = query.Split('&');
    
        foreach(string p in parts)
        {
            if (p.StartsWith(parameter))
            {
                value = p.Substring(p.IndexOf('=') + 1);
                break;
            }
        }
    
        return value;
    }
    
  13. One of the parameters of the deep link is the time, this is passed as a FileTime, add a new method to the CortanSkills class to convert this to a C# DateTime

    private DateTime TimeStampToDateTime(long timeStamp)
    {
         return DateTime.FromFileTime(timeStamp);
    }
    
  14. Add the HandleSkill method to the CortanaSkills class. The arguments providing insight will be extracted from the deep link uri and then a new order created using the Delivery Location and Delivery Time parameters. (The cuisine choice is ignored in the this example, and instead the last order placed by the customer is used for the delivery)

    private async void HandleAction(IActivatedEventArgs args, Frame    rootFrame)
    {
            ProtocolActivatedEventArgs protocolArgs = args as    ProtocolActivatedEventArgs;
    
            if (protocolArgs != null)
            {
                Uri link = protocolArgs.Uri;
                string cuisines = QueryParameterValue   (link.Query, "cuisine");
                string location = QueryParameterValue   (link.Query, "location");
                string time = QueryParameterValue(link.Query,    "time");
    
                DateTime deliveryTime = TimeStampToDateTime   (long.Parse(time));
    
                List<OrderHistory> Orders = await    XmlService.GetOrderHistoryData();
                if (Orders.Count > 0)
                {
                    CheckoutViewModel checkout = new    CheckoutViewModel(Orders[0]);
                    checkout.Delivery = true;
                    checkout.DeliveryAddress = location;
                    checkout.DeliveryTime = deliveryTime;
    
                    rootFrame.Navigate(typeof(Checkout),    checkout);
                } 
                else
                {
                    rootFrame.Navigate(typeof(Menu));
                }
            }
            else
            {
                rootFrame.Navigate(typeof(Menu));
            }
            Window.Current.Activate();
    }
    
  15. In order for this to work the OnActivated method of the Application needs to be overridden and call the CortanaSkills.Activated method to handle the Skill. In the App.xaml.cs file override the OnActivated method as shown.

    protected override void OnActivated(IActivatedEventArgs args)
    {
        base.OnActivated(args);
        Cortana.CortanaActions actions = new    Cortana.CortanaActions();
        actions.Activated(args);
    }
    
  16. Finally you need to register the app to support the protocol, this is done in the App Manifest file. Find the Package.appxmanifest in the solution explorer, and right click on it. From the context menu select view code. In the Application element you need to add an Extensions element as shown.

    <Applications>
    <Application Id="App"
      Executable="$targetnametoken$.exe"
      EntryPoint="ContosoCafe.App">
      <uap:VisualElements
        DisplayName="ContosoCafe"
        Square150x150Logo="Assets\Square150x150Logo.png"
        Square44x44Logo="Assets\Square44x44Logo.png"
        Description="ContosoCafe"
        BackgroundColor="transparent">
        <uap:DefaultTile    Wide310x150Logo="Assets\Wide310x150Logo.png"/>
        <uap:SplashScreen Image="Assets\SplashScreen.png" />
      </uap:VisualElements>
      <Extensions>
        <uap:Extension Category="windows.protocol">
          <uap:Protocol Name="contosocafe">
            <uap:DisplayName>Contoso Cafe</uap:DisplayName>
          </uap:Protocol>
        </uap:Extension>
      </Extensions>
    </Application>
    </Applications>
    
  17. Rebuild the application and run it (F5). Make an order for pickup. Remember what you ordered.

  18. Select the Cortana search from the Windows task bar and find an item which meets the criteria of the insights (you might have to change your lunch time preferences and create a meeting at that time). You should then see the order food action available.

    Figure 9 Order food Skill when there is a meeting conflict with lunch Figure 9 Order food action when there is a meeting conflict with lunch

  19. When you select this you will be prompted to share the context insights with the contoso café app

    Figure 10 provide Cortana permission to share your meeting data and preferences with the app Figure 10 provide Cortana permission to share your meeting data and preferences with the app

  20. Then the app will launch showing your last order for delivery this time.

    Figure 11 Contoso Cafe showing the last order for deliver Figure 11 Contoso Cafe showing the last order for delivery

  21. Click the Order for Delivery button and you will see the time and location of the meeting has been used for the order.

    Figure 12  Delivery Address and Delivery Time added to the order and shown in the checkout page Figure 12 Delivery Address and Delivery Time added to the order and shown in the checkout page

Congratulations you have now built you first application using Cortana Skills. The most valuable apps will take advantage of the insights and contextual information which Cortana can provide in the Deep link.

What can you do to improve the user experience of your app by integrating with Cortana Skill ?

© 2017 Microsoft