Export (0) Print
Expand All

Ingesting Assets in Bulk with the REST API

Updated: August 18, 2014

Ingesting Assets in Bulk or “Bulk Ingesting”, involves decoupling asset creation from the upload process. The reasoning for this is uploading large asset files is the bottleneck for asset creation. By using a bulk ingesting approach, you create a manifest (IngestManifest) that describes the asset and its associated files. Then use upload method of your choice to upload the associated files to the manifest’s blob container. Microsoft Azure Media Services watches the blob container associated to the manifest awaiting the file uploads. Once a file is uploaded to the blob container, Microsoft Azure Media Services completes the asset creation based on the configuration of the asset in the manifest (IngestManifestAsset).

This topic describes ingesting assets in bulk using the REST API. For information on ingesting assets in bulk using the Microsoft Azure Media Services SDK, see Ingesting Assets in Bulk with the Media Services SDK for .NET. For more information on Ingesting Assets, see Ingesting Assets with the Media Services REST API.

The basic workflow for bulk ingesting is divided into the following sections:

ImportantImportant
When working with the Media Services REST API, the following considerations apply:

noteNote
Media Services uses the value of the IAssetFile.Name property when building URLs for the streaming content (for example, http://{WAMSAccount}.origin.mediaservices.windows.net/{GUID}/{IAssetFile.Name}/streamingParameters.) For this reason, percent-encoding is not allowed. The value of the Name property cannot have any of the following percent-encoding-reserved characters: !*'();:@&=+$,/?%#[]". Also, there can only be one ‘.’ for the file name extension.

Create the IngestManifest

The IngestManifest is a container for a set of assets, asset files, and statistic information that can be used to determine the progress of bulk ingesting for the set.

Example Request

The following example request was generated with the Example Code provided at the end of this topic.

POST https:// media.windows.net/API/IngestManifests HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
DataServiceVersion: 3.0
MaxDataServiceVersion: 3.0
x-ms-version: 2.5
Authorization: Bearer http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaims%2fnameidentifier=070500D0-F35C-4A5A-9249-485BBF4EC70B&http%3a%2f%2fschemas.microsoft.com%2faccesscontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&Audience=urn%3aWindowsAzureMediaServices&ExpiresOn=1334275521&Issuer=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&HMACSHA256=GxdBb%2fmEyN7iHdNxbawawHRftLhPFFqxX1JZckuv3hY%3d
Host: media.windows.net
Content-Length: 36
Expect: 100-continue

{ "Name" : "ExampleManifestREST" }

Create the Assets

Before creating the IngestManifestAsset, you need to create the Asset that will be completed using bulk ingesting. An asset is a container for multiple types or sets of objects in Media Services, including video, audio, images, thumbnail collections, text tracks, and closed caption files. In the REST API, creating an Asset requires sending a HTTP POST request to Microsoft Azure Media Services and placing any property information about your asset in the request body.In this example, the Asset is created using the StorageEncrption(1) option included with the request body.

Example Request

The following example request was generated with the Example Code provided at the end of this topic.

POST https://media.windows.net/API/Assets HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
DataServiceVersion: 3.0
MaxDataServiceVersion: 3.0
x-ms-version: 2.5
Authorization: Bearer http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaims%2fnameidentifier=070500D0-F35C-4A5A-9249-485BBF4EC70B&http%3a%2f%2fschemas.microsoft.com%2faccesscontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&Audience=urn%3aWindowsAzureMediaServices&ExpiresOn=1334275521&Issuer=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&HMACSHA256=GxdBb%2fmEyN7iHdNxbawawHRftLhPFFqxX1JZckuv3hY%3d
Host: media.windows.net
Content-Length: 55
Expect: 100-continue

{ "Name" : "ExampleManifestREST_Asset", "Options" : 1 }

Create the IngestManifestAssets

IngestManifestAssets represent Assets within an IngestManifest that are used with bulk ingesting. The basically link the asset to the manifest. Microsoft Azure Media Services watches internally for the file upload based on IngestManifestFiles collection associated to the IngestManifestAsset. Once these files are uploaded, the asset is completed. You can create a new IngestManifestAsset with a HTTP POST request. In the request body, include the IngestManifest Id and the Asset Id that the IngestManifestAsset should link together for bulk ingesting.

Example Request

The following example request was generated with the Example Code provided at the end of this topic.

POST https://media.windows.net/API/IngestManifestAssets HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
DataServiceVersion: 3.0
MaxDataServiceVersion: 3.0
x-ms-version: 2.5
Authorization: Bearer http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaims%2fnameidentifier=070500D0-F35C-4A5A-9249-485BBF4EC70B&http%3a%2f%2fschemas.microsoft.com%2faccesscontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&Audience=urn%3aWindowsAzureMediaServices&ExpiresOn=1334275521&Issuer=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&HMACSHA256=GxdBb%2fmEyN7iHdNxbawawHRftLhPFFqxX1JZckuv3hY%3d
Host: media.windows.net
Content-Length: 152
Expect: 100-continue
{ "ParentIngestManifestId" : "nb:mid:UUID:5c77f186-414f-8b48-8231-17f9264e2048", "Asset" : { "Id" : "nb:cid:UUID:b757929a-5a57-430b-b33e-c05c6cbef02e"}}

Optionally Create ContentKeys used for Encryption

If your asset will use encryption, you must create the ContentKey to be used for encryption before creating the IngestManifestFiles for the asset. If you will not need to use encryption, you can skip over this section. Since we use StorageEncryption in the example asset created above, we will demonstrate creating a ContentKey for the asset. To create a new ContentKey for our asset, we send a HTTP POST request. The following properties are included in the request body.

 

Request Body Property Description

Id

The ContentKey Id which we generate ourselves using the following format, “nb:kid:UUID:<NEW GUID>”.

ContentKeyType

This is the content key type as an integer for this content key. We pass the value 1 for storage encryption.

EncryptedContentKey

We create a new content key value which is a 256-bit (32 byte) value. The key is encrypted using the storage encryption X.509 certificate which we retrieve from Microsoft Azure Media Services by executing a HTTP GET request for the GetProtectionKeyId and GetProtectionKey Methods.

ProtectionKeyId

This is the protection key id for the storage encryption X.509 certificate that was used to encrypt our content key.

ProtectionKeyType

This is the encryption type for the protection key that was used to encrypt the content key. This value is StorageEncryption(1) for our example.

Checksum

The MD5 calculated checksum for the content key. It is computed by encrypting the content Id with the content key. The example code demonstrates how to calculate the checksum.

For more details on asset encryption with an x.509 certificate, see Encrypt the Asset (Optional).

Example Create ContentKey Request

The following example request was generated with the Example Code provided at the end of this topic.

POST https://media.windows.net/api/ContentKeys HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
DataServiceVersion: 3.0
MaxDataServiceVersion: 3.0
x-ms-version: 2.5
Authorization: Bearer http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaims%2fnameidentifier=070500D0-F35C-4A5A-9249-485BBF4EC70B&http%3a%2f%2fschemas.microsoft.com%2faccesscontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&Audience=urn%3aWindowsAzureMediaServices&ExpiresOn=1334275521&Issuer=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&HMACSHA256=GxdBb%2fmEyN7iHdNxbawawHRftLhPFFqxX1JZckuv3hY%3d
Host: media.windows.net
Content-Length: 572
Expect: 100-continue

{"Id" : "nb:kid:UUID:316d14d4-b603-4d90-b8db-0fede8aa48f8", "ContentKeyType" : 1, "EncryptedContentKey" : "Y4NPej7heOFa2vsd8ZEOcjjpu/qOq3RJ6GRfxa8CCwtAM83d6J2mKOeQFUmMyVXUSsBCCOdufmieTKi+hOUtNAbyNM4lY4AXI537b9GaY8oSeje0NGU8+QCOuf7jGdRac5B9uIk7WwD76RAJnqyep6U/OdvQV4RLvvZ9w7nO4bY8RHaUaLxC2u4aIRRaZtLu5rm8GKBPy87OzQVXNgnLM01I8s3Z4wJ3i7jXqkknDy4VkIyLBSQvIvUzxYHeNdMVWDmS+jPN9ScVmolUwGzH1A23td8UWFHOjTjXHLjNm5Yq+7MIOoaxeMlKPYXRFKofRY8Qh5o5tqvycSAJ9KUqfg==", "ProtectionKeyId" : "7D9BB04D9D0A4A24800CADBFEF232689E048F69C", "ProtectionKeyType" : 1, "Checksum" : "TfXtjCIlq1Y=" }

Link the ContentKey to the Asset

The ContentKey is associated to one or more assets by sending a HTTP POST request. The following request is an example to link the example ContentKey to the example asset by Id.

Example Link ContentKey Request

The following example request was generated with the Example Code provided at the end of this topic.

POST https://media.windows.net/API/Assets('nb:cid:UUID:b3023475-09b4-4647-9d6d-6fc242822e68')/$links/ContentKeys HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
DataServiceVersion: 3.0
MaxDataServiceVersion: 3.0
x-ms-version: 2.5
Authorization: Bearer http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaims%2fnameidentifier=070500D0-F35C-4A5A-9249-485BBF4EC70B&http%3a%2f%2fschemas.microsoft.com%2faccesscontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&Audience=urn%3aWindowsAzureMediaServices&ExpiresOn=1334275521&Issuer=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&HMACSHA256=GxdBb%2fmEyN7iHdNxbawawHRftLhPFFqxX1JZckuv3hY%3d
Host: media.windows.net
Content-Length: 113
Expect: 100-continue

{ "uri": "https://media.windows.net/api/ContentKeys('nb%3Akid%3AUUID%3A32e6efaf-5fba-4538-b115-9d1cefe43510')"}

Create the IngestManifestFiles for each Asset

An IngestManifestFile represents an actual video or audio blob object that will be uploaded as part of bulk ingesting for an asset. Encryption related properties are not required unless the asset is using an encryption option. The example used in this section demonstrates creating an IngestManifestFile that uses StorageEncryption for the Asset previously created.

Example Request

The following example request was generated with the Example Code provided at the end of this topic.

 POST https://media.windows.net/API/IngestManifestFiles HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
DataServiceVersion: 3.0
MaxDataServiceVersion: 3.0
x-ms-version: 2.5
Authorization: Bearer http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaims%2fnameidentifier=070500D0-F35C-4A5A-9249-485BBF4EC70B&http%3a%2f%2fschemas.microsoft.com%2faccesscontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&Audience=urn%3aWindowsAzureMediaServices&ExpiresOn=1334275521&Issuer=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&HMACSHA256=GxdBb%2fmEyN7iHdNxbawawHRftLhPFFqxX1JZckuv3hY%3d
Host: media.windows.net
Content-Length: 367
Expect: 100-continue

{ "Name" : "REST_Example_File.wmv", "ParentIngestManifestId" : "nb:mid:UUID:5c77f186-414f-8b48-8231-17f9264e2048", "ParentIngestManifestAssetId" : "nb:maid:UUID:beed8531-9a03-9043-b1d8-6a6d1044cdda", "IsEncrypted" : "true", "EncryptionScheme" : "StorageEncryption", "EncryptionVersion" : "1.0", "EncryptionKeyId" : "nb:kid:UUID:32e6efaf-5fba-4538-b115-9d1cefe43510" }

Upload the Files to Blob Storage

You can use any high speed client application capable of uploading the asset files to the blob storage container Uri provided by the BlobStorageUriForUpload property of the IngestManifest. One notable high speed upload service is Aspera On Demand for Azure Application.

Monitor Bulk Ingest Progress

You can monitor the progress of bulk ingesting operations for an IngestManifest by polling the Statistics property of the IngestManifest. That property is a complex type, IngestManifestStatistics. To poll the Statistics property, submit a HTTP GET request passing the IngestManifest Id

Example Request

The following example request was generated with the Example Code provided at the end of this topic.

GET https://media.windows.net/API/IngestManifests('nb:mid:UUID:5c77f186-414f-8b48-8231-17f9264e2048') HTTP/1.1
Content-Type: application/json;odata=verbose
Accept: application/json;odata=verbose
DataServiceVersion: 3.0
MaxDataServiceVersion: 3.0
x-ms-version: 2.5
Authorization: Bearer http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaims%2fnameidentifier=070500D0-F35C-4A5A-9249-485BBF4EC70B&http%3a%2f%2fschemas.microsoft.com%2faccesscontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&Audience=urn%3aWindowsAzureMediaServices&ExpiresOn=1334275521&Issuer=https%3a%2f%2fwamsprodglobal001acs.accesscontrol.windows.net%2f&HMACSHA256=GxdBb%2fmEyN7iHdNxbawawHRftLhPFFqxX1JZckuv3hY%3d
Host: media.windows.net

Example Code

This example code requires references to the following assemblies. Target Framework is set to .NET Framework 4 in the project settings.

  • System.Runtime.Serialization

  • System.Web

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;
using System.Web;
using System.Runtime.Serialization.Json;
using System.Xml;
using System.Security.Cryptography;
using System.Security.Cryptography.X509Certificates;

namespace Microsoft.Samples.RestBulkIngest
{
  class Program
  {

    private static Uri serviceURI = new Uri("https://media.windows.net/API/");
    private static readonly string accessControlURI = "https://wamsprodglobal001acs.accesscontrol.windows.net/v2/OAuth2-13";
    private static readonly string clientSecret = ""; // Fill in your client secret/passcode here.
    private static readonly string clientId = ""; // Fill in your client ID here
    private static readonly string scope = "WindowsAzureMediaServices";

    private static string token = null;


    static void Main(string[] args)
    {
      //===[ Retrieves an ACS token ]=============================================================//
      token = GetACSToken(accessControlURI, clientId, clientSecret, scope);

      //===[ Create a named IngestManifest ]======================================================//
      Console.WriteLine("\n===============================================");
      Console.WriteLine("========[ CREATE BULK INGEST MANIFEST ]========");
      Console.WriteLine("===============================================");
      XmlDocument xmlManifestDoc = CreateIngestManifest("ExampleManifestREST");
      string manifestId = xmlManifestDoc.GetElementsByTagName("Id")[0].InnerText;


      //===[ Create an StorageEncryption Asset for bulk ingesting ]===============================//
      Console.WriteLine("\n===============================================");
      Console.WriteLine("================[ CREATE ASSET ]===============");
      Console.WriteLine("===============================================");
      XmlDocument assetXmlResponse = CreateAsset("ExampleManifestREST_Asset", "1");
      string assetId = assetXmlResponse.GetElementsByTagName("Id")[0].InnerText;


      //===[ Create an IngestManifestAsset to link the asset to the manifest ]====================//
      Console.WriteLine("\n===============================================");
      Console.WriteLine("=======[ CREATE INGEST MANIFEST ASSET ]========");
      Console.WriteLine("===============================================");
      XmlDocument xmlManifestAssetDoc = CreateIngestManifestAsset(manifestId, assetId);
      string manifestAssetId = xmlManifestAssetDoc.GetElementsByTagName("Id")[0].InnerText;


      //===[ Create a ContentKey for encryption ]==================================================//
      Console.WriteLine("\n===============================================");
      Console.WriteLine("============[ CREATE CONTENT KEY ]=============");
      Console.WriteLine("===============================================");
      int storageEncryptionType = 1;
      XmlDocument xmlContentKey = CreateContentKey(storageEncryptionType);
      string contentKeyId = xmlContentKey.GetElementsByTagName("Id")[0].InnerText;


      //===[ Link the ContentKey to the Asset for encryption ]=====================================//
      Console.WriteLine("\n===============================================");
      Console.WriteLine("==========[ LINK CONTENTKET TO ASSET ]=========");
      Console.WriteLine("===============================================");
      LinkContentKeyToAsset(assetId, xmlContentKey.GetElementsByTagName("uri")[0].InnerText);


      //===[ Create the IngestManifestFile so that Azure Media Service knows what ]========//
      //===[ files to look for in the blob container associated to the manifest.          ]========//
      Console.WriteLine("\n===============================================");
      Console.WriteLine("========[ CREATE INGEST MANIFEST FILE ]========");
      Console.WriteLine("===============================================");
      CreateIngestManifestFile(manifestId, manifestAssetId, "REST_Example_File.wmv", contentKeyId);


      //===[ Poll the IngestManifest Statistics property for bulk ingress progress ]===============//
      Console.WriteLine("\n===============================================");
      Console.WriteLine("======[ POLL INGEST MANIFEST STATISTICS ]======");
      Console.WriteLine("===============================================");
      PollIngestManifestStatisticsToCompletion(manifestId);
    }



    private static string GetACSToken(string accessControlUri, string clientId, string clientSecret, string scope)
    {
      HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(accessControlUri);
      request.Method = "POST";
      request.ContentType = "application/x-www-form-urlencoded";
      request.KeepAlive = true;
      string token = null;

      //Note: You need to insert your client Id and secret into this string in order for it to work.
      var requestBytes = Encoding.ASCII.GetBytes("grant_type=client_credentials&client_id=" + clientId + "&client_secret=" + HttpUtility.UrlEncode(clientSecret) + "&scope=urn%3a" + scope);
      request.ContentLength = requestBytes.Length;

      var requestStream = request.GetRequestStream();
      requestStream.Write(requestBytes, 0, requestBytes.Length);
      requestStream.Close();

      var response = (HttpWebResponse)request.GetResponse();

      if (response.StatusCode == HttpStatusCode.OK)
      {
        using (Stream responseStream = response.GetResponseStream())
        {
          using (StreamReader stream = new StreamReader(responseStream))
          {
            string responseString = stream.ReadToEnd();
            var reader = JsonReaderWriterFactory.CreateJsonReader(Encoding.UTF8.GetBytes(responseString), new XmlDictionaryReaderQuotas());

            while (reader.Read())
            {
              if ((reader.Name == "access_token") && (reader.NodeType == XmlNodeType.Element))
              {
                if (reader.Read())
                {
                  token = reader.Value;
                  break;
                }
              }
            }
          }
        }
      }

      return token;
    }



    private static XmlDocument GenerateRequestAndGetResponse(string verb, string resourcePath, string query, string requestbody)
    {
      var uriBuilder = new UriBuilder(serviceURI);
      uriBuilder.Path += resourcePath;
      if (query != null)
      {
        uriBuilder.Query = query;
      }
      HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uriBuilder.Uri);
      request.Method = verb;
      request.AllowAutoRedirect = false;

      if (resourcePath == "$metadata")
        request.MediaType = "application/xml";
      else
      {
        request.ContentType = "application/json;odata=verbose";
        request.Accept = "application/json;odata=verbose";
      }

      // Uncomment when Media Services supports the JSON OData light.
      //request.ContentType = "application/json;odata=light";
      //request.Accept = "application/json;odata=light";

      request.Headers.Add("DataServiceVersion", "3.0");
      request.Headers.Add("MaxDataServiceVersion", "3.0");
      request.Headers.Add("x-ms-version", "2.5");
      request.Headers.Add(HttpRequestHeader.Authorization, "Bearer " + token);

      if (requestbody != null)
      {
        var requestBytes = Encoding.ASCII.GetBytes(requestbody);
        request.ContentLength = requestBytes.Length;

        var requestStream = request.GetRequestStream();
        requestStream.Write(requestBytes, 0, requestBytes.Length);
        requestStream.Close();
      }
      else
      {
        request.ContentLength = 0;
      }

      XmlDocument xmlResponse = null;

      try
      {
        // Generate HTTP request and pass in JSON defining the Asset's name.    
        var response = (HttpWebResponse)request.GetResponse();

        //TODO: Put in retry logic if token has expired.
        switch (response.StatusCode)
        {
          case HttpStatusCode.MovedPermanently:
            serviceURI = new Uri(response.Headers["Location"]);
            xmlResponse = GenerateRequestAndGetResponse(verb, resourcePath, query, requestbody);
            break;
          case HttpStatusCode.Created:
          case HttpStatusCode.OK:
            using (Stream responseStream = response.GetResponseStream())
            {
              using (StreamReader stream = new StreamReader(responseStream))
              {
                string responseString = stream.ReadToEnd();
                var reader = JsonReaderWriterFactory.CreateJsonReader(Encoding.UTF8.GetBytes(responseString), new XmlDictionaryReaderQuotas());

                xmlResponse = new XmlDocument();
                reader.Read();
                xmlResponse.LoadXml(reader.ReadInnerXml());
              }
            }
            break;

          default:
            Console.WriteLine(response.StatusDescription);
            break;
        }
      }
      catch (WebException ex)
      {
        Console.WriteLine(ex.Message);
      }

      return xmlResponse;
    }


    private static XmlDocument CreateAsset(string name, string options = null)
    {
      XmlDocument assetXmlResponse = null;

      string requestbody;

      if (options != null)
      {
        requestbody = "{ \"Name\" : \"" + name + "\", \"Options\" : " + options + " }";
      }
      else
      {
        requestbody = "{ \"Name\" : \"" + name + "\" }";
      }


      // Generate HTTP request and pass in JSON defining the Asset's name.    
      assetXmlResponse = GenerateRequestAndGetResponse("POST", "Assets", null, requestbody);

      Console.WriteLine("\nAsset Id: {0}", assetXmlResponse.GetElementsByTagName("Id")[0].InnerText);
      Console.WriteLine("Name: {0}", assetXmlResponse.GetElementsByTagName("Name")[0].InnerText);
      Console.WriteLine("State: {0}", assetXmlResponse.GetElementsByTagName("State")[0].InnerText);
      Console.WriteLine("Options: {0}", assetXmlResponse.GetElementsByTagName("Options")[0].InnerText);

      return assetXmlResponse;
    }


    private static XmlDocument CreateIngestManifest(string name)
    {
      XmlDocument manifestXmlResponse = null;

      manifestXmlResponse = GenerateRequestAndGetResponse("POST", "IngestManifests", null, "{ \"Name\" : \"" + name + "\" }");

      Console.WriteLine("\nManifest Id: {0}", manifestXmlResponse.GetElementsByTagName("Id")[0].InnerText);
      Console.WriteLine("Name: {0}", manifestXmlResponse.GetElementsByTagName("Name")[0].InnerText);
      Console.WriteLine("State: {0}", manifestXmlResponse.GetElementsByTagName("State")[0].InnerText);

      return manifestXmlResponse;
    }


    private static XmlDocument CreateIngestManifestAsset(string manifestId, string assetId)
    {
      XmlDocument manifestAssetXmlResponse = null;

      string requestbody;

      requestbody = "{ \"ParentIngestManifestId\": \"" + manifestId + "\", " +
                      "\"Asset\" : { \"Id\": \"" + assetId + "\" } }";


      // Generate HTTP request and pass in JSON defining the Asset's Id and the Manifest Id.    
      manifestAssetXmlResponse = GenerateRequestAndGetResponse("POST", "IngestManifestAssets", null, requestbody);

      Console.WriteLine("\nIngestManifestAsset Id: {0}", manifestAssetXmlResponse.GetElementsByTagName("Id")[0].InnerText);
      Console.WriteLine("ParentIngestManifest Id: {0}", manifestAssetXmlResponse.GetElementsByTagName("ParentIngestManifestId")[0].InnerText);

      return manifestAssetXmlResponse;
    }


    private static XmlDocument CreateIngestManifestFile(string manifestId, string manifestAssetId, string fileName, string encryptionKeyId = null)
    {
      XmlDocument manifestAssetFileXmlResponse = null;

      string requestbody;

      if (encryptionKeyId == null)
      {
        requestbody = "{ \"Name\" : \"" + fileName + "\", " +
        "\"ParentIngestManifestId\" : \"" + manifestId + "\", " +
        "\"ParentIngestManifestAssetId\" : \"" + manifestAssetId + "\" " +
        "}";
      }
      else
      {
        requestbody = "{ \"Name\" : \"" + fileName + "\", " +
        "\"ParentIngestManifestId\" : \"" + manifestId + "\", " +
        "\"ParentIngestManifestAssetId\" : \"" + manifestAssetId + "\", " +
        "\"IsEncrypted\" : \"true\", " +
        "\"EncryptionScheme\" : \"StorageEncryption\", " +
        "\"EncryptionVersion\" : \"1.0\", " +
        "\"EncryptionKeyId\" : \"" + encryptionKeyId + "\" " +
        "}";
      }


      // Generate HTTP request and pass in JSON defining the Asset's name.    
      manifestAssetFileXmlResponse = GenerateRequestAndGetResponse("POST", "IngestManifestFiles", null, requestbody);

      Console.WriteLine("\nIngestManifestFile Id: {0}", manifestAssetFileXmlResponse.GetElementsByTagName("Id")[0].InnerText);

      return manifestAssetFileXmlResponse;
    }


    private static XmlDocument GetEntity(string resourcePath, string query = null)
    {
      XmlDocument xmlResponse = null;

      xmlResponse = GenerateRequestAndGetResponse("GET", resourcePath, query, null);

      return xmlResponse;
    }


    private static void PollIngestManifestStatisticsToCompletion(string manifestId)
    {
      int pollInterval = 30000; // ms

      Console.WriteLine("\nBulk Ingest Statistics Polling Interval : {0} sec.\n", pollInterval / 1000);

      bool bContinue = true;

      while (bContinue)
      {
        XmlDocument manifest = GetEntity("IngestManifests('" + manifestId + "')");

        Console.WriteLine("Waiting on all file uploads.");
        Console.WriteLine("PendingFilesCount: " + manifest.GetElementsByTagName("PendingFilesCount")[0].InnerText);
        Console.WriteLine("FinishedFilesCount: " + manifest.GetElementsByTagName("FinishedFilesCount")[0].InnerText);
        Console.WriteLine("ErrorFilesCount: " + manifest.GetElementsByTagName("ErrorFilesCount")[0].InnerText);
        Console.WriteLine("ErrorFilesDetails: {0}\n", manifest.GetElementsByTagName("ErrorFilesDetails")[0].InnerText);

        if (Convert.ToInt32(manifest.GetElementsByTagName("PendingFilesCount")[0].InnerText) == 0)
          bContinue = false;
        else
          System.Threading.Thread.Sleep(pollInterval);
      }
    }



    private static XmlDocument CreateContentKey(int encryptionType)
    {
      XmlDocument contentKeyXmlResponse = null;

      AesCryptoServiceProvider myAes = new AesCryptoServiceProvider();

      // This value of myAes.Key will be the 256 byte content key that must be used for encrypting the file before upload. //
      byte[] contentKey = myAes.Key;
      Console.WriteLine("\nUnencrypted content key : {0}\n", Convert.ToBase64String(contentKey));

      if (encryptionType != 1)
        return null;

      //===[ Get the protection key Id for StorageEncryption(1) X.509 certificate ]=================================//
      XmlDocument xmlProtectionKeyId = GetEntity("GetProtectionKeyId", "contentKeyType=" + encryptionType.ToString());
      string protectionKeyId = xmlProtectionKeyId.GetElementsByTagName("GetProtectionKeyId")[0].InnerText;

      //===[ Get the protection key for StorageEncryption(1) X.509 certificate ]==============================//
      XmlDocument xmlProtectionKey = GetEntity("GetProtectionKey", "ProtectionKeyId='" + protectionKeyId + "'");

      //===[ use the X509Certificate2 class to create a RSACryptoServiceProvider to use for encryption ]===//
      string certString = xmlProtectionKey.GetElementsByTagName("GetProtectionKey")[0].InnerText;
      byte[] certBytes = Convert.FromBase64String(certString);
      X509Certificate2 certToUse = new X509Certificate2(certBytes);
      var rsaProvider = (RSACryptoServiceProvider)certToUse.PublicKey.Key;

      //===[ Encrypt the content key using the RSACryptoServiceProvider ]=============================//
      string encryptedContentKey = Convert.ToBase64String(rsaProvider.Encrypt(contentKey, false));
      Console.WriteLine("Encrypted content key : {0}\n", encryptedContentKey);

      //===[ NOTE: ContentKey has user-defined Id value ]===//
      var contentKeyGuidId = Guid.NewGuid();

      //===[ Generate a MD5 checksum of the 
      string checkSumMD5 = CalculateContentKeyChecksum(contentKey, contentKeyGuidId);

      string requestBody = "{\"Id\" : \"nb:kid:UUID:" + contentKeyGuidId + "\", " +
                            "\"ContentKeyType\" : 1, " +
                            "\"EncryptedContentKey\" : \"" + encryptedContentKey + "\", " +
                            "\"ProtectionKeyId\" : \"" + protectionKeyId + "\", " +
                            "\"ProtectionKeyType\" : " + encryptionType.ToString() + ", " +
                            "\"Checksum\" : \"" + checkSumMD5 + "\" }";

      contentKeyXmlResponse = GenerateRequestAndGetResponse("POST", "ContentKeys", null, requestBody);

      string contentKeyId = contentKeyXmlResponse.GetElementsByTagName("Id")[0].InnerText;
      Console.WriteLine("ContentKey Id: {0}\n", contentKeyId);

      // From __metadata uri element...
      string contentKeyUri = contentKeyXmlResponse.GetElementsByTagName("uri")[0].InnerText;
      Console.WriteLine("__metadata uri : {0}", contentKeyUri);

      return contentKeyXmlResponse;
    }


    public static string CalculateContentKeyChecksum(byte[] contentKey, Guid keyId)
    {
      const int ChecksumLength = 8;
      const int KeyIdLength = 16;

      byte[] encryptedKeyId = null;

      // Checksum is computed by AES-ECB encrypting the KID
      // with the content key.
      using (AesCryptoServiceProvider rijndael = new AesCryptoServiceProvider())
      {
        rijndael.Mode = CipherMode.ECB;
        rijndael.Key = contentKey;
        rijndael.Padding = PaddingMode.None;

        ICryptoTransform encryptor = rijndael.CreateEncryptor();
        encryptedKeyId = new byte[KeyIdLength];
        encryptor.TransformBlock(keyId.ToByteArray(), 0, KeyIdLength, encryptedKeyId, 0);
      }

      byte[] retVal = new byte[ChecksumLength];
      Array.Copy(encryptedKeyId, retVal, ChecksumLength);

      return Convert.ToBase64String(retVal);
    }


    private static void LinkContentKeyToAsset(string assetId, string contentKeyUri)
    {
      XmlDocument xmlResponse = GenerateRequestAndGetResponse("POST", "Assets('" + assetId + "')/$links/ContentKeys", null, "{ \"uri\": \"" + contentKeyUri + "\"}");
    }
  }
}

See Also


Build Date:

2014-09-09

Community Additions

ADD
Show:
© 2014 Microsoft