September 2015
Volume 30 Number 9
Microsoft Azure - Protect Sensitive Information with Azure Key Vault
By Rahul Nath | September 2015
Many applications built today deal with sensitive information in some form or another, often connection strings to an external system such as a database. These keys are typically deployed as part of the application’s configuration file and are available as plain text to anyone who can access the deployment servers. Clearly, this poses a huge security threat.
Microsoft Azure Key Vault is a cloud-hosted, hardware security modules (HSM)-backed service for managing cryptographic keys and other sensitive information—secrets in Key Vault terminology—in a central location, and can be accessed via a REST API by specifying the appropriate URI for the key or secret. Key Vault also allows you to create a software key, instead of an HSM-backed one; in both cases the private portion of the key doesn’t leave the Key Vault boundary and can never be seen or shared.
In this article, I’ll walk you through a scenario in which a large company, Contoso, is building a line-of-business (LOB) application, with development outsourced to various vendors. The application interacts with multiple third-party services and deals with sensitive information, including customer personally identifiable information (PII). I’ll look first at the existing implementation and the security issues this poses for the company, then show how the company uses Azure Key Vault to solve these issues.
The Existing Application
Contoso is a large car manufacturing company currently building an LOB application to provide a connected car experience for customers, vehicles and other interested parties on a single platform. Because Contoso has only a small IT team in-house, it outsourced the development of the application to various vendors, each building different parts of the application. The application is cloud-hosted and exposes different APIs for various entities to interact with it. It also uses third-party application APIs to consume data from external services, and calls into internal Contoso applications exposed via Web services.
The application deals with PII, which is confidential and should be stored securely. Initially, not much thought was given to how sensitive information should be handled, and various vendors had their own approaches. Most of the connection strings were part of the application configuration file so it could be changed per deployment; some keys used for encrypting PII information were hardcoded into the application; and some of the certificate details to connect with the Contoso Web services were in the application’s database. This became unmanageable for Contoso, and the company ended up sharing the sensitive information with the application development vendors, which isn’t recommended. Contoso also realized that anybody who had access to their deployment servers could very easily tamper with the PII data, which would be a serious threat. The company was forced to unify the approach for maintaining and accessing sensitive information.
The Application Refit
Contoso looked into various alternatives for managing sensitive information and unifying the experience so it could be made consistent across different applications. The company wanted a central store for its keys that could be easily managed by its security team, and it wanted to provide different applications with different access policies to the information. Moreover, Contoso didn’t want any of this to require day-to-day monitoring with a lot of maintenance overhead.
The company chose Azure Key Vault, which allows the use of software and HSM-backed keys, as well as importing existing keys from a PFX file. Key Vault allows you to store small bits of information (secrets) securely in the cloud and access them as required. Because this service is completely managed and maintained on the Azure platform, you are freed from all the overhead of maintaining it.
Key Vault is built in such a way that the keys can’t be seen or extracted by anyone, making it a very secure platform. It provides a rich API for accessing and interacting with the vault, as well as a C# API library for applications that run on the Microsoft .NET Framework. The ability to control how different applications and vendors can interact with it is a significant factor that makes Key Vault a good choice for Contoso, as the company wants to restrict the ways various application vendors can interact with the secure information. Moreover, Contoso engineers already heavily use Windows PowerShell scripts for managing their various other servers, and Key Vault is completely manageable through Windows PowerShell.
Creating the Key Vault: The security administrator of the Contoso IT team is responsible for setting up the Key Vault and managing the keys and secrets. Azure Key Vault can be created and managed using Windows PowerShell cmdlets and is available with the latest Azure PowerShell (0.9.2 or higher). Once connected to Azure subscriptions at the Azure PowerShell prompt, you need to switch to AzureResourceManager mode, which the Key Vault cmdlets require. Then, using the New-AzureKeyVault cmdlet, you can create a new vault by specifying a vault name (which is globally unique), resource group and location:
Switch-AzureMode AzureResourceManager
New-AzureResourceGroup –Name 'ContosoResourceGroup' –Location 'East Asia'
New-AzureKeyVault -VaultName 'ContosoKeyVault' -ResourceGroupName
'ContosoResourceGroup' -Location 'East Asia'
On successful execution of the commands, the details of the newly created key vault are output to the Azure PowerShell console, which includes the vault name and the URI to uniquely identify the vault.
Setting up the Key Vault: Azure Key Vault enables creation and storage of cryptographic keys and also the storage of secrets, which are limited-size octet objects that have no specific semantics.
Cryptographic keys in the vault are represented as JSON Web Key objects and currently only RSA keys are supported. Once a key is created in the vault, only the public portion of the key is available outside the vault boundary.
A key vault can contain both keys and secrets, and external access to these can be controlled separately. Both keys and secrets are versioned objects in the vault and a new one is uniquely identified by the vault URL and the object name and its version number. Whenever an object is created with an existing name, a new object with the same name and a new version number is created and this becomes the current version. Trying to access an object without a version number returns the current version. The object identifier, which is used to uniquely identify an object within a vault, takes the following format, and is used to interact with keys and secrets using the API:
https://{keyvault-name}.vault.azure.net/{object-type}/{object-name}/{object-version}
Where:
keyvault-name : Globally unique key vault name
object-type : Either "keys" or "secrets," indicating the type of object
object-name : Unique name within a key vault
object-version : System generated string, optionally used to
identify a specific version of object
Keys in Azure Key Vault are represented as JSON Web Key (JWK) objects, with the base JWK specifications extended to enable key types that are unique to the Azure Key Vault implementation. Currently Azure Key Vault supports only the RSA algorithm, an asymmetric algorithm. Azure Key Vault supports Create, Import, Update, Delete, List, Get, Backup and Restore operations on keys, and these operations are available using the REST API and Windows PowerShell cmdlets. Because the Contoso application uses keys to encrypt and decrypt certain PII information, a key needs to be created in the vault. Contoso engineers use the following script to add a new key to the Vault, with a name that’s unique within their vault:
Add-AzureKeyVaultKey -VaultName 'ContosoKeyVault' -Name 'ContosoPIIKey'
-Destination 'Software'
On successful execution this creates a new software RSA key in the ContosoKeyVault, which can be identified with a unique identifier (for example, https://ContosoKeyVault.vault.azure.net/keys/ContosoPIIKey/ bfacf5f768ae42ffb0a0bca448aead87), which the engineers will share with the application vendors.
Secrets in Azure Key Vault are octet sequences with a maximum size of 25K each; any type of data is accepted and stored securely. The Key Vault supports Create, Get, List, Delete and Update operations on secrets, and these operations are available using the REST API and Windows PowerShell cmdlets. Because the Contoso application saves connection strings in the application configuration files, Contoso engineers decide to move this over to the Key Vault, where it can be stored securely and still be accessible using a unique identifier. The following script was used to add their SQL database connection string to the vault:
$ContosoSQLConnectionString = ConvertTo-SecureString -String
"ContosoSQLConnectionString"
-AsPlainText -Force
Set-AzureKeyVaultSecret -VaultName "ContosoKeyVault" -Name
"ContosoSQLConnectionString"
-SecretValue $ContosoSQLConnectionString
On successful execution this creates a secret value in the ContosoKeyVault, which can be identified using a unique identifier (such as https://ContosoKeyVault.vault.azure.net/secrets/ContosoSQLConnectionString/ 90018dbb96a84117a0d2847ef8e7189d), which again the engineers share with the application vendors.
Authenticating applications to use Key Vault: The newly created vault is currently accessible only from the Azure account that created it and not by anyone else. Contoso needs to give various client applications access to the vault. Access to an Azure key vault is secured using an Azure Active Directory (Azure AD) application token. To do this, the Contoso engineers first need to create an application in the Azure AD and secure it with either an authentication key (a shared secret) or a certificate. Creating an Azure AD application secured using authentication key access can be accomplished using the Azure Management Portal, under the Applications tab in the Active Directory option. But securing its key vault with an authentication key is not what Contoso prefers, as that would again mean putting sensitive information in the client application configuration file, which is something they wanted to avoid in the first place. Also, because the key vault is the single store for all its sensitive information, allowing access to the credentials to it is something that should be strongly avoided. Therefore, Contoso decides to use certificate-based authentication, so it can deploy the certificate directly to the application requiring it and further can protect the certificate using a password.
Creating an Azure AD application that uses certificate authentication can be done via the Windows PowerShell command line. Contoso already has a mechanism to generate certificates for securing its other internal applications, so the company uses the same service to generate a certificate for the Azure AD application. Once the company has the certificate, the Azure AD application can be created using the following script:
$certificateFilePath = "C:\certificates\ContosoADApplication.cer"
$x509Certificate2 = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$x509Certificate2.Import($certificateFilePath)
$rawCertData = $x509Certificate2.GetRawCertData()
$credentialValue = [System.Convert]::ToBase64String($rawCertData)
$startDate= [System.DateTime]::Now
$endDate = $startDate.AddYears(1)
$newADApplication = New-AzureADApplication -DisplayName
"ContosoKeyVaultADApplication"
-HomePage "https://www.contoso.com" -IdentifierUris "https://www.contoso.com"
-KeyValue $credentialValue -KeyType "AsymmetricX509Cert" -KeyUsage "Verify"
-StartDate $startDate -EndDate $endDate
With the key vault and the Azure AD application created, Contoso now has to link these together so the application token received from the Azure AD application can be used to authenticate against the key vault. This can be done using the Set-AzureKeyVaultAccessPolicy cmdlet. You also set the permissions the Azure AD application has on the key vault at this time, using the PermissionToKeys and PermissionToSecrets parameters.
The PermissionToKeys parameter specifies an array of key operation permissions to be granted to the application and has a list of acceptable values (Decrypt, Encrypt, UnwrapKey, WrapKey, Verify, Sign, Get, List, Update, Create, Import, Delete, Backup, Restore, All). The PermissionToSecrets parameter specifies an array of secret operation permissions to be granted to the application and also has a list of acceptable values (Get, List, Set, Delete, All). The permissions are applicable to all the keys and secrets in the vault; giving selective permissions to certain keys or secrets in the vault isn’t allowed. If selective permission to keys or secrets is required, you need to create separate vaults with the selected keys and secrets to manage the permissions. There is no setup fee for the key vault itself, but this could change and you would want to check key vault pricing before creating separate vaults. To give different access levels to the same keys and secrets, you can create multiple Azure AD applications and register them with different permission levels.
Contoso engineers use the following script to associate the Azure AD application and the key vault, and to specify the permissions the application has on the vault:
$ServicePrincipal = New-AzureADServicePrincipal -ApplicationId $newADApplication.ApplicationId
Set-AzureKeyVaultAccessPolicy -VaultName 'ContosoKeyVaultRahul' -ObjectId $ServicePrincipal.Id
-PermissionsToKeys encrypt,decrypt,get -PermissionsToSecrets get
$ServicePrincipal.ApplicationId
Connecting to Key Vault from client application: Once the Azure AD application has been created and associated with the key vault, you can use it from a client application to authenticate with the key vault. Because the Contoso application is developed on the .NET platform, the application vendors will use the Microsoft.Azure.KeyVault NuGet package (bit.ly/1Ji6xcS) to connect to the key vault.
Authenticating a client application with the Azure AD application is easily accomplished using the Azure AD Authentication Library (ADAL), which is also available as a NuGet package. All the application needs to authenticate with the Azure AD application is a client ID and the certificate identifier (such as the thumbprint) that can be placed in the configuration file. These can be safely deployed in the configuration file because, by themselves, they are not sensitive information.
The C# SDK provides a KeyVaultClient class, which is shown in Figure 1. Its constructor takes a callback method to provide the valid token from the Azure AD application. The callback method is called whenever a cryptographic action is performed using the client. A valid certificate is fetched using the custom function GetCertificateByThumbprint, using the thumbprint information from the application’s configuration file, and used to authenticate against the Azure AD application. The ADAL library caches the token received from the Azure AD application the first time and serves the token from the cache for every subsequent call until the token expires.
Figure 1 Connecting to Key Vault
var keyVaultClient = new KeyVaultClient(async (authority, resource, scope) =>
{
string azureAdApplicationId =
ConfigurationManager.AppSettings["AzureAdApplicationId"];
string certificateThumbprint =
ConfigurationManager.AppSettings["CertificateThumbprint"];
X509Certificate2 certificate =
GetCertificateByThumbprint(certificateThumbprint);
var clientAssertionCertificate =
new ClientAssertionCertificate(azureAdApplicationId, certificate);
var authenticationContext = new AuthenticationContext(authority);
var result =
await authenticationContext.AcquireTokenAsync(resource,
clientAssertionCertificate);
return result.AccessToken;
});
Configuring client applications to use keys and secrets: The client application vendors now need to update the existing application to use the key and secret identifiers shared by Contoso engineers to get the required connection string and key information from the vault. Because the key identifiers by themselves are not sensitive information, they can safely be placed in the application’s configuration file.
Where connection string or similar sensitive information that has been saved as a secret in the key vault is required, the existing code is replaced to read from the vault using the KeyVaultClient:
var connectionStringIdentifier =
ConfigurationManager.AppSettings["ConnectionStringIdentifier"];
var contosoSQLConnectionString =
await keyVaultClient.GetSecretAsync(connectionStringIdentifier);
First, this code looks up the connection string secret identifier that was stored in the configuration file and uses that with the SDK client to retrieve the secret value, which in this case is a connection string. The connection string can then be used as required to connect to the application database.
Where the application has to encrypt or decrypt PII information, the existing code is replaced with code to perform the operation using the SDK client and specifying the key identifier to be used. The following code shows how some text can be encrypted and then decrypted back to the original text:
// Within the actual application, encryption and decryption
// would happen at different parts.
var contosoPIIKeyIdentifier =
ConfigurationManager.AppSettings["ContosoPIIKeyIdentifier"];
Byte[] textToEncrypt = Encoding.Unicode.GetBytes("Consumer PII Information");
var encryptedResult =
await keyVaultClient.EncryptAsync(contosoPIIKeyIdentifier, "RSA_OAEP", textToEncrypt);
var decryptedResult =
await keyVaultClient.DecryptAsync(contosoPIIKeyIdentifier,
"RSA_OAEP", encryptedResult.Result);
var text = Encoding.Unicode.GetString(decryptedResult.Result);
Maintaining the key vault: Contoso engineers are responsible for maintaining the key vault over its lifetime, which includes changing the key value associated with a key identifier; updating the secret value associated to a secret identifier; creating new keys or secret values; updating the certificate used for Azure AD application authentication and so on.
To create or update keys and secrets in the vault, scripts like the previous ones are used. Whenever the object identifier specified for a key or a secret doesn’t exist in the vault, a new object is created. For an existing identifier, the vault automatically creates a new version and makes that the current version. The older value is retained in the vault and can be used by referring to the identifier name and the version number in the URI. Client applications can decide whether to include the version number in the identifiers based on their requirements. For different application versions, you can choose either to have different vaults or identifiers with different names in the same vault, or identifiers with the same name but with explicitly specified version numbers in the identifier URI. The Contoso security policy demands a frequent change (rolling) of keys, secret values like connection strings and certificates, for which the engineers follow the following workflow:
- When keys stored in the vault are updated to newer versions, client applications that store data encrypted with those keys need to make sure the encrypted data is migrated to use the new key.
- The Contoso client application maintains the key version used to encrypt the data and always uses that when decrypting the same data.
- When encrypting the data, the application always uses new key information. This allows the application to progressively migrate to the new key values, as and when they’re created.
For updating secret values, the process depends on the kind of value the secret represents. For values that provide information like connection strings, you first switch the secret’s value to the standby service and then roll the credential information of the primary service and update the secret with the updated credential for the primary service. For certain services, like Azure Storage, this is simpler, because there are primary and secondary keys that can be used interchangeably when either is being updated. For secrets whose value is related only to the application, the secret is updated directly to reflect any new value.
When certificates used to secure the Azure AD application need to be changed, Contoso engineers use Windows PowerShell scripts to perform the update. To update the certificate for an Azure AD application, the Azure AD module needs to be installed. You’ll find the module at bit.ly/1OdZTIS. Contoso engineers then use the script in Figure 2 to upload the new certificate for securing the Azure AD application.
Figure 2 Azure PowerShell Script for Uploading a New Certificate
$certificateFilePath = "C:\certificates\ContosoADApplicationNew.cer"
$x509Certificate2 =
New-Object System.Security.Cryptography.X509Certificates.X509Certificate2
$x509Certificate2.Import($certificateFilePath)
$rawCertData = $x509Certificate2.GetRawCertData()
$credentialValue = [System.Convert]::ToBase64String($rawCertData)
$startDate= [System.DateTime]::Now
$endDate = $startDate.AddYears(1)
$msolCredentials = get-credential
connect-msolservice -credential $msolCredentials
New-MsolServicePrincipalCredential -ServicePrincipalName $ServicePrincipal.ApplicationId
-Type Asymmetric -Value $credentialValue -StartDate $startDate -EndDate $endDate
When a certificate credential is no longer used by any application, Contoso engineers run the following script to remove the credential key from the service principal, using the Get-MsolServicePrincipalCredential cmdlet to get the identifier of the credential that needs to be removed:
$servicePrincipalCredential = Get-MsolServicePrincipalCredential
-ServicePrincipalName $ServicePrincipal.ApplicationId -ReturnKeyValues 0
Remove-MsolServicePrincipalCredential
-ServicePrincipalName $ServicePrincipal.ApplicationId
-KeyIds $servicePrincipalCredential[0].KeyId
This script simply shows the usage of the cmdlet, by just removing the first credential from the list. In a real application, you might explicitly state the identifier of the certificate credential to be removed.
Handling multiple deployments: The application currently depends only on the Azure AD application ID, the certificate thumbprint for identifying the certificate to authenticate with the Azure AD application, and various key and secret identifiers. For different deployments of the application, different key vaults can be used. For dev deployment, each application vendor can create a key vault using their Azure subscription following the procedures I’ve discussed; set up the vault and populate it with the keys and secrets the application expects; and then update the configuration file. When the Contoso team needs to deploy the application for testing, they can update the configuration file with the details from the vault that’s deployed under its subscription.
Wrapping Up
Contoso is happy to have made the switch to Azure Key Vault, which helped the company to easily handle most of the security risks to which its application had been subjected. It was able to accomplish everything it set out to do, with very few tweaks to the application code, and this also made it easy to manage various types of sensitive information. The ease with which Contoso was able to make this switch encourages the company to revisit its other LOB applications and also make them more secure. Azure Key Vault is something that most of the application can make use of right away, and makes storing and managing cryptographic keys and other sensitive information easy and secure.
Rahul Nath is a developer, consultant and blogger with experiences ranging from building rich windows client applications to large-scale applications on the Azure platform. You can follow him on Twitter at twitter.com/rahulpnath or read his blog at rahulpnath.com.
Thanks to the following Microsoft technical expert for reviewing this article: Amit Bapat
Amit Bapat Currently Program Manager for Microsoft Azure Key Vault. Before Microsoft, Amit worked at Intel in various roles including product marketing for Intel Xeon processors and in technical roles for various software projects and as a CAD engineer for designing Intel CPUs.