August 2019
Volume 34 Number 8
[Azure]
Affairs of State: Serverless and Stateless Code Execution with Azure Functions
By Srikantan Sankaran | August 2019
The serverless architecture pattern has emerged as a backbone of legacy modernization in the public cloud. Most platform services today are conceptualized, packaged and offered with a focus on high throughput, low cost, and superior ease of use and adoption. They generally address economics of scale and provide a platform for solution designers to realize capabilities that would otherwise require significant complexity and infrastructure.
Of course, “Pay as you Go” as a commercial model is one of the key tenets of the public cloud. “Serverless” extends this model further by making key decisions on when and how compute is provisioned, which helps manage costs during runtime. To elaborate on this philosophy and successfully adopt this pattern, it helps to consider Azure Functions as a key service as we lay out design and implementation tenets.
Azure Functions host custom code without requiring the user to specify or provision the compute on which it should run. By provisioning compute resources on demand, it supports unpredictable workloads and exhibits true “pay per use” qualities. Resources scale on the fly to match incoming loads, ensuring high availability and minimizing management overhead.
Azure Functions was envisioned as a service tasked primarily with handling rapid bursts of events at scale in a stateless manner. But the role of the technology has expanded significantly. In this article, we explore the different workload and deployment scenarios of Azure Functions and how they help enterprises implement serverless capabilities.
Design Options
The serverless capabilities of Azure Functions can be used across a variety of use-case scenarios, enabling organizations to efficiently deploy functionality, without becoming mired in overhead and infrastructure. There are important design considerations that teams should consider as they embark. Let’s explore some of those here.
Stateless vs. Stateful Execution The single most common use case for Azure Functions involves executing rapid bursts of stateless custom code at scale. These scenarios are characterized by their short duration—no more than five minutes—and code that holds no state or locks across requests. There’s no requirement to maintain a strict sequence in the execution of requests or implementing a transaction context across requests.
Note that while some of these scenarios aren’t meant for execution through Stateless Azure Functions, they can be implemented through additional design elements.
Azure Functions provides hooks for integration with a host of Azure Services through triggers and input and output binding. It can also be used to intersperse process flows implemented using Azure Logic App with custom logic.
Azure Functions can be hosted using either the Consumption tier that provides true Serverless characteristics, or via an App Service plan that uses a dedicated capacity option. In the latter case, some of the limitations in the Consumption tier, like the five-minute timeout per request, will not apply. You can learn more about the differences between these hosting options at bit.ly/2xcCXsO.
What happens when your custom code needs to orchestrate a set of stateless tasks for sequential execution, or calls for parallel execution of stateless tasks at scale with an aggregation of results in a fan-out and fan-in scenario? Or consider long-lived orchestrations that need to be monitored for state changes through external events or need human intervention during the execution lifecycle. Or consider the need for reliable execution that implements the notion of retry from the last failure point, through checkpoints and replay, rather than a regenerative execution of all tasks.
This stuff requires a lot more than a simple stateless Azure Functions service can provide. In these instances, the Azure Durable Functions extension can be used to implement robust design patterns. Durable Functions relies on an Orchestrating Function that’s responsible for handling state and ensuring reliability across numerous Activity Functions.
In short, the Activity Function is like the Stateless Azure Functions, in that it handles all the I/O operations through input and output bindings. But unlike Stateless Azure Functions, it’s triggered exclusively through an Orchestrating Function.
The code in Figure 1 shows an example of an Orchestration Function that implements a chaining of multiple Activity Functions.
Figure 1 Chaining Multiple Activity Functions
public static async Task<object> Run (DurableOrchestrationContext context)
{
try
{
var output1 = await context.CallActivityAsync<object>("ActivityFunction1");
var output2 = await context.CallActivityAsync<object>(" ActivityFunction2",
output1);
var output3 = await context.CallActivityAsync<object>(" ActivityFunction3",
output2);
return await context.CallActivityAsync<object>(" ActivityFunction4", output3);
}
catch (Exception)
{
// Error handling or compensation goes here.
}
}
Durable Entities, a feature recently added to Durable Functions, implements the Virtual Actor pattern as a Function. Stateful, Durable Entities can be managed using orchestrating Functions, and accessed from Orchestration Clients. Support for Durable Entities is in alpha stage at the time of this writing and is not yet ready for production deployments. To understand what Durable Entities are and how Durable Functions offers them, refer to bit.ly/2RAA12B.
Workloads and Binding We’ve already discussed that serverless architectures like Azure Functions are inherently suited for unpredictable workloads, where periods of inactivity are punctuated by sudden bursts of demand. For these scenarios, the Consumption tier or the Premium tier would likely be the best choice.
For continuous and predictable loads, however, it might be more cost effective to deploy solutions to pre-provisioned, dedicated resources, using the App Service tier for Azure Functions.
Another thing to consider is the balance between connections baked into code and declarative binding. The latter are design-time configurations for Functions that eliminate the need for the Function code to manage connections to triggers and input and output binding sources. When the function app scales out to multiple instances under load, connections implemented via declarative binding are reused across all instances.
Connections implemented in code, by contrast, require that the developer manage the data source connections across requests. Creating and disposing connections in the Function code for every request limits scalability and performance under load. However, taking this approach is unavoidable, for instance, when the data source isn’t supported through design-time bindings. In such a case, care should be taken to reuse client-side connections across requests, using static or singleton client connections.
Guidance on connection management in Functions is available at bit.ly/2ZSf5a3.
Control in Execution
Serverless architectures excel at providing high availability, auto scaling and resiliency, with minimal user intervention. In this regard, they’re superior even to Platform-as-a-Service (PaaS) services. However, there are certain configuration options available in serverless environments that can be used to tune the performance characteristics of Azure Functions. Let’s walk through those now.
Batching for Performance: The throughput and performance of a function improves when it’s configured to receive incoming messages in batches, rather than employing per-message execution. However, this execution is dependent on the Azure Service configured in the binding and whether or not the Azure Function trigger for that service supports batching of messages. For example, Azure Event Hubs supports batching of messages consumed by client applications. The batch size is specified in the host.json file of the function app.
Concurrency of Requests: You can determine how many requests are handled by each instance of a function app by specifying the concurrency of the request triggers. You can tweak this number to suit the nature of the workloads being processed and then let the function app scale as appropriate. This configuration is specified in the host.json file of the function app.
Scaling Out: Azure Functions automatically scales the infrastructure to handle incoming loads with no user intervention. How steeply the scale out happens depends on the number of events that need to be processed. However, there are certain limits within which a single function app can scale. In the consumption tier, each function app instance can handle multiple concurrent requests, and each function app itself can scale out to at most 200 instances (see bit.ly/2JlU7tV). In addition, the consumption tier doesn’t provide the option to choose the capacity of the compute resources for this workload.
How do you scale out beyond these maximum limits? One option would be to clone the function app and have the client applications distribute the load between them. An alternative would be to leverage the premium tier of Azure Functions, which lets you choose from a variety of compute sizes suitable for the workload at hand. Note that the premium tier for function apps is available in public preview at this time and cannot be used in production workloads.
Cold Start Ups: After a period of inactivity when all the instances have timed out, a sudden load applied to a function app can produce a delay as instances must start to actively handle requests again. This delay might not be acceptable when handling certain mission-critical scenarios.
Cost-Performance Trade-offs: The premium tier in function apps provides options to specify the number of warm instances that are always available as a standby, to prevent cold start-up delays. Apart from exhibiting the same serverless capabilities as the consumption tier, it provides additional benefits, like the option to choose from a variety of compute sizes on which to run the instances, the ability to access resources inside a virtual network and exemption from a default timeout duration as in the consumption tier. All these features come at a cost, but the benefits they offer can more than compensate for the trade-offs in mission-critical use cases.
Note that Azure Functions is available both on Linux and Windows compute, but not all the features are available in both offerings. Azure Functions on Linux is currently available in preview.
Portability
Azure Functions comprises a runtime and a scaling component. The former is available as a Docker container, which means Functions can be deployed on-premises, at the edge or on Azure Cloud. The scaling capabilities are achieved by deploying a Functions runtime inside a Kubernetes cluster in conjunction with the open source component, Kubernetes-based Event-driven autoscaling (KEDA).
We’ve already covered how Azure Functions can run natively on Azure, but Functions can also be deployed to a Kubernetes cluster. On Azure, KEDA can be deployed to a Kubernetes cluster created using Red Hat OpenShift or Azure Kubernetes Service (AKS).
On the edge or on-premises, KEDA can be installed in a Kubernetes cluster using the Core Tools for Functions. Functions packaged as containers would run in the Kubernetes pods.
Scaling of the containers running Functions happens based on the number of messages in the queues configured as triggers for the function app. Refer to guidance at bit.ly/2J9gYIO for more on implementing KEDA for Functions and Kubernetes.
Development Best Practices
Covered in this section are some valuable best practices for the development of Azure Functions. Let’s start with exception handling from triggers and bindings. Azure Functions provides declarative bindings for triggers, input and output data sources. When there’s an error in executing triggers or invoking data sources, these must be caught explicitly in code. The exception handler should implement the semantics of retries with exponential back-off. Services like Azure Service Bus Queues and Blob Stores implicitly support retries in the triggers, but for other services, these must be explicitly implemented.
To capture runtime exceptions when using other services like Event Hubs, the output binding must use the IAsyncCollector type, instead of an out parameter. Calling the AddAsync method of the collector inside an exception handling block ensures that the runtime exceptions are caught.
In Figure 2, a transient exception with code 50002 is thrown by Event Hubs, which indicates that the requests are being throttled. In the exception handler, retry logic that uses an exponential back-off policy should be implemented.
.png "Handling Exceptions in Output Binding")
Figure 2 Handling Exceptions in Output Binding
The documentation page at bit.ly/31UDLRl covers how an exponential back-off could be implemented in a durable function app.
Figure 3 shows the metrics from the Event Hubs instance after it’s subjected to load, to simulate request throttling.
.png "Monitoring Event Hub for Request Throttling")
Figure 3 Monitoring Event Hub for Request Throttling
Authentication and Identities
Azure Functions supports different forms of authentication of client applications. This could be in the form of embedding an authentication key in the HTTP header of the request to the function app, or using an oAuth token when users access the Function through a Web application that implements Azure AD sign-in (bit.ly/2XzrulR). In some cases when the function app is exposed to the external users through Azure API Management, authorization of the user requests could be done at the gateway using the subscription keys embedded in the request.
When the custom code in Azure Functions needs to access other services in Azure, it’s a good practice to bootstrap the credentials with it using system-based or user-based managed identities (bit.ly/2IPntl0). Currently, not all services in Azure support access using managed identities. The list of supported services can be found at bit.ly/2ISVUYp.
In the following code snippet, taken from a Visual Studio 2017 Azure Functions project, I show how managed identity can be used to connect to and access Azure SQL Database. For security, you no longer need to store SQL authentication credentials inside the connection string and then store the connection string itself in Azure Key Vault.
string connString =
"Server = tcp:onedbserver.database.windows.net,1433; Database = onedb;";
using (SqlConnection con = new SqlConnection(connString))
{
con.AccessToken = await (
new AzureServiceTokenProvider()).GetAccessTokenAsync(
"https://database.windows.net/");
con.Open();
try
{
using (SqlCommand command =
new SqlCommand("SELECT * FROM SalesLT.Product", con))
{
SqlDataReader reader = command.ExecuteReader();
...
Next, publish the function app to Azure, enable Managed Identity for it, and provide database access to it. Refer to the steps documented at bit.ly/2JaGOfr that explain the process of service principal identity creation and assignment to an Azure SQL Database.
Azure App Service and Azure Functions provide convenience options to simplify integration with Azure Key Vault using managed identities to access secrets. Check out the article at bit.ly/2X1f7ed for detailed guidance in implementing this feature.
Wrapping up
Azure Functions provides enterprises with a serverless way to develop, deploy and run custom code that can run anywhere, in Azure, on-premises or on the edge. The advent of additional features to the platform, in the form of durable functions and entities, unlocks the opportunities to extend serverless to a host of new requirements and design patterns. The extensive integration of Azure Functions with other Azure services and development tools ensures that developers are most productive when building applications, regardless of the programming language and platform, using an IDE of their choice.
Sandeep Alur is director of Partner Engagements (ISV) at Microsoft. His experience ranges from the original dotcom days to the distributed/cloud computing era to the next generation of intelligent computing. Reach him at Sandeep.Alur@microsoft.com.
Srikantan Sankaran is a principal technical evangelist in the One Commercial Partner team in India, based out of Bangalore. He works with numerous ISVs in India and helps them architect and deploy their solutions on Microsoft Azure. Reach him at Srikantan.Sankaran@microsoft.com.
Thanks to the following Microsoft technical expert for reviewing this article: Hemant Kathuria