Exercise 2: Background Processing with Worker Roles and Queues

A worker role runs in the background to provide services or execute time related tasks like a service process.

In this exercise, you create a worker role to read work items posted to a queue by the web role front-end. To process the work item, the worker role extracts information about a guest book entry from the message and then retrieves the corresponding entity from table storage. It then fetches the associated image from blob storage and creates its thumbnail, which it also stores as a blob. Finally, to complete the processing, it updates the URL of the generated thumbnail blob in the guest book entry.

Task 1 – Creating a Worker Role to Process Images in the Background

In this task, you add a worker role project to the solution and update it so that it reads items posted by the front-end from the queue and processes them.

1. If not already open, launch Visual Studio as administrator from Start | All Programs | Microsoft Visual Studio 2010 by right clicking the Microsoft Visual Studio 2010 shortcut and choosing Run as administrator.
2. In the File menu, choose Open and then Project/Solution. In the Open Project dialog, browse to Ex2-UsingWorkerRolesAndQueues\Begin in the Source folder of the lab, select Begin.sln in the folder for the language of your preference (Visual C# or Visual Basic) and click Open. Alternatively, you may continue with the solution that you obtained after completing the previous exercise.
3. In Solution Explorer, right-click the Roles node in the GuestBook project, point to Add and then select New Worker Role Project.

4. In the Add New Role Project dialog, select the Worker Role category and choose the Worker Role template for the language of your choice (Visual C# or Visual Basic). Set the name of the worker role to GuestBook_WorkerRole and click Add.

   Figure 23

   Adding a worker role project to the solution (C#)

   Figure 24

   Adding a worker role project to the solution (Visual Basic)

5. In the new worker role project, add a reference to the data model project. In Solution Explorer, right-click the GuestBook_WorkerRole project and select Add Reference, switch to the Projects tab, select GuestBook_Data and then click OK.
6. Next, add a reference to the System.Drawing assembly, only this time, in the Add Reference dialog, switch to the .NET tab instead, select the System.Drawing component and then click OK.

7. Now, open the WorkerRole.cs file (for Visual C# projects) or WorkerRole.vb file (for Visual Basic projects) of the GuestBook_WorkerRole project and insert the followings namespace declarations.

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Namespaces – CS)

   C#

   using System.Drawing; using System.Drawing.Drawing2D; using System.Drawing.Imaging; using System.IO; using GuestBook_Data;

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Namespaces – VB)

   Visual Basic

   Imports System.Drawing Imports System.Drawing.Drawing2D Imports System.Drawing.Imaging Imports System.IO Imports GuestBook_Data

8. Add member fields to the WorkerRole class for the blob container and the queue, as shown below.

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Fields – CS)

   C#

   public class WorkerRole : RoleEntryPoint

   FakePre-6fd51e74522046bf94d06ea172be7d76-f53846de60394b37962dedaf4443f5e8 private CloudQueue queue; private CloudBlobContainer container;FakePre-343dc5e298c0494d98ab530a6c81e569-18ffab960a2744e086714dde0e9b5835FakePre-70daa7420d2f49de8ca8dd02a4decc89-abdd4b1874ab4ad8badd5345fdc93a15

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Fields – VB)

   Visual Basic

   Public Class WorkerRole

   FakePre-febc895dc0d24ee89e4dfe562e51350b-084ea531291e408fb997194898da49d8FakePre-5c002f43f4674310b46ad4a8a5eaa6eb-26c09a3b2b2c482a9a5e3da4dddd5889 Private queue As CloudQueue Private container As CloudBlobContainerFakePre-1d7750841c7740a7bc5a6a94a83bc97e-c40de039422147b0a525ee4c7d3dc20eFakePre-6fbae59177634600870f3be59310e9b0-bda6bcb8daea4358852ded38f68bb9c3

9. Insert the following code into the body of the OnStart method immediately after the line that subscribes the RoleEnvironmentChanging event and before the call to the OnStart method in the base class.

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole OnStart – CS)

   C#

   public class WorkerRole : RoleEntryPoint

   FakePre-40827dcd540446a2a7108db633181549-b6e96b3bb5fb49cca4e700ebf8521721FakePre-77283dae8f05499db49dd05c2c74d069-2fd9db9cf82f4c34a6e026f62d369b16FakePre-9edf1baf30b147c4aadebb6a93e46111-7ca5fc421db04aa987a9cccc66332d16FakePre-363492d463d145f4856b59eaad4db29e-86deea29cd32407a9e25ded0ffb7aa92FakePre-137b71678bc5415ca852a4c01ec99007-2fe8d0580a9b481586d3f8fa0d197ef4FakePre-b1375ce5df274b55be16b22caefd86e0-3d0d3cb9fe4542e98917e487dd8a71ceFakePre-3279effa2e8247eaa958fa9b799463df-03627afc63464beb99dd378ed8fdbf7c // read storage account configuration settings CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) => { configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)); }); var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString"); // initialize blob storage CloudBlobClient blobStorage = storageAccount.CreateCloudBlobClient(); container = blobStorage.GetContainerReference("guestbookpics"); // initialize queue storage CloudQueueClient queueStorage = storageAccount.CreateCloudQueueClient(); queue = queueStorage.GetQueueReference("guestthumbs"); Trace.TraceInformation("Creating container and queue..."); bool storageInitialized = false; while (!storageInitialized) { try { // create the blob container and allow public access container.CreateIfNotExist(); var permissions = container.GetPermissions(); permissions.PublicAccess = BlobContainerPublicAccessType.Container; container.SetPermissions(permissions); // create the message queue(s) queue.CreateIfNotExist(); storageInitialized = true; } catch (StorageClientException e) { if (e.ErrorCode == StorageErrorCode.TransportError) { Trace.TraceError("Storage services initialization failure. " + "Check your storage account configuration settings. If running locally, " + "ensure that the Development Storage service is running. Message: '{0}'", e.Message); System.Threading.Thread.Sleep(5000); } else { throw; } } }FakePre-e5c86b6e401f4a7ca420ae189eebf589-e08c8e8a5aee40eb9c8f5285512690e9FakePre-ace3bdfd0bef4b8781710ac99a00b20b-2603aee658f149cd9848bbc64c711e8bFakePre-e06a013ee794430cae50236a22005b8f-acf7166f93fe43c28289bfab324e1a10FakePre-1c6e51f0c4c14d5388a501b112c429ce-bfe0440fc3ec498d9af5afb3774d68e2FakePre-5f15f1bab90a4c21a15bce8e3c8a2ab1-df15a0e4f1b742e4bc508e56f7432e3cFakePre-8ea1507a1ce74fafa780ae6d28547024-e18d6eea6cb5483a97c6ff5e5b8d05dfFakePre-200da1c307054f04bd4e6fa69fb94eb1-863c0561158b4f859849ccd11845ed56FakePre-074e2a9140a946c1862ab44308569d2f-54fc5b993a03469db4432443e5388548FakePre-8223e4c525fc44dc9e7ddf04e03489cb-2d68d9c57a9a4254811055c49d4392b2FakePre-60cea9cf8a28434e9b9a771265a30976-31d4c081e8434aa0a2eeaa271537c974FakePre-9ffaed82fb69412aacf9befb888a2c87-c2489852c6e34045b7c0bf18ee9c6022

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole OnStart – VB)

   Visual Basic

   Public Class WorkerRole

   FakePre-0e6cdf1024ba4986b6bc65030527a2ad-2198ed5ba3264ad3b98bbbe146b4b345FakePre-c4866de3707642959d65cd124d7120c8-d9e2b1e107964ce7851f1c662b94271bFakePre-ed87a8dafb2046bfa89849527c80595b-79607ba376ed4131a0aabb3b68eeab76FakePre-0cc724714551464286ce289d30f16a9d-2556a51ff054432cacfeb8682b5c9e34FakePre-36eb3bc6be5e43a7bbaad56704ee7d0a-025af0d7f24d43bd82e54b7f578a12d5FakePre-226cd880f8a144a5bf46f102124895f9-a7ac406f8a354966ac8805fc3c12359eFakePre-deeac3eb7039407aadab9eeb0c07b44f-5b97fbfb23fa463abc0c5f7a3edabcb7 ' read storage account configuration settings CloudStorageAccount.SetConfigurationSettingPublisher(Function(configName, configSetter) configSetter(RoleEnvironment.GetConfigurationSettingValue(configName))) Dim storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString") ' initialize blob storage Dim blobStorage = storageAccount.CreateCloudBlobClient() container = blobStorage.GetContainerReference("guestbookpics") ' initialize queue storage Dim queueStorage = storageAccount.CreateCloudQueueClient() queue = queueStorage.GetQueueReference("guestthumbs") Trace.TraceInformation("Creating container and queue...") Dim storageInitialized = False Do While (Not storageInitialized) Try ' create the blob container and allow public access container.CreateIfNotExist() Dim permissions = container.GetPermissions() permissions.PublicAccess = BlobContainerPublicAccessType.Container container.SetPermissions(permissions) ' create the message queue(s) queue.CreateIfNotExist() storageInitialized = True Catch e As StorageClientException If (e.ErrorCode = StorageErrorCode.TransportError) Then Trace.TraceError("Storage services initialization failure. " _ & "Check your storage account configuration settings. If running locally, " _ & "ensure that the Development Storage service is running. Message: '{0}'", e.Message) System.Threading.Thread.Sleep(5000) Else Throw End If End Try LoopFakePre-e2d4c00009ad45e9af61acf49d107aa3-d8e578610b6f43e0b6f3a88e5083fa88FakePre-524c66e532ae4fb583d2d6936fbf63ac-8f79f84a0d3a4831a0329b4ab8ad5df8FakePre-6b10280cf2394ed0990dbfb5c9f1b0e7-3522c401f47c41afa1f253cb2ab17782FakePre-cf6b2eb509a04e259dea8877827a5afd-cd6ef7422f674ff4bf2d2174f8e2d038FakePre-2ab5941df55740099d6cd205f96bc0d0-40a9dbe6f9034ba692e5bdc8d5b355e7FakePre-db9b909e2a6b448e8c858418f8764070-29ceb5aa136f42cdb0118f478bdb98f4FakePre-75b1d6bd633a4083b9fcea9d61a91ac3-a6afc6081543468d86a2c246ea7c0c71FakePre-988fa3347c1a414e97b6d0a9f61ab3c9-0a72275145ee46e8a972f6ce8cdbc88bFakePre-4407042a74dc427bbddaca3d4dbac386-14af7e5bf77d4def90bbde66bdda5e38FakePre-a8f28e2d33774e7dbc4d1651bd7772bf-ed9f4a08412941ab89963253e01f1382FakePre-d76721200052400485176d2e09b2079c-6f30a010542943de94f3014d52ead5f7FakePre-32c3e453d08143e89dde38c7a8041638-acdaed87f4df48dd89e0de996d920dcc

10. Replace the body of the Run method with the code shown below.

    (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Run – CS)

    C#

    public class WorkerRole : RoleEntryPoint

    FakePre-a0013091d11040d880a8b2e592e7b28f-ca8ee08248204556ba9d6cd8437056ecFakePre-4ac6310af323476e84605dcf96055cf0-2c2c31047f814081bb6f75cfd7613c79FakePre-657dcf897808478b9b0dbca8d3dab868-3d7f8fe61e714a9a90553e799941f922FakePre-677e475b7311435d8e3b75cb178b989f-ca511f4e234449b1a2c3835695351655 Trace.TraceInformation("Listening for queue messages..."); while (true) { try { // retrieve a new message from the queue CloudQueueMessage msg = queue.GetMessage(); if (msg != null) { // parse message retrieved from queue var messageParts = msg.AsString.Split(new char[] { ',' }); var imageBlobUri = messageParts[0]; var partitionKey = messageParts[1]; var rowkey = messageParts[2]; Trace.TraceInformation("Processing image in blob '{0}'.", imageBlobUri); string thumbnailBlobUri = System.Text.RegularExpressions.Regex.Replace(imageBlobUri, "([^\\.]+)(\\.[^\\.]+)?$", "$1-thumb$2"); CloudBlob inputBlob = container.GetBlobReference(imageBlobUri); CloudBlob outputBlob = container.GetBlobReference(thumbnailBlobUri); using (BlobStream input = inputBlob.OpenRead()) using (BlobStream output = outputBlob.OpenWrite()) { ProcessImage(input, output); // commit the blob and set its properties output.Commit(); outputBlob.Properties.ContentType = "image/jpeg"; outputBlob.SetProperties(); // update the entry in table storage to point to the thumbnail GuestBookDataSource ds = new GuestBookDataSource(); ds.UpdateImageThumbnail(partitionKey, rowkey, thumbnailBlobUri); // remove message from queue queue.DeleteMessage(msg); Trace.TraceInformation("Generated thumbnail in blob '{0}'.", thumbnailBlobUri); } } else { System.Threading.Thread.Sleep(1000); } } catch (StorageClientException e) { Trace.TraceError("Exception when processing queue item. Message: '{0}'", e.Message); System.Threading.Thread.Sleep(5000); } }FakePre-b8d7e548828e49ef998c0957cad818d6-1a3354fb4cf54fdcaf4c1e0e336f0533FakePre-8015a250cbc141769dbfdf2e2564ddd7-468a76d331ec4b8aa15241401ca7f00dFakePre-dd3224675a8449bea05b7d2f96a02811-452bde521bc34f19b1f81732804d3cd6FakePre-7319a843393e41e08e0cc5b3141702c8-a62977a55e144e3cbfaabf6781dd8ef1FakePre-744c18d5a5eb47f2972e16ba3495b073-ec48eb6beab34f189c97e410554b6d11FakePre-1d7c29d4908e48b9b3d1fc0191c9c969-a1d69feb95cc40d0a758b89324c46482FakePre-5be960eb62e945bd980d75b42559d8b5-28f8ad4ea6184bc98ca3d0760b689bd5FakePre-1c2bb737d7f5461a9f7ef96a865197e6-71777b334edb4b0ea7e6e8172101c5ddFakePre-547687abbd144807afb63b9e28f23cad-c4169f443eea4620a6e4f86837889ea8FakePre-6838aac6b0c8463eba922f014197caf2-079c2449431a4a1ca605d8041d60f4e4FakePre-b7ef8ded84914495bfb39a7fede22fd4-1062f0f95aaf4321b6563de2689240c9

    (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Run – VB)

    Visual Basic

    Public Class WorkerRole

    FakePre-fb9a0517d52f427992c922f19253a607-e106608833d84564a2009fa4129abbbfFakePre-e169fa49d3be47498b282d16df0cc002-d089bed9a47a425894da198a2756b19bFakePre-62093de5b4cd486ea259825aefdf6d2d-9b37d514e6ad49ee9fda4c5308420153FakePre-e4c116c4b1184d5e9ab166536529295d-ba70819184f4468db42cdb5febcf0a99 Trace.TraceInformation("Listening for queue messages...") Do Try ' retrieve a new message from the queue Dim msg As CloudQueueMessage = queue.GetMessage() If msg IsNot Nothing Then ' parse message retrieved from queue Dim messageParts = msg.AsString.Split(New Char() {","c}) Dim imageBlobUri = messageParts(0) Dim partitionKey = messageParts(1) Dim rowKey = messageParts(2) Trace.TraceInformation("Processing image in blob '{0}'.", imageBlobUri) Dim thumbnailBlobUri As String = System.Text.RegularExpressions.Regex.Replace(imageBlobUri, "([^\\.]+)(\\.[^\\.]+)?$", "$1-thumb$2") ' download original image from blob storage Dim inputBlob As CloudBlockBlob = container.GetBlockBlobReference(imageBlobUri) Dim outputBlob As CloudBlockBlob = container.GetBlockBlobReference(thumbnailBlobUri) Using input As BlobStream = inputBlob.OpenRead() Using output As BlobStream = outputBlob.OpenWrite() ProcessImage(input, output) ' commit the blob and set its properties output.Commit() outputBlob.Properties.ContentType = "image/jpeg" outputBlob.SetProperties() ' update the entry in table storage to point to the thumbnail Dim ds = New GuestBookDataSource() ds.UpdateImageThumbnail(partitionKey, rowKey, thumbnailBlobUri) ' remove message from queue queue.DeleteMessage(msg) Trace.TraceInformation("Generated thumbnail in blob '{0}'.", thumbnailBlobUri) End Using End Using Else System.Threading.Thread.Sleep(1000) End If Catch e As StorageClientException Trace.TraceError("Exception when processing queue item. Message: '{0}'", e.Message) System.Threading.Thread.Sleep(5000) End Try LoopFakePre-e33f1a2b66d346709412a03998d0e371-3b93edee1365453c83847ba73d76b600FakePre-aed7757deed547a3a6eb597cbd726080-0489e5cc869f49a1a2f87b639e9ca1cfFakePre-9da1f49514684db480740b25e895e860-815b287383c045a788fd9594a90d1365FakePre-ab3442da85ac4bdfa1a70d5bc983113b-4ec3b682cfaf4117b02a77ee3af913c7FakePre-4ef980a5815d4b2381d1882a9956d69d-4a919a13ecf341c3a43eec0d336713b2FakePre-49a1c98a7c504f1fb49a050362491ba3-dbd35754be7340ceba1d6cbeb6f09b87FakePre-70e908c27e2c4a88824c69408132f6ca-d07c9768320d46dc965a661d05585322FakePre-2a0bea46c10e43e397d7d472c599dd17-4aa4934e209c40cc9b20f7311340c5c4FakePre-916064e1017c49bb8493ddb0b31c0265-35188f3ca0c74476ac92decd28c064d0FakePre-97340d18ba4c41f898b44bb691d2f665-c163c09fa5934d5e9fa7eb4d24b28741

11. Finally, add the following method to the WorkerRole class to create thumbnails from a given image.

    (Code Snippet – Introduction to Windows Azure -Ex2 ProcessImage – CS)

    C#

    public class WorkerRole : RoleEntryPoint

    FakePre-0f7479b7b39244ea94d31a8e5237092f-0b179ed111544cbeaf77b2f4c1764246FakePre-e7247d8e770145dab5446411cfd15b5b-a5ba172101594b85a9e25a4e6ab20407 public void ProcessImage(Stream input, Stream output) { int width; int height; var originalImage = new Bitmap(input); if (originalImage.Width > originalImage.Height) { width = 128; height = 128 * originalImage.Height / originalImage.Width; } else { height = 128; width = 128 * originalImage.Width / originalImage.Height; } var thumbnailImage = new Bitmap(width, height); using (Graphics graphics = Graphics.FromImage(thumbnailImage)) { graphics.InterpolationMode = InterpolationMode.HighQualityBicubic; graphics.SmoothingMode = SmoothingMode.AntiAlias; graphics.PixelOffsetMode = PixelOffsetMode.HighQuality; graphics.DrawImage(originalImage, 0, 0, width, height); } thumbnailImage.Save(output, ImageFormat.Jpeg); }FakePre-2df0b636ef2e489fa28697c5cfec7c2b-80059afd60dc43b49a5133817b296c0aFakePre-dc0d0e1a970640d888f111974938127f-049a6f922cbc4fe6892251822d244b90FakePre-75759a5a83024c7c9c41094306fffb6f-508f62196ab348028717033e580c2f3cFakePre-bf8d81336ed04cdd9358569ff9dc28ae-dfd1fa0729c147e9b69b8fbd1d6b29efFakePre-68e65193bb184a689a1faccdbb56ed95-350b41a781784c798d2c42e5dda86ab8

    (Code Snippet – Introduction to Windows Azure - Ex2 ProcessImage – VB)

    Visual Basic

    Public Class WorkerRole

    FakePre-e5af9f6f78eb468c90723e3b2bb07ee1-3d5ef1f692c94585bb0a5f6dceb04905FakePre-2d1ea63ec7f249b19b50292782520a29-c6e1ece5a5324ba8a12e94e99c434b58 Private Sub ProcessImage(ByVal input As Stream, ByVal output As Stream) Dim width As Integer Dim height As Integer Dim originalImage As New Bitmap(input) If originalImage.Width > originalImage.Height Then width = 128 height = 128 * originalImage.Height / originalImage.Width Else height = 128 width = 128 * originalImage.Width / originalImage.Height End If Dim thumbnailImage As New Bitmap(width, height) Using graphic = Graphics.FromImage(thumbnailImage) graphic.InterpolationMode = InterpolationMode.HighQualityBicubic graphic.SmoothingMode = SmoothingMode.AntiAlias graphic.PixelOffsetMode = PixelOffsetMode.HighQuality graphic.DrawImage(originalImage, 0, 0, width, height) End Using thumbnailImage.Save(output, ImageFormat.Jpeg) End SubFakePre-45742732e40a458fbe291f99b23aa714-2c7b928a601a4778b284f7400e312518FakePre-f13e3e5a84cc48389db69e7ece0c3cf4-9eb2d734f2734b399477f9324e5c6408FakePre-52c00c3b0b1448fd8b2193e654d52287-f6e4680dc4e1460db45bc2abfb72e282FakePre-db1b2246293b4f299a9483c80ea63e3b-614131234b854318b3f99023d09a2c61FakePre-9a2b550f140c4b6c952132a9d0af93cb-dc213cbaea2c483290f0ec0eceb0a9afFakePre-27092a75211f4f7397b9045649cdd7d9-5e5adcc464234bef985670b431093912FakePre-339e4b3786674b0085d9d91c614c4af3-221d15b1609d40b8a90e273b1c147885FakePre-fac0f9ae30d642b29afb483042f583c3-e0d4790ad2b441818749d11d026ed47a
    Note:

    Even though the code shown above uses classes in the System.Drawing namespace for simplicity, you should be aware that the classes in this namespace were designed for use with Windows Forms. They are not supported for use within a Windows or ASP.NET service. You should conduct exhaustive testing if you intend to use these classes in your own Windows Azure applications.

12. The worker role also uses Windows Azure storage services and you need to configure your storage account settings, just as you did in the case of the web role. To create the storage account setting, in Solution Explorer, expand the Roles node of the GuestBook project, double-click GuestBook_WorkerRole to open the properties for this role and select the Settings tab. Click Add Setting, type “DataConnectionString” in the Name column, change the Type to ConnectionString, and then click the button labeled with an ellipsis. In the Storage Account Connection String dialog, choose the option labeled Use the Windows Azure storage emulator and click OK. Press CTRL + S to save your changes.

Verification

You now launch the updated application in the Windows Azure compute emulator to verify that the worker role can retrieve queued work items and generate the corresponding thumbnails.

1. Press F5 to launch the service in the local compute emulator.
2. Switch to Internet Explorer to view the application. Provided you completed the verification section of the previous exercise successfully, you will see the guest book entry that you entered, including the uploaded image displayed in its original size. If you recall, during the last task of that exercise, you updated the web role code to post a work item to a queue for each new entry submitted. These messages remain in the queue even though the web role was subsequently recycled.

3. Wait a few seconds until the worker role picks up the queued message and processes the image that you are viewing. Once that occurs, it generates a thumbnail for this image and updates the corresponding URL property for the entry in table storage. Eventually, because the page refreshes every few seconds, it will show the thumbnail image instead.

   Figure 25

   Home page showing the thumbnail generated by the worker role

4. If you are using Visual Studio 2010, in Server Explorer, expand the Blobs node in the Windows Azure Storage node, and then double-click the guestbookpics container. Notice that it now contains an additional blob for the generated thumbnail image.

   Figure 26

   Blob container showing the blob for the generated thumbnail

5. Add some more guest book entries. Notice that the images update after a few seconds once the worker role processes the thumbnails.
6. Press SHIFT + F5 to stop the debugger and shut down the deployment in the compute emulator.