Introduction to Windows Azure
Exercise 2: Background Processing with Worker Roles and Queues

Collapse All Expand All Language Filter: All Language Filter: Multiple Language Filter: Visual Basic Language Filter: C#

Visual Basic Usage
C#

A worker role runs in the background to provide services or execute time related tasks like a service process.

In this exercise, you create a worker role to read work items posted to a queue by the web role front-end. To process the work item, the worker role extracts information about a guest book entry from the message and then retrieves the corresponding entity from table storage. It then fetches the associated image from blob storage and creates its thumbnail, which it also stores as a blob. Finally, to complete the processing, it updates the URL of the generated thumbnail blob in the guest book entry.

Task 1 – Creating a Worker Role to Process Images in the Background

In this task, you add a worker role project to the solution and update it so that it reads items posted by the front-end from the queue and processes them.

1. If not already open, launch Visual Studio as administrator from Start | All Programs | Microsoft Visual Studio 2010 by right clicking the Microsoft Visual Studio 2010 shortcut and choosing Run as administrator.
2. In the File menu, choose Open and then Project/Solution. In the Open Project dialog, browse to \Source\Ex2-UsingWorkerRolesAndQueues\Begin, select Begin.sln in the folder for the language of your preference (Visual C# or Visual Basic) and click Open. Alternatively, you may continue with the solution that you obtained after completing the previous exercise.
3. In Solution Explorer, right-click the Roles node in the GuestBook project, point to Add and then select New Worker Role Project.
4. In the Add New Role Project dialog, select the Worker Role category and choose the Worker Role template for the language of your choice (Visual C# or Visual Basic). Set the name of the worker role to GuestBook_WorkerRole and click Add.

   Figure 23
   Adding a worker role project to the solution (C#)

   Figure 24
   Adding a worker role project to the solution (Visual Basic)

5. In the new worker role project, add a reference to the data model project. In Solution Explorer, right-click the GuestBook_WorkerRole project and select Add Reference, switch to the Projects tab, select GuestBook_Data and then click OK.
6. Next, add a reference to the System.Drawing assembly, only this time, in the Add Reference dialog, switch to the .NET tab instead, select the System.Drawing component and then click OK.
7. Now, open the WorkerRole.cs file (for Visual C# projects) or WorkerRole.vb file (for Visual Basic projects) of the GuestBook_WorkerRole project and insert the followings namespace declarations.

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Namespaces – CS)

   C# | Copy Code
   --- | ---
       using System.Drawing;
       using System.Drawing.Drawing2D;
       using System.Drawing.Imaging;
       using System.IO;
       using GuestBook_Data;

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Namespaces – VB)

   Visual Basic | Copy Code
   --- | ---
       Imports System.Drawing
       Imports System.Drawing.Drawing2D
       Imports System.Drawing.Imaging
       Imports System.IO
       Imports GuestBook_Data

8. Add member fields to the WorkerRole class for the blob container and the queue, as shown below.

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Fields – CS)

   C# | Copy Code
   --- | ---
       public class WorkerRole : RoleEntryPoint
       {
         private CloudQueue queue;
         private CloudBlobContainer container;
         ...
       }

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Fields – VB)

   Visual Basic | Copy Code
   --- | ---
       Public Class WorkerRole
         Inherits RoleEntryPoint

         Private queue As CloudQueue
         Private container As CloudBlobContainer
         ...
       End Class

9. Insert the following code into the body of the OnStart method immediately after the line that subscribes the RoleEnvironmentChanging event and before the call to the OnStart method in the base class.

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole OnStart – CS)

   C# | Copy Code
   --- | ---
       public class WorkerRole : RoleEntryPoint
       {
         ...
         public override bool OnStart()
         {
           // Set the maximum number of concurrent connections
           ServicePointManager.DefaultConnectionLimit = 12;

           // read storage account configuration settings
           CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
           {
             configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
           });
           var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");

           // initialize blob storage
           CloudBlobClient blobStorage = storageAccount.CreateCloudBlobClient();
           container = blobStorage.GetContainerReference("guestbookpics");

           // initialize queue storage
           CloudQueueClient queueStorage = storageAccount.CreateCloudQueueClient();
           queue = queueStorage.GetQueueReference("guestthumbs");

           Trace.TraceInformation("Creating container and queue...");

           bool storageInitialized = false;
           while (!storageInitialized)
           {
             try
             {
               // create the blob container and allow public access
               container.CreateIfNotExist();
               var permissions = container.GetPermissions();
               permissions.PublicAccess = BlobContainerPublicAccessType.Container;
               container.SetPermissions(permissions);

               // create the message queue(s)
               queue.CreateIfNotExist();

               storageInitialized = true;
             }
             catch (StorageClientException e)
             {
               if (e.ErrorCode == StorageErrorCode.TransportError)
               {
                 Trace.TraceError("Storage services initialization failure. "
                   + "Check your storage account configuration settings. If running locally, "
                   + "ensure that the Development Storage service is running. Message: '{0}'", e.Message);
                 System.Threading.Thread.Sleep(5000);
               }
               else
               {
                 throw;
               }
             }
           }

           return base.OnStart();
         }
         ...
       }

   (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole OnStart – VB)

   Visual Basic | Copy Code
   --- | ---
       Public Class WorkerRole
         Inherits RoleEntryPoint
         ...
         Public Overrides Function OnStart() As Boolean

           ' Set the maximum number of concurrent connections
           ServicePointManager.DefaultConnectionLimit = 12

           ' read storage account configuration settings
           CloudStorageAccount.SetConfigurationSettingPublisher(Function(configName, configSetter) configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)))
           Dim storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString")

           ' initialize blob storage
           Dim blobStorage = storageAccount.CreateCloudBlobClient()
           container = blobStorage.GetContainerReference("guestbookpics")

           ' initialize queue storage
           Dim queueStorage = storageAccount.CreateCloudQueueClient()
           queue = queueStorage.GetQueueReference("guestthumbs")

           Trace.TraceInformation("Creating container and queue...")

           Dim storageInitialized = False
           Do While (Not storageInitialized)
             Try
               ' create the blob container and allow public access
               container.CreateIfNotExist()
               Dim permissions = container.GetPermissions()
               permissions.PublicAccess = BlobContainerPublicAccessType.Container
               container.SetPermissions(permissions)

               ' create the message queue(s)
               queue.CreateIfNotExist()

               storageInitialized = True
             Catch e As StorageClientException
               If (e.ErrorCode = StorageErrorCode.TransportError) Then
                 Trace.TraceError("Storage services initialization failure. " _
                     & "Check your storage account configuration settings. If running locally, " _
                     & "ensure that the Development Storage service is running. Message: '{0}'", e.Message)
                 System.Threading.Thread.Sleep(5000)
               Else
                 Throw
               End If
             End Try
           Loop

           Return MyBase.OnStart()

         End Function
         ...
       End Class

10. Replace the body of the Run method with the code shown below.

    (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Run – CS)

    C# | Copy Code
    --- | ---
        public class WorkerRole : RoleEntryPoint
        {
          ...
          public override void Run()
          {
            Trace.TraceInformation("Listening for queue messages...");

            while (true)
            {
              try
              {
                // retrieve a new message from the queue
                CloudQueueMessage msg = queue.GetMessage();
                if (msg != null)
                {
                  // parse message retrieved from queue
                  var messageParts = msg.AsString.Split(new char[] { ',' });
                  var imageBlobUri = messageParts[0];
                  var partitionKey = messageParts[1];
                  var rowkey = messageParts[2];
                  Trace.TraceInformation("Processing image in blob '{0}'.", imageBlobUri);

                  string thumbnailBlobUri = System.Text.RegularExpressions.Regex.Replace(imageBlobUri, "([^\\.]+)(\\.[^\\.]+)?$", "$1-thumb$2");

                  CloudBlob inputBlob = container.GetBlobReference(imageBlobUri);
                  CloudBlob outputBlob = container.GetBlobReference(thumbnailBlobUri);

                  using (BlobStream input = inputBlob.OpenRead())
                  using (BlobStream output = outputBlob.OpenWrite())
                  {
                    ProcessImage(input, output);

                    // commit the blob and set its properties
                    output.Commit();
                    outputBlob.Properties.ContentType = "image/jpeg";
                    outputBlob.SetProperties();

                    // update the entry in table storage to point to the thumbnail
                    GuestBookDataSource ds = new GuestBookDataSource();
                    ds.UpdateImageThumbnail(partitionKey, rowkey, thumbnailBlobUri);

                    // remove message from queue
                    queue.DeleteMessage(msg);

                    Trace.TraceInformation("Generated thumbnail in blob '{0}'.", thumbnailBlobUri);
                  }
                }
                else
                {
                  System.Threading.Thread.Sleep(1000);
                }
              }
              catch (StorageClientException e)
              {
                Trace.TraceError("Exception when processing queue item. Message: '{0}'", e.Message);
                System.Threading.Thread.Sleep(5000);
              }
            }
          }
          ...
        }

    (Code Snippet – Introduction to Windows Azure - Ex2 WorkerRole Run – VB)

    Visual Basic | Copy Code
    --- | ---
        Public Class WorkerRole
          Inherits RoleEntryPoint
          ...
          Public Overrides Sub Run()

            Trace.TraceInformation("Listening for queue messages...")
            Do
              Try
                ' retrieve a new message from the queue
                Dim msg As CloudQueueMessage = queue.GetMessage()
                If msg IsNot Nothing Then
                  ' parse message retrieved from queue
                  Dim messageParts = msg.AsString.Split(New Char() {","c})
                  Dim imageBlobUri = messageParts(0)
                  Dim partitionKey = messageParts(1)
                  Dim rowKey = messageParts(2)
                  Trace.TraceInformation("Processing image in blob '{0}'.", imageBlobUri)

                  Dim thumbnailBlobUri As String = System.Text.RegularExpressions.Regex.Replace(imageBlobUri, "([^\\.]+)(\\.[^\\.]+)?$", "$1-thumb$2")

                  ' download original image from blob storage
                  Dim inputBlob As CloudBlockBlob = container.GetBlockBlobReference(imageBlobUri)
                  Dim outputBlob As CloudBlockBlob = container.GetBlockBlobReference(thumbnailBlobUri)
                  Using input As BlobStream = inputBlob.OpenRead()
                    Using output As BlobStream = outputBlob.OpenWrite()
                      ProcessImage(input, output)

                      ' commit the blob and set its properties
                      output.Commit()
                      outputBlob.Properties.ContentType = "image/jpeg"
                      outputBlob.SetProperties()

                      ' update the entry in table storage to point to the thumbnail
                      Dim ds = New GuestBookDataSource()
                      ds.UpdateImageThumbnail(partitionKey, rowKey, thumbnailBlobUri)

                      ' remove message from queue
                      queue.DeleteMessage(msg)

                      Trace.TraceInformation("Generated thumbnail in blob '{0}'.", thumbnailBlobUri)
                    End Using
                  End Using
                Else
                  System.Threading.Thread.Sleep(1000)
                End If
              Catch e As StorageClientException
                Trace.TraceError("Exception when processing queue item. Message: '{0}'", e.Message)
                System.Threading.Thread.Sleep(5000)
              End Try
            Loop

          End Sub
          ...
        End Class

11. Finally, add the following method to the WorkerRole class to create thumbnails from a given image.

    (Code Snippet – Introduction to Windows Azure -Ex2 ProcessImage – CS)

    C# | Copy Code
    --- | ---
        public class WorkerRole : RoleEntryPoint
        {
          ...
          public void ProcessImage(Stream input, Stream output)
          {
            int width;
            int height;
            var originalImage = new Bitmap(input);

            if (originalImage.Width > originalImage.Height)
            {
              width = 128;
              height = 128 * originalImage.Height / originalImage.Width;
            }
            else
            {
              height = 128;
              width = 128 * originalImage.Width / originalImage.Height;
            }

            var thumbnailImage = new Bitmap(width, height);

            using (Graphics graphics = Graphics.FromImage(thumbnailImage))
            {
              graphics.InterpolationMode = InterpolationMode.HighQualityBicubic;
              graphics.SmoothingMode = SmoothingMode.AntiAlias;
              graphics.PixelOffsetMode = PixelOffsetMode.HighQuality;
              graphics.DrawImage(originalImage, 0, 0, width, height);
            }

            thumbnailImage.Save(output, ImageFormat.Jpeg);
          }
        }

    (Code Snippet – Introduction to Windows Azure - Ex2 ProcessImage – VB)

    Visual Basic | Copy Code
    --- | ---
        Public Class WorkerRole
          Inherits RoleEntryPoint
          ...
          Private Sub ProcessImage(ByVal input As Stream, ByVal output As Stream)

            Dim width As Integer
            Dim height As Integer
            Dim originalImage As New Bitmap(input)

            If originalImage.Width > originalImage.Height Then
              width = 128
              height = 128 * originalImage.Height / originalImage.Width
            Else
              height = 128
              width = 128 * originalImage.Width / originalImage.Height
            End If

            Dim thumbnailImage As New Bitmap(width, height)

            Using graphic = Graphics.FromImage(thumbnailImage)
              graphic.InterpolationMode = InterpolationMode.HighQualityBicubic
              graphic.SmoothingMode = SmoothingMode.AntiAlias
              graphic.PixelOffsetMode = PixelOffsetMode.HighQuality
              graphic.DrawImage(originalImage, 0, 0, width, height)
            End Using

            thumbnailImage.Save(output, ImageFormat.Jpeg)

          End Sub

        End Class

    Note:
    Even though the code shown above uses classes in the System.Drawing namespace for simplicity, you should be aware that the classes in this namespace were designed for use with Windows Forms. They are not supported for use within a Windows or ASP.NET service. You should conduct exhaustive testing if you intend to use these classes in your own Windows Azure applications.

12. The worker role also uses Windows Azure storage services and you need to configure your storage account settings, just as you did in the case of the web role. To create the storage account setting, in Solution Explorer, expand the Roles node of the GuestBook project, double-click GuestBook_WorkerRole to open the properties for this role and select the Settings tab. Click Add Setting, type “DataConnectionString” in the Name column, change the Type to ConnectionString, and then click the button labeled with an ellipsis. In the Storage Account Connection String dialog, choose the option labeled Use the Windows Azure storage emulator and click OK. Press CTRL + S to save your changes.

Verification

You now launch the updated application in the Windows Azure compute emulator to verify that the worker role can retrieve queued work items and generate the corresponding thumbnails.

1. Press F5 to launch the service in the local compute emulator.
2. Switch to Internet Explorer to view the application. Provided you completed the verification section of the previous exercise successfully, you will see the guest book entry that you entered, including the uploaded image displayed in its original size. If you recall, during the last task of that exercise, you updated the web role code to post a work item to a queue for each new entry submitted. These messages remain in the queue even though the web role was subsequently recycled.
3. Wait a few seconds until the worker role picks up the queued message and processes the image that you are viewing. Once that occurs, it generates a thumbnail for this image and updates the corresponding URL property for the entry in table storage. Eventually, because the page refreshes every few seconds, it will show the thumbnail image instead.

   Figure 25
   Home page showing the thumbnail generated by the worker role

4. If you are using Visual Studio 2010, in Server Explorer, expand the Blobs node in the Windows Azure Storage node, and then double-click the guestbookpics container. Notice that it now contains an additional blob for the generated thumbnail image.

   Figure 26
   Blob container showing the blob for the generated thumbnail

5. Add some more guest book entries. Notice that the images update after a few seconds once the worker role processes the thumbnails.
6. Press SHIFT + F5 to stop the debugger and shut down the deployment in the compute emulator.