Azure Durable Function gotcha

I’ve been creating a particular sequential workflow with Durable Functions and I hit a strange problem where it would get to about step 3 and it just would not proceed past it. I couldn’t see any obvious error. The issue was that the state being stored from that call was larger than the maximum size for an Azure Table Row (in this case it was a return value). This is important because that’s where Durable Functions stores the state. So be careful out there, keep your state small or save it yourself (e.g. to Blob)

Event Grid – Filters with Custom Events

In End-To-End Custom Events I posted about how to create a .net publisher and a c# Azure Function to listen to the post. However, Event Grid comes with some additional tricks, one of these is Filters. To test this I created two additional Azure Functions, HttpTriggerCSharpOranges and HttpTriggerCSharpApples. For this example they are exactly the same bar the log text, here is the Orange variant, you can guess what the Apple one looks like 😉

using System.Net;

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
    log.Info("C# HTTP trigger function processed an Orange request.");       

   // Get request body
   dynamic data = await req.Content.ReadAsAsync<object>();

   // Set name to query string or body data
   string subject = data[0].subject;
   log.Info(subject);

   return subject == null
                ? req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a subject on the query string or in the request body")
                : req.CreateResponse(HttpStatusCode.OK, "Hello " + subject);
}

The next step is to create the filtered subscriptions. The REST API pointed the way but it wasn’t that clear what I needed to define in AZ. So I entered;

az eventgrid topic event-subscription  create -h

That led to the following two filtered subscription definitions (remember to grab the Function URL by selecting the function ‘Copy Function URL’;

az eventgrid topic event-subscription create --name functionlisteneroranges --endpoint <function URL to oranges> -g gridResourceGroup --topic-name <your topic> --subject-begins-with oranges

az eventgrid topic event-subscription create --name functionlistenerapples --endpoint <function URL for apples> -g gridResourceGroup --topic-name <your topic> --subject-begins-with apples

All that’s left to do is to change the subject in the calling code to start with either apples or oranges;

new PaulsTopic
{
  Id = Guid.NewGuid().ToString(),
  EventTime = DateTime.UtcNow,
  EventType = EventType.recordInserted,
  Subject = "oranges\\paulstopics\\mytest",
  Data = new SomeData
  {
    Name = "Paul",
    Rate = 99.9
   }
}

Now when you run, the subscription pushes the correct message to the correct listener. Nice 🙂

Event Grid – End to End Custom events in .net

Event Grid is a new Azure offering that allows you to listen to a number of events from different Azure sources. One source that interested my is ‘Custom Event’. How can you get this working in .net? This is one way;

First I followed the steps in this;
Quick Start – Create and route custom events with Azure Event Grid

That provides you with the custom topic you need. In my case ‘pmtopic’. A quick note, make sure you create/have a resource group in “West US2”. I found that parts of it worked in other resource groups but other parts failed with strange errors.

Next up, create an app to ‘publish’ a custom event. I chose a Console app, but it’s just plain .net.

        public enum EventType
        {
            recordInserted
        }

        public abstract class EventTopic
        {
            
            [JsonProperty(PropertyName = "id")]
            public string Id { get; set; }

            [JsonProperty(PropertyName = "subject")]
            public string Subject { get; set; }

            [JsonProperty(PropertyName = "eventtype")]
            [JsonConverter(typeof(StringEnumConverter))]
            public EventType EventType { get; set; }

            [JsonProperty(PropertyName = "eventtime")]
            public DateTime EventTime { get; set; }

        }

        public class SomeData
        {
            public string Name { get; set; }
            public double Rate { get; set; }

        }
        public class PaulsTopic : EventTopic
        {
            [JsonProperty(PropertyName = "data")]
            public SomeData Data { get; set; }
        }

        static void Main(string[] args)
        {

            PaulsTopic[] paulsTopic = new PaulsTopic[] 
            {
                new PaulsTopic
                {
                    Id = Guid.NewGuid().ToString(),
                    EventTime = DateTime.UtcNow,
                    EventType = EventType.recordInserted,
                    Subject = "paulstopics\\mytest",
                    Data = new SomeData
                    {
                        Name = "Paul",
                        Rate = 99.9
                    }
                }
            };

            

            var messageBody = JsonConvert.SerializeObject(paulsTopic);

            PostMessage(messageBody).Wait();           
        }

        private static async Task PostMessage(string messageBody)
        {            
            using (var httpClient = new HttpClient())
            {
                StringContent content = new StringContent(messageBody);
                var request = new HttpRequestMessage()
                {
                    RequestUri = new Uri("https://<your topic endpoint here>"),
                    Method = HttpMethod.Post,
                    Content = content
                };

                request.Headers.Add("aeg-sas-key", "<your key here>");


                var response = await httpClient.SendAsync(request);
                if (!response.IsSuccessStatusCode)
                {
                    Console.WriteLine(response.ReasonPhrase);
                    var body = response.Content.ReadAsStringAsync().Result;
                }
            }
        }

The key parts appear to be that you need to set the properties in EventTopic, they don’t have to be anything specific but they need to be there. If you are following the Quick Start guide then you should be able to run the above and a new message will appear at RequestBin. I also wanted to use .net to listen to the event. For this I chose to create an Azure Function. For this I followed;
Create Serviceless Function. My function looked list this;

using System.Net;

public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
    log.Info("C# HTTP trigger function processed a request.");
      
   // Get request body
   dynamic data = await req.Content.ReadAsAsync<object>();

   // Set name to query string or body data
   string subject = data[0].subject;
   log.Info(subject);

   return subject == null
                ? req.CreateResponse(HttpStatusCode.BadRequest, "Please pass a subject on the query string or in the request body")
                : req.CreateResponse(HttpStatusCode.OK, "Hello " + subject);
}

I also switched diagnostics to ‘Info’ so I could see the output in the Function monitoring system. Note that you are sending in an array of messages so remember to get an array from the message. One missing bit of the jigsaw is you now have create the subscription from your topic to the function. To do that select the Function and ‘Get Function URL’. Insert that into the following CLI/Powershell script;

az eventgrid topic event-subscription create --name functionlistener --endpoint <function endpoint URL>  -g gridResourceGroup --topic-name <your topic name>

Now when you run your publishing app you should see an invocation in the Function monitoring system, nice 🙂

Next stop, I need to start experimenting with case sensitivity and security, but in truth it wasn’t that hard to get here and it looks like a interesting event mechanism.

Edit – see also this great post; Azure Custom Events

DeleteDocumentAsync – PartitionKey value must be supplied

I was trying to delete a document in DocumentDB / CosmoDB and I was surprised to be faced with the error message of, ‘PartitionKey value must be supplied’. There doesn’t appear to be a way of passing the partition key to DeleteDocumentAsync. The trick is to use the request options. For example, if your collection is partitioned by ProductID;

await this.documentClient.DeleteDocumentAsync(productDocument._self, new RequestOptions { PartitionKey = new Microsoft.Azure.Documents.PartitionKey(productDocument.ProductID) });

How to Backup SQL to Azure Storage

SQL Server 2016 allows you to backup a database to Azure Storage. I found the information about this to be disjoint at best. So, if you want to do an old style (non-snapshot) backup of a database this is how I achieved it;

Background

My Server is running on a Virtual Machine in on the same Azure Resource Group as the storage account, I’m not too concerned about SQL knowing the storage account keys, so I’m going to show the shortest path to getting this working

Steps

Assume we’re backing up a Database called Products.

  1. Create Storage Account – Create a ‘standard’ non premium storage account. Don’t be tempted to use premium, it will not work. Let’s call it sqlnoprembackup
  2. Create a container in the blob storage, perhaps ‘backupsql1’. Grab the full path to the container, if you’re using the portal select the container and view the properties. In this example it is; ‘https://sqlnoprembackup.blob.core.windows.net/backupsql1’
  3. Grab the account name and key for the storage account (Access Keys if you’re using the portal). Here it is sqlnoprembackup and the first long key.
  4. Create the certificate on SQL Server, let’s call it backupstorage;
  5. IF NOT EXISTS 
    (SELECT * FROM sys.credentials  
    WHERE name = ‘backupstorage’) 
    CREATE CREDENTIAL [backupstorage] WITH IDENTITY = ‘sqlnoprembackup’ 
    ,SECRET = ‘<insert secret storage key from (3) here>’; 

  6. Backup the database, use the full path to the container from (2). Note stats=5 will display the % progress at 5% internals in the messages tab;
  7. BACKUP DATABASE Products
    TO URL = ‘https://sqlnoprembackup.blob.core.windows.net/backupsql1/products.bak’  
    WITH CREDENTIAL = ‘backupstorage’  
         ,COMPRESSION 
         ,STATS = 5; 

  8. Run it. Yay.

    Restoring the database

    To do a full restore of the database;

  1. Take the database offline;
    ALTER DATABASE Products SET OFFLINE WITH NO_WAIT
  2. To restore the database;
  3. RESTORE DATABASE Products FROM URL = ‘https://sqlnoprembackup.blob.core.windows.net/backupsql1/products.bak’  
    WITH CREDENTIAL = ‘backupstorage’, REPLACE, STATS=5 

    NB, take a look at Snapshot backups, it’s a much better solution. Sometimes, as in my case, it’s not available and this hybrid approach works quite nicely.

    When you are developing a web site and want to use a custom font, via font-face, you might find that IE on Windows Phone refuses to use it. Turns out that IIS doesn’t include the mime-type by default so you need to add it. If you don’t have access to the site directly (maybe Azure web site or a shared server) you can add it in your site’s web.config webserver section;

    <system.webServer>
        <staticContent>
          <mimeMap fileExtension=".woff" mimeType="application/x-font-woff" />
        </staticContent>
    </system.webServer>