Job 1 when upgrading to Bot Framework v4

Just a quick one, when you are creating a brand new Bot Framework v4 project, the first thing you should do is go to the properties and change the Application->Target Framework to Asp.net core 2.1 and then go to Manage NuGet and update all the asp and bot libraries.

Advertisements

LUIS now understands people’s names

This one managed to slip past me, but recently Microsoft Language Understand Intelligent Service (LUIS) has been updated with a pre-built entity to match people’s names. Seems like an obvious requirement but is pretty tricky to implement. To match people’s names;

  1. Open your KB in the LUIS UI of your choice
  2. Go to the Entity menu
  3. select Add Prebuilt entity
  4. choose personName

Now you can create an Intent such as “Call” with “Call Jane” and “Jane” will be matched to personName. So far (and testing is difficult) I’ve thrown all my usual problematic name variants (Western, Nordic, Chinese, Indian) and it’s managed them all, even “Call peter phoxterd-hemmington-smythe” 🙂

Handling flow from Prompt Validators

Creating a basic flow with Botframework V4 is pretty straightforward. You define your steps and any other dialogs or prompts that you may wish to use. For example, consider this basic name capturing definition;

Add(Inputs.Text, new Microsoft.Bot.Builder.Dialogs.TextPrompt(TextValidator));
Add(Name, new WaterfallStep[]
{
    // Each step takes in a dialog context, arguments, and the next delegate.
    async (dc, args, next) =>
    {
        // Prompt for the user's first name.
        await dc.Prompt(Inputs.Text, "Hey, what's your first name?");
    },
    async (dc, args, next) =>
    {
        dc.ActiveDialog.State["1stName"] = args["Text"].ToString();
     
        // Prompt for the user's second name.
        await dc.Prompt(Inputs.Text, "what's your second name?");
    },
...
private Task TextValidator(ITurnContext context, TextResult toValidate)
{
    if (toValidate.Text == ".")
    {
        toValidate.Status = null;

    }
    
    return Task.CompletedTask;
}

The above code will reject the second name if the user types in a full-stop. But what will happen next needs to be considered. The current code will correctly reject the full-stop but it will restart the dialog and ask the user for their first name again. There are a couple of ways to handle this, and they can be implemented in unison.

Use a Retry Message

When a prompt has a Retry-Message associated with it then any failure from the Validator will keep the user on the same waterfall step.

...
 async (dc, args, next) =>
    {     
        dc.ActiveDialog.State["1stName"] = args["Text"].ToString();
        // Prompt for the user's second name.
        await dc.Prompt(Inputs.Text, "what's your second name?", 
            new PromptOptions
            {
                RetryPromptString = "what's your really your second name?"
            });
    },
...

The above code is probably all your need, but you may also want to equip your waterfall steps to be more…re-entrant.

Replay dialog

This is particularly useful if you provide a way to edit the user’s choices. The idea is that each step examines its arguments or state and if already fulfilled simply passes the flow onto the next step. In the final step you decide if everything has completed or if the flow should start again.
Example step;

async (dc, args, next) =>
    {
        if (args.ContainsKey("1stStep")
        {
          await next(args);
        }
        else
        {
          // Prompt for the user's first name.
          await dc.Prompt(Inputs.Text, "Hey, what's your first name?");
        }
    },
...

Final step;

async (dc, args, next) =>
    {
        if (AllConditionsMet(args))
        {
          // finish the dialog
          await dc.End(dc.ActiveDialog.State);
        }
        else
        {
          // replay the flow
          await dc.Replace("yourdialogName", dc.ActiveDialog.State);
        }
    },

Combining Retry and Replay

There is nothing stopping you using a combination of the methods. If you want to enable an editing scenario but want the framework to ensure a criteria is met for specific steps then just implement both.

Matching names in LUIS

I recently hit a thorny issue with allowing my Bot to understand names via LUIS. I tried to train the LUIS model with various utterances of, ‘how is ‘ but it really wasn’t getting anywhere. There are just so many variations of names that unless I retrieved a large name data set then this really wasn’t going to work. So I fell back to using a very specific match. Yes, that is not really in the spirit of LUIS but at least it keeps all the language models in one place. My trick is pretty simple;

  • Create your Intent; e.g. HowIs
  • Add your *specific* utterance; e.g. how is paul
  • Go to the Entities options and create a new Entity called ‘Name’…
  • Set the Entity Type to RegEx…
  • Add your RegEx, e.g. (?<=^.{7})(.*)
  • Hit train
  • Now when you visit your Intent you should see the name aspect labelled as ‘Name’. As I say, it’s pretty crude and frankly you don’t need LUIS to do such a match. But if you believe there is value in keeping all your language matching models in the same place then it’s the best solution I currently have.

    NB I tried using a training list and Pattern and neither seemed to help

    Unit Test Flow in Botframework V4

    In theory unit testing in Botframework V4 should easy enough as it’s based on .net core. But, as with many things, it might not be that obvious. So here is a quick snippet showing a test for a Waterfall flow;

    [TestMethod]
    public async Task Waterfall()
    {
        var convoState = new ConversationState<Dictionary>(new MemoryStorage());
    
        var adapter = new TestAdapter()
            .Use(convoState);
    
        //var dialogState = convoState.CreateProperty("dialogState");
        var dialogs = new DialogSet();
        dialogs.Add("textPrompt", new TextPrompt());
    
        dialogs.Add("test",
            new WaterfallStep[]
            {
                async (dc, args, next) =>
                {
                        await dc.Context.SendActivity("Welcome! We need to ask a few questions to get started.");
                        await dc.Prompt("textPrompt", "What's your name?");
                },
                async (dc, args, next) =>
                {
                    dc.ActiveDialog.State["name"] = args["Value"];
                    await dc.Context.SendActivity($"Thanks {args["Value"]}!");
                    await dc.End();
                }
            }
        );
    
        await new TestFlow(adapter, async (turnContext) =>
        {
            var state = turnContext.GetConversationState<Dictionary>();
            var dc = dialogs.CreateContext(turnContext, state);
            await dc.Continue();
            if (!turnContext.Responded)
            {
                await dc.Begin("test");
            }
        })
        .Send("Hello")
        .AssertReply("Welcome! We need to ask a few questions to get started.")
        .AssertReply("What's your name?")
        .Send("Paul")
        .AssertReply("Thanks Paul!")
        .StartTest();
    }
    

    A couple of points to note, you have to send a message to start the flow, and make sure you call turnContext.GetConversationState rather than using a local version of the state.

    Download Transcript option in Botframework v4

    This is a whistle stop tour for adding a method to allow the user to download their transcript.

    Automatically record the transcript

    You need to add the built-in transcript services. In this example I’ll use the In-Memory store, you’ll want to evaluate the others for production code. Add this to the startup.cs

    var memoryTranscriptStore = new MemoryTranscriptStore();
    …
    options.Middleware.Add(new TranscriptLoggerMiddleware(memoryTranscriptStore));
    

    Next we’ll create a component to expose the ITranscriptStore (memoryTranscriptStore) to the dialog context. We’ll do that via our own middleware;

    public class TranscriptProvider : IMiddleware
    {
        private readonly ITranscriptStore transcriptStore;
    
        public TranscriptProvider(ITranscriptStore transcriptStore)
        {
            this.transcriptStore = transcriptStore;
        }
    
        public async Task OnTurn(ITurnContext context, MiddlewareSet.NextDelegate next)
        {
            context.Services.Add(transcriptStore);
            await next();
        }
    }
    

    Now add this to the startup middleware;

    options.Middleware.Add(new TranscriptProvider(memoryTranscriptStore));
    

    Download the transcript

    The middleware will now capture all the activity traffic to and from the bot and user. We can add a simple mechanism to request the transcript file. In your bot’s OnTurn method we can hardcode a ‘Transcript’ message/command;

    if (context.Activity.Text.Equals("Transcript", StringComparison.InvariantCultureIgnoreCase))
    {
        var transcriptStore = context.Services.Get();
        var transcripts = await transcriptStore.GetTranscriptActivities(context.Activity.ChannelId, context.Activity.Conversation.Id);
    
        var transcriptContents = new StringBuilder();
        foreach (var transcript in transcripts.Items.Where(i => i.Type == ActivityTypes.Message))
        {
            transcriptContents.AppendLine((transcript.From.Name == "Bot" ? "\t\t" : "") + transcript.AsMessageActivity().Text);
        }
    
        byte[] bytes = StringToBytes(transcriptContents.ToString());
    
        var contentType = "text/plain";
        var attachment = new Attachment
        {
            Name = "Transcript.txt",
            ContentUrl = $"data:{contentType};base64,{Convert.ToBase64String(bytes)}",
            ContentType = contentType
        };
    
        var activity = MessageFactory.Attachment(attachment);
        await context.SendActivity(activity);
    
        return;
    }
    ...
    private byte[] StringToBytes(string transcriptToSend)
    {
        byte[] bytes = new byte[transcriptToSend.Length * sizeof(char)];
        System.Buffer.BlockCopy(transcriptToSend.ToCharArray(), 0, bytes, 0, bytes.Length);
        return bytes;
    }
    

    When your user types in ‘Transcript’ they’ll be provided with a download attachment called ‘Transcript.txt’.

    Production Ready

    The above code is great for early testing but you should probably consider using the Download Activity Type and providing a URL to the full transcript. The above code has a nasty weakness in that it must fit inside the maximum reply payload for the bot, ~94K. You could just truncate the body but I’ll leave that up to you. Note, as of writing the emulator has an issue where it will allow you to click on the download but then it gets into a pickle and launches Windows Store. If you try this on a webchat it works fine.

    Manipulating waterfall steps Botframework v4

    To be honest I’m not even sure that this is strictly supported or that it is even a good idea, but as a point of interest you can manipulate the waterfall steps in v4. E.g. The standard flow is; step 1 -> step 2 -> step 3 -> step n. If your code realises that the user should skip a step then it can invoke the ‘next’ function;

    async (dc, args, next) =>
    {
        if (someCondition)
        {
            await next(args);
            return;
        }
    }
    

    That’s pretty easy. The difficult question, and one that I’m not even sure is (or should be) a valid one, how do you go back a step? Well, it is possible but it’s messy and you can’t get back to the initial step (although that is just starting again). You can go back a step by manipulating the dialog state;

    // at step n
    async (dc, args, next) =>
    {
        if (someCondition)
        {
            var targetStep = 1;
            dc.ActiveDialog.Step = targetStep - 1;
            await Step1Prompt();
            return;
        }
    }
    

    I don’t recommend this approach, it’s ugly, but you know…possible.