Upgrading your bot to .net core 2.2, warnings

When you upgrade your bot from .net core 2.0/2.1 to 2.2 you may see an warning that states; ‘A PackageReference to ‘Microsoft.AspNetCore.All’ specified a Version of 2.2.1. Specifying the version of this package is not recommended’.

To avoid this warning;

  1. Right click the bot project in the solution explorer and ‘unload’ it.
  2. Right click the unloaded project and edit it
  3. Change
    <PackageReference Include=”Microsoft.AspNetCore.All” Version=”2.2.1″/>
    To
    <PackageReference Include=”Microsoft.AspNetCore.All” />
  4. Reload the project.

 

Advertisements

Getting started with LUIS Containers

LUIS Containers is a really interesting (preview) feature. One of the issues with LUIS, and machine learning in general, is that each endpoint learns differently. E.g. if you create two LUIS endpoints, JANET and JOHN, and import the exact same JSON model then Janet and John will likely give you different scores for exactly the same utterance. Whilst that is just something you need to appreciate it, it does make a lot of workflow scenarios very difficult, E.g. testing. One potential solution to this is to export the LUIS model as a Container. In my initial tests this seems to clone the server, i.e. you get JANET2 rather than daughter of JANET – you get the same scores for the same utterance. Here is my add-on help to the documentation.

The main documentation is currently located at Install and run LUIS docker containers. If you don’t want to follow my guide then my advice is to read that but before you do anything read the next document along as that has a better explanation of the settings  Configure Language Understanding Docker containers

My guide to get up an running with LUIS Containers (for Windows 10)

I’m assuming you’ll be using the Command prompt and not a Bash terminal (as used in the official guide).

  1. Install Docker – if you’re new to Docker then I’d recommend at least following the first page of Get Started. Install Docker for Windows 10.
  2. Create a folder on your local disk that will container the LUIS model, e.g. C:\LUIS\Input and a folder for the output, e.g. C:\LUIS\Output
  3. Open the Docker settings panel (from the taskbar tools) and ensure you are sharing the local disk where you created the folders in (2)
  4. Go to LUIS and export the version of the model you want to use. Remember to select export as a Container.
  5. Get the Docker image for LUIS,
    docker pull mcr.microsoft.com/azure-cognitive-services/luis:latest
    
  6. Now create the container, remember this is all one line in CMD, so consider writing a batch file for it.
    docker run --rm -it -p 5000:5000 --memory 4g --cpus 2 --mount type=bind,src=c:\luis\input,target=/input --mount type=bind,src=c:\luis\output,target=/output mcr.microsoft.com/azure-cognitive-services/luis Eula=Accept Billing=https://YOUR_REGION.api.cognitive.microsoft.com/luis/v2.0 ApiKey=YOUR_API_KEY
    

    Where the Billing endpoint can be take from the first part of the Endpoint address in LUIS ‘Keys and Endpoint settings’ and the ApiKey is only the set of digits from the key of the same page. E.g. 3q919c439w2445f217b3w262622331c1

  7. You can then use your REST client of choice. You may use http://localhost:5000/swagger/index.html but be warned you’ll need to convert the AppId to a GUID (online coverter)…I know, right?? You should get an error saying something like; No model found with the given Application ID. If you do, then look at the output from your Docker console window. It should say something like could not find file xyz.gz.
  8. Copy the file you downloaded in (4) and put it into the input folder you created in (2). Carefully examine the name of the file. It needs to perfectly match the name of the file in the error message in (7)
  9. Ctrl+c the Docker container. Re-run the command from (6). Retry your REST call. You should now be working 🙂

NB, when you finish with this and try it again at a later date you may get errors when restarting your container, step (6). It may say something like, Error starting userland proxy. This appears to be a problem with Docker on Windows 10. You need to reopen the Docker desktop from the Taskbar and select Restart. This can take a little time but keep hovering over the icon and it will show you when it’s running again. Then you can re-issue step (6) and everything should be fine again.

One last note is that you need to stay online when using the Container. You can temporarily go offline but any prolonged absence and the LUIS Container will take itself offline with failed to reach metering endpoint,  resource temporarily unavailable. Shame, that scuppers offline use. Oh well, can’t have everything you want 🙂

 

Fetching all the LUIS intents in the Bot Framework

I decided that today was the day that I could no longer write a useful LUIS + Bot by only consuming the top scoring intent. So I checked the little Include all predicted intent scores switch in LUIS and ensured the ‘REST-API’ results had returned all the predictions. Yay. Changed my code to consume them to discover I was still only getting the top intent. Turns out you to do a little more work with the Bot SDKs to see the other intents;

v3 SDK

In your code that implements the LUISDialog base class;

protected override LuisRequest ModifyLuisRequest(LuisRequest request)
{
request.Verbose = true;
return base.ModifyLuisRequest(request);
}

v4 SDK

In the code where you create your LUIS Application and Recognizer, add the IncludeAllIntents options;

var app = new LuisApplication(luis.AppId, luis.AuthoringKey, luis.GetEndpoint());
var recognizer = new LuisRecognizer(

app,

new LuisPredictionOptions { IncludeAllIntents = true });

botframeworkoptions.state is obsolete

An recent update to the Bot Framework means that you may see some obsolete messages when trying to add conversation/user state, etc. to the options object in the setup. Don’t worry, it’s easy enough to alter.

...
// Create and add conversation state.
var conversationState = new ConversationState(dataStore);
// REMOVE THIS -> options.State.Add(conversationState);
services.AddSingleton(conversationState);
...

Introduction to writing Bot Application video tutorial

Introduction to the tutorial series is about writing bot applications using the Microsoft Bot Framework v4. The goal of this tutorial is to help someone who is interested in writing a Bot application that may not have much experience in software development. It will be a, “warts and all” journey to show how I would develop a bot. So I will be making mistakes and showing you how I go about resolving them. It will get you to a point where you can write your own conversational bot and provide you with some of the additional tools to help you resolve issues that you may find.

Introduction

Ep1 Setup

Ep2 Bot Basics

Ep3 Language Understanding

Ep4 Bot & LUIS

Ep5a Dialogs

Ep5b Dialogs

Ep6 Wrap up

Get Sentiment Score from LUIS RecognizerResult

Another quick extension method to help with getting a sentiment score from a RecognizerResult;

public static double? GetSentimentScore(this RecognizerResult luisResult)
{
    double? result = null;

    if (luisResult != null)
    {        
        var data = luisResult.Properties["sentiment"];
        var sentimentValues = data as IDictionary;
        var score = sentimentValues["score"] as JValue;
        result = (double)score.Value;
    }

    return result;
}

Automate Bot tests with saved transcript files

I’ve recently worked on a testing framework with Ben Williams (@negativeeddy) that allows you to write your own chat scripts. However, I thought that saving transcripts of your chat from the Bot Emulator would be a really helpful way to produce automated testing. Unfortunately I could not find any out-of-the-box support for running transcript files. Thus the following was born;

Testing with Transcript files

In my example I have the following chat conversation with my Adventure Bot;


USER
Examine table

BOT
The table is old but in good condition.
On the table there is a key 'Room Key A'.
ACTION
Take Room Key A

USER
Take key

BOT
You Take key.
ACTION
look
inventory

When you save the transcript file from the Emulator it contains the serialized version of those activities. Here’s a snippet of the above;

{
“type”: “message”,
“text”: “examine table”,
“from”: {
“id”: “5c8620e2-ccd8-4624-9c91-9842610220a4”,
“name”: “User”
},
{
“type”: “message”,
“serviceUrl”: “http://localhost:50090&#8221;,
“channelId”: “emulator”,
“from”: {
“id”: “1”,
“name”: “Bot”,
“role”: “bot”
},
“conversation”: {
“id”: “b1bbf220-011b-11e9-aa8d-d94902c621f5|livechat”
},
“recipient”: {
“id”: “5c8620e2-ccd8-4624-9c91-9842610220a4”,
“role”: “user”
},
“text”: “The table is old but in good condition.\nOn the table there is a key ‘Room Key A’.\n”,
“inputHint”: “acceptingInput”,
“suggestedActions”: {
“actions”: [
{
“type”: “imBack”,
“title”: “take Room Key A”,
“value”: “take Room Key A”
}
]
},

Write a test

Now we have the transcript file we want to use it from a test. First create a Test project and get the contents of the file. Note, I’m using MSTest and Moq. I’m taking a hacky short cut and reading the whole file but it should be using streams;


[TestMethod]
public async Task TakeKeyFromTranscriptTest()
{
  var transcriptFile = await File.ReadAllTextAsync("Transcripts\\examineTable.transcript");
  var activityTranscript = JsonConvert.DeserializeObject(transcriptFile);
  await TestBotTranscriptAsync(activityTranscript);
}

Set-up the test

Now we configure the test and the service-under-test, or in our case – a dialog.


private async Task TestBotTranscriptAsync(List activityTranscript, object request = null)
{
  // create mock version
  var mockDependency = new Mock();
  RecognizerResult recognizerResult = FakeRecognizerResult(
   "examine table",
   "Examine",
   "table");

  mockDependency.Setup(x => x.RecognizeAsync(
   It.Is(i => i.Activity.Text.Equals("Examine table", StringComparison.InvariantCultureIgnoreCase)),
   It.IsAny()))
   .Returns(Task.FromResult(recognizerResult));

  recognizerResult = FakeRecognizerResult(
   "take Room Key A",
   "Take",
   "key");

  // set up mock method
  mockDependency.Setup(x => x.RecognizeAsync(
   It.Is(i => i.Activity.Text.Equals("take Room Key A", StringComparison.InvariantCultureIgnoreCase)),
   It.IsAny()))
   .Returns(Task.FromResult(recognizerResult));

  this.Recognizer = mockDependency.Object;

  await CreateDialogTest(
   nameof(AdventureDialog),
   (id, svc, log) => new AdventureDialog(id, svc, log),
   options: request)
   .AssertBotTranscript(activityTranscript)
   .StartTestAsync();
}

private static RecognizerResult FakeRecognizerResult(string query, string intent, string noun)
{
    var resultJson = "{\"text\":\"" + query +
        "\",\"alteredText\":null,\"intents\":{\"" + intent +
        "\":{\"score\":0.9696778}},\"entities\":{\"$instance\":{\"noun\":[{\"startIndex\":8,\"endIndex\":13,\"text\":\"" + noun +
        "\",\"type\":\"noun\",\"score\":0.997288644}]},\"noun\":[\"" + noun + "\"]}}";
    var recognizerResult = JsonConvert.DeserializeObject(resultJson);
    return recognizerResult;
}

The Botness of the test is inside the CreateDialogTest method;


public TestFlow CreateDialogTest(string dialogId,
Func createDialog,
object options = null)
where TDialog : Dialog
{
  // setup the bot state for the test
  var storage = new MemoryStorage();
  var conversationState = new ConversationState(storage);
  var userState = new UserState(storage);

  var accessors = new AdventureCBCBotAccessors(conversationState, userState)
  {
   AdventureState = conversationState.CreateProperty(AdventureCBCBotAccessors.AdventureStateName),
   DialogState = conversationState.CreateProperty(AdventureCBCBotAccessors.DialogStateName),
  };

  // create the test adapter with any required middleware
  var adapter = new TestAdapter(sendTraceActivity: true)
   .Use(new GameServicesMiddleware(accessors.AdventureState))
   .Use(new AutoSaveStateMiddleware(conversationState));

  // Create or Moq any services
  ILoggerFactory factory = new LoggerFactory()
   .AddConsole()
   .AddDebug();

  var services = new BotServices(
   new BotConfiguration(),
   TestEndpoint
  );

  if (this.Recognizer != null)
  {
   services.LuisServices.Add("LuisAdventureBot", this.Recognizer);
  }

  // Create the dialog under test
  var dialogState = conversationState.CreateProperty("dialogState");
  var dialogs = new DialogSet(dialogState);
  TDialog dialog = createDialog.Invoke(dialogId, services, factory);
   dialogs.Add(dialog);

  // Start the test flow-loop to execute the chat
  return new TestFlow(adapter, async (turnContext, cancellationToken) =>
  {
   var dc = await dialogs.CreateContextAsync(turnContext, cancellationToken);

   var result = await dc.ContinueDialogAsync(cancellationToken);

   if (result.Status == DialogTurnStatus.Complete)
   {
    if (result.Result != null)
    {
     await turnContext.SendActivityAsync(result.Result.ToString());
    }
    return;
   }

   if (!turnContext.Responded)
   {
    await dc.BeginDialogAsync(dialog.Id, options, cancellationToken);
   }
  });

public EndpointService TestEndpoint { get; set; } = new EndpointService() { Endpoint = "https://example.com/bot" };
}

Execute a test

You may have noticed the lines;


.AssertBotTranscript(activityTranscript)
.StartTestAsync();

These are were the execution and testing happen. StartTestAsync start the test adapter but the AssertBotTranscript activity extension is where the magic happens. Before we look at that we need to create a couple of helper classes. These classes are responsible for finding all the message activies and to identify which ones are from the user or the bot.


public enum StepActor
{
  User,
  Bot
}

public class BotScript
{

  public BotScript(List transcript)
  {
   this.Steps = this.ParseScript(transcript);
  }

  public string UserName { get; set; } = "User";

  public IEnumerable Steps { get; }

  private IEnumerable ParseScript(List transcript)
  {
   foreach (var activity in transcript
    .SkipWhile(t => t.Type != ActivityTypes.Message || t.From.Name != this.UserName)
    .Where(t => t.Type == ActivityTypes.Message))
   {
    if (activity.From.Name == this.UserName)
    {
     BotStep step = new BotStep
     {
       Actor = StepActor.User,
       Text = activity.Text,
     };

     yield return step;
    }
    else
    {
     BotStep step = new BotStep
     {
       Actor = StepActor.Bot,
       Text = activity.Text,
       Actions = activity.SuggestedActions?.Actions.Select(a => a.Title).ToList(),
     };

     yield return step;
    }
   }
 }
}

public class BotStep
{
  public StepActor Actor { get; internal set; }
  public string Text { get; internal set; }
  public List Actions { get; internal set; }
}

Now we have our User and Bot steps we can execute the test. This is achieved by enumerating all the steps from the transcripts. If it is a User step then we send it into the Flow. If it’s a Bot step then we assert that the Bot has responded with the expected value in the step.


public static TestFlow AssertBotTranscript(this TestFlow testFlow, List transcript)
{
  var botScript = new BotScript(transcript);
  return testFlow.AssertBotScript(botScript);
}

public static TestFlow AssertBotScript(this TestFlow testFlow, BotScript script)
{
  foreach (var step in script.Steps)
  {
   if (step.Actor == StepActor.Bot)
   {
     testFlow = testFlow.AssertBotStep(step);
   }
   else
   {
     testFlow = testFlow.Send(step.Text);
   }
  }
 return testFlow;
}

public static TestFlow AssertBotStep(this TestFlow testFlow, BotStep expectedResponse)
{
 return testFlow.AssertReply(activity =>
 {
  var messageActivity = activity.AsMessageActivity();

  // check main text of message
  if (!string.IsNullOrEmpty(expectedResponse.Text))
  {
   string fixedExpected = SanitizeText(expectedResponse.Text);
   string actualResponse = SanitizeText(messageActivity.Text);

   Assert.AreEqual(fixedExpected, actualResponse);
  }
}
}

private static string SanitizeText(string text)
{
  string fixedReply = text;

  if (!string.IsNullOrEmpty(fixedReply))
  {
   fixedReply = fixedReply.Replace("\r", string.Empty).Replace("\n", string.Empty);
  }

  return fixedReply;
}

All that’s left to is to run the test;

Capture

 

It is just a start

I’ve left out of lot of additional code you need, e.g. handling Adaptive cards, Actions, Hero Cards, etc. but it’s exactly the same idea. You need to flesh out the BotStep class and AssertReply to handle the different forms of message activities. Hopefully, this will just be included in the Framework of the Bot Community framework and you will never need to write this yourself. But for now, this should help you out.