JsonReaderException: Error reading JArray using Adaptive Cards with Bot Framework

AdpativeCards should work well in the Bot. You should be able to produce great looking cards and also be able to respond to user input. In the latter case you are likely to run into a nasty little problem with how the serialization works with the Bot Framework and Adaptive Cards. If you’re struggling then there is a cheeky workaround;
The links starts at ~11mins into the whole tutorial https://youtu.be/2Hy9m5MvD4o?t=673

Advertisements

Gotcha with Bot Framework AutoSaveStateMiddleware and “long” flows

The AutoSaveStateMiddleware is a great addition to a Bot project. On every turn it can save the BotState objects you configure it to look after. Takes a way a lot of noise and accidentally forgetting to call save yourself. However, this is a little gotcha with this mechanism. Consider a classic Pizza ordering scenario;

  • USER – What pizzas do you have?
  • BOT – We have Vegetarian, Pepperoni, Ham & Cheese, Ham & Mushroom

From a Bot Framework perspective you might implement that as a series of Waterfall steps – potentially using a 3rd party service, i.e. potentially unreliable – slow;

  • Gather Available Ingredients For Location Step, NextAsync
  • Gather Available Pizza Recipes for Available Ingredients Step, NextAsync
  • Show Available Pizza Step

So far so good. Now, let’s say that before we started this Waterfall off we provided the user with a ‘See our drinks menu’ hero card button. What we expect to happen is that the drinks button will be shown before the potentially longer running food menu is processed and shown to the user. But what happens if the user presses that drinks button before the food menu has run through its steps and displayed its results? Well, something not so pleasant is the answer. Since the AutoSaveState will fire on the Turn, it won’t have fired until it prompts the user for their pizza choice. Therefore, even though the Drink button is in the same overall flow, the same activity, when the message is received by the bot it will NOT have an Active Dialog set in its context. This means that the message will NOT be passed onto what we think is our active dialog.

What solutions are there?

  • You can manually call the Save State, but it’s sort of a pain because that’s why we’re using the AutoSaveState
  • Simply ignore this issue. You may lose messages but the user will just try again
  • Use a base class for your dialogs that overrides NextAsync and saves the Dialog state

Upgrading your bot to .net core 2.2, warnings

When you upgrade your bot from .net core 2.0/2.1 to 2.2 you may see an warning that states; ‘A PackageReference to ‘Microsoft.AspNetCore.All’ specified a Version of 2.2.1. Specifying the version of this package is not recommended’.

To avoid this warning;

  1. Right click the bot project in the solution explorer and ‘unload’ it.
  2. Right click the unloaded project and edit it
  3. Change
    <PackageReference Include=”Microsoft.AspNetCore.All” Version=”2.2.1″/>
    To
    <PackageReference Include=”Microsoft.AspNetCore.All” />
  4. Reload the project.

 

Getting started with LUIS Containers

LUIS Containers is a really interesting (preview) feature. One of the issues with LUIS, and machine learning in general, is that each endpoint learns differently. E.g. if you create two LUIS endpoints, JANET and JOHN, and import the exact same JSON model then Janet and John will likely give you different scores for exactly the same utterance. Whilst that is just something you need to appreciate it, it does make a lot of workflow scenarios very difficult, E.g. testing. One potential solution to this is to export the LUIS model as a Container. In my initial tests this seems to clone the server, i.e. you get JANET2 rather than daughter of JANET – you get the same scores for the same utterance. Here is my add-on help to the documentation.

The main documentation is currently located at Install and run LUIS docker containers. If you don’t want to follow my guide then my advice is to read that but before you do anything read the next document along as that has a better explanation of the settings  Configure Language Understanding Docker containers

My guide to get up an running with LUIS Containers (for Windows 10)

I’m assuming you’ll be using the Command prompt and not a Bash terminal (as used in the official guide).

  1. Install Docker – if you’re new to Docker then I’d recommend at least following the first page of Get Started. Install Docker for Windows 10.
  2. Create a folder on your local disk that will container the LUIS model, e.g. C:\LUIS\Input and a folder for the output, e.g. C:\LUIS\Output
  3. Open the Docker settings panel (from the taskbar tools) and ensure you are sharing the local disk where you created the folders in (2)
  4. Go to LUIS and export the version of the model you want to use. Remember to select export as a Container.
  5. Get the Docker image for LUIS,
    docker pull mcr.microsoft.com/azure-cognitive-services/luis:latest
    
  6. Now create the container, remember this is all one line in CMD, so consider writing a batch file for it.
    docker run --rm -it -p 5000:5000 --memory 4g --cpus 2 --mount type=bind,src=c:\luis\input,target=/input --mount type=bind,src=c:\luis\output,target=/output mcr.microsoft.com/azure-cognitive-services/luis Eula=Accept Billing=https://YOUR_REGION.api.cognitive.microsoft.com/luis/v2.0 ApiKey=YOUR_API_KEY
    

    Where the Billing endpoint can be take from the first part of the Endpoint address in LUIS ‘Keys and Endpoint settings’ and the ApiKey is only the set of digits from the key of the same page. E.g. 3q919c439w2445f217b3w262622331c1

  7. You can then use your REST client of choice. You may use http://localhost:5000/swagger/index.html but be warned you’ll need to convert the AppId to a GUID (online coverter)…I know, right?? You should get an error saying something like; No model found with the given Application ID. If you do, then look at the output from your Docker console window. It should say something like could not find file xyz.gz.
  8. Copy the file you downloaded in (4) and put it into the input folder you created in (2). Carefully examine the name of the file. It needs to perfectly match the name of the file in the error message in (7)
  9. Ctrl+c the Docker container. Re-run the command from (6). Retry your REST call. You should now be working 🙂

NB, when you finish with this and try it again at a later date you may get errors when restarting your container, step (6). It may say something like, Error starting userland proxy. This appears to be a problem with Docker on Windows 10. You need to reopen the Docker desktop from the Taskbar and select Restart. This can take a little time but keep hovering over the icon and it will show you when it’s running again. Then you can re-issue step (6) and everything should be fine again.

One last note is that you need to stay online when using the Container. You can temporarily go offline but any prolonged absence and the LUIS Container will take itself offline with failed to reach metering endpoint,  resource temporarily unavailable. Shame, that scuppers offline use. Oh well, can’t have everything you want 🙂

 

Fetching all the LUIS intents in the Bot Framework

I decided that today was the day that I could no longer write a useful LUIS + Bot by only consuming the top scoring intent. So I checked the little Include all predicted intent scores switch in LUIS and ensured the ‘REST-API’ results had returned all the predictions. Yay. Changed my code to consume them to discover I was still only getting the top intent. Turns out you to do a little more work with the Bot SDKs to see the other intents;

v3 SDK

In your code that implements the LUISDialog base class;

protected override LuisRequest ModifyLuisRequest(LuisRequest request)
{
request.Verbose = true;
return base.ModifyLuisRequest(request);
}

v4 SDK

In the code where you create your LUIS Application and Recognizer, add the IncludeAllIntents options;

var app = new LuisApplication(luis.AppId, luis.AuthoringKey, luis.GetEndpoint());
var recognizer = new LuisRecognizer(

app,

new LuisPredictionOptions { IncludeAllIntents = true });

botframeworkoptions.state is obsolete

An recent update to the Bot Framework means that you may see some obsolete messages when trying to add conversation/user state, etc. to the options object in the setup. Don’t worry, it’s easy enough to alter.

...
// Create and add conversation state.
var conversationState = new ConversationState(dataStore);
// REMOVE THIS -> options.State.Add(conversationState);
services.AddSingleton(conversationState);
...