LUIS

Getting started with LUIS Containers

pauliomLeave a comment

LUIS Containers is a really interesting (preview) feature. One of the issues with LUIS, and machine learning in general, is that each endpoint learns differently. E.g. if you create two LUIS endpoints, JANET and JOHN, and import the exact same JSON model then Janet and John will likely give you different scores for exactly the same utterance. Whilst that is just something you need to appreciate it, it does make a lot of workflow scenarios very difficult, E.g. testing. One potential solution to this is to export the LUIS model as a Container. In my initial tests this seems to clone the server, i.e. you get JANET2 rather than daughter of JANET – you get the same scores for the same utterance. Here is my add-on help to the documentation.

The main documentation is currently located at Install and run LUIS docker containers. If you don’t want to follow my guide then my advice is to read that but before you do anything read the next document along as that has a better explanation of the settings  Configure Language Understanding Docker containers

My guide to get up an running with LUIS Containers (for Windows 10)

I’m assuming you’ll be using the Command prompt and not a Bash terminal (as used in the official guide).

  1. Install Docker – if you’re new to Docker then I’d recommend at least following the first page of Get Started. Install Docker for Windows 10.
  2. Create a folder on your local disk that will container the LUIS model, e.g. C:\LUIS\Input and a folder for the output, e.g. C:\LUIS\Output
  3. Open the Docker settings panel (from the taskbar tools) and ensure you are sharing the local disk where you created the folders in (2)
  4. Go to LUIS and export the version of the model you want to use. Remember to select export as a Container.
  5. Get the Docker image for LUIS, 
    docker pull mcr.microsoft.com/azure-cognitive-services/luis:latest
  6. Now create the container, remember this is all one line in CMD, so consider writing a batch file for it. 
    docker run --rm -it -p 5000:5000 --memory 4g --cpus 2 --mount type=bind,src=c:\luis\input,target=/input --mount type=bind,src=c:\luis\output,target=/output mcr.microsoft.com/azure-cognitive-services/luis Eula=Accept Billing=https://YOUR_REGION.api.cognitive.microsoft.com/luis/v2.0 ApiKey=YOUR_API_KEY

    Where the Billing endpoint can be take from the first part of the Endpoint address in LUIS ‘Keys and Endpoint settings’ and the ApiKey is only the set of digits from the key of the same page. E.g. 3q919c439w2445f217b3w262622331c1

  7. You can then use your REST client of choice. You may use http://localhost:5000/swagger/index.html but be warned you’ll need to convert the AppId to a GUID (online coverter)…I know, right?? You should get an error saying something like; No model found with the given Application ID. If you do, then look at the output from your Docker console window. It should say something like could not find file xyz.gz.
  8. Copy the file you downloaded in (4) and put it into the input folder you created in (2). Carefully examine the name of the file. It needs to perfectly match the name of the file in the error message in (7)
  9. Ctrl+c the Docker container. Re-run the command from (6). Retry your REST call. You should now be working 🙂

NB, when you finish with this and try it again at a later date you may get errors when restarting your container, step (6). It may say something like, Error starting userland proxy. This appears to be a problem with Docker on Windows 10. You need to reopen the Docker desktop from the Taskbar and select Restart. This can take a little time but keep hovering over the icon and it will show you when it’s running again. Then you can re-issue step (6) and everything should be fine again.

One last note is that you need to stay online when using the Container. You can temporarily go offline but any prolonged absence and the LUIS Container will take itself offline with failed to reach metering endpoint,  resource temporarily unavailable. Shame, that scuppers offline use. Oh well, can’t have everything you want 🙂

 

Fetching all the LUIS intents in the Bot Framework

pauliomLeave a comment

I decided that today was the day that I could no longer write a useful LUIS + Bot by only consuming the top scoring intent. So I checked the little Include all predicted intent scores switch in LUIS and ensured the ‘REST-API’ results had returned all the predictions. Yay. Changed my code to consume them to discover I was still only getting the top intent. Turns out you to do a little more work with the Bot SDKs to see the other intents;

v3 SDK

In your code that implements the LUISDialog base class;

protected override LuisRequest ModifyLuisRequest(LuisRequest request)
{
request.Verbose = true;
return base.ModifyLuisRequest(request);
}

v4 SDK

In the code where you create your LUIS Application and Recognizer, add the IncludeAllIntents options;

var app = new LuisApplication(luis.AppId, luis.AuthoringKey, luis.GetEndpoint());
var recognizer = new LuisRecognizer(

app,

new LuisPredictionOptions { IncludeAllIntents = true });

Get Sentiment Score from LUIS RecognizerResult

pauliomLeave a comment

Another quick extension method to help with getting a sentiment score from a RecognizerResult;

public static double? GetSentimentScore(this RecognizerResult luisResult)
{
    double? result = null;

    if (luisResult != null)
    {        
        var data = luisResult.Properties["sentiment"];
        var sentimentValues = data as IDictionary;
        var score = sentimentValues["score"] as JValue;
        result = (double)score.Value;
    }

    return result;
}

Extracting an Entity from LUIS in Bot Framework

pauliomLeave a comment

Recent changes to the supporting LUIS libraries in the Bot Framework have puzzled me. I find that whilst getting the Top Intent from the RecognizerResult recognizerResult?.GetTopScoringIntent(); is easy, getting hold of an Entity seems very strange. I’m not sure if it’s my SDK, LUIS models or what, but this is the code I’m currently having to use…and I confess I still find it all very odd [Edit 20-Nov-2018] Code changed to better cope with arrays coming and going :s;

public static T GetEntity<T>(this RecognizerResult luisResult, string entityKey, string valuePropertyName = "text")
{
    if (luisResult != null)
    {
        //// var value = (luisResult.Entities["$instance"][entityKey][0]["text"] as JValue).Value;
        var data = luisResult.Entities as IDictionary<string, JToken>;

        if (data.TryGetValue("$instance", out JToken value))
        {
            var entities = value as IDictionary<string, JToken>;
            if (entities.TryGetValue(entityKey, out JToken targetEntity))
            {
                var entityArray = targetEntity as JArray;
                if (entityArray.Count > 0)
                {
                    var values = entityArray[0] as IDictionary<string, JToken>;
                    if (values.TryGetValue(valuePropertyName, out JToken textValue))
                    {
                        var text = textValue as JValue;
                        if (text != null)
                        {
                            return (T)text.Value;
                        }
                    }
                }
            }
        }
    }

    return default(T);
}

Hope it helps

Tip to improving your LUIS with Entities model

pauliomLeave a comment

When creating an Intent that includes Entities, especially personName, then you may have to increase the number of utterances to achieve a match with a high confidence score. However, try to avoid using the specific test as your utterance. For example;

Consider the Intent, ‘WhenIsTheirBirthday’. You might provide utterances such as, ‘When is Jane’s Birthday’, ‘When is Bert’s Birthday’, etc. When you Test the phrases you might find a test that only results in a very low confidence score, e.g. ‘When is Tim’s Birthday’. Don’t be tempted to add that as an utterance, if you do you will be providing very specific training and you cannot be certain that your are really improving the model. In that example, you would just add some more utterances that are similar, but not the same, and re-train. Keep doing that and you should see, ‘When is Tim’s Birthday’ confidence start to rise until it goes above your required confidence level.

LUIS now understands people’s names

pauliomLeave a comment

This one managed to slip past me, but recently Microsoft Language Understand Intelligent Service (LUIS) has been updated with a pre-built entity to match people’s names. Seems like an obvious requirement but is pretty tricky to implement. To match people’s names;

  1. Open your KB in the LUIS UI of your choice
  2. Go to the Entity menu
  3. select Add Prebuilt entity
  4. choose personName

Now you can create an Intent such as “Call” with “Call Jane” and “Jane” will be matched to personName. So far (and testing is difficult) I’ve thrown all my usual problematic name variants (Western, Nordic, Chinese, Indian) and it’s managed them all, even “Call peter phoxterd-hemmington-smythe” 🙂

Gotcha with Skype, Bot Framework and LUIS

pauliomLeave a comment

I hit a very annoying problem today with one of my Bots. It uses LUIS to react to the intent of what the user is saying. Two of the intents are “Weather” and “HowAreYou”. I’d run through the emulator and all the flows were working well. The user says something like;

  • “What’s up?” and the Bot responds with, “Ok, thanks how are you?”
  • “What’s the weather like?”, you receive, “lovely and sunny”

I published to Skype and suddenly the flows were broken.

  • “What’s up?” – “Cold today, brrrr”
  • “What’s the weather like?” – “lovely and sunny”

Why has that broken? Well the problem is that when Skype sends a request it is NOT sending the text in plain, it’s HTML encoding it. So rather than LUIS receiving "What's up?" it’s actually receiving, "what&pos;s up". This is then mapped to the Weather intent rather than “HowAreYou”. This is frustrating because the LUIS support is via a Middleware service and I haven’t found an option that allows me to tell it to decode the in-bound text. So my solution is to place a decoder in the Middleware pipe *before* the LUIS Middleware;

 public class HtmlDecoderMiddleware : IMiddleware
    {
        public async Task OnTurn(ITurnContext context, MiddlewareSet.NextDelegate next)
        {
            var text = context.Activity.Text;
            context.Activity.Text = WebUtility.HtmlDecode(text);
            await next();
        }
    }

Matching names in LUIS

pauliomLeave a comment

I recently hit a thorny issue with allowing my Bot to understand names via LUIS. I tried to train the LUIS model with various utterances of, ‘how is ‘ but it really wasn’t getting anywhere. There are just so many variations of names that unless I retrieved a large name data set then this really wasn’t going to work. So I fell back to using a very specific match. Yes, that is not really in the spirit of LUIS but at least it keeps all the language models in one place. My trick is pretty simple;

  • Create your Intent; e.g. HowIs
  • Add your *specific* utterance; e.g. how is paul
  • Go to the Entities options and create a new Entity called ‘Name’…
  • Set the Entity Type to RegEx…
  • Add your RegEx, e.g. (?<=^.{7})(.*)
  • Hit train

    • Now when you visit your Intent you should see the name aspect labelled as ‘Name’. As I say, it’s pretty crude and frankly you don’t need LUIS to do such a match. But if you believe there is value in keeping all your language matching models in the same place then it’s the best solution I currently have.

    NB I tried using a training list and Pattern and neither seemed to help

    MS Bot framework / LUIS gotcha

    pauliomLeave a comment

    I’ve been training the LUIS service for use with the Microsoft Bot Framework and I did the cardinal sin of changing two things in my code; add some ‘clever’ OO stuff and add a new set of LUIS Intents. When I tested my Bot I kept getting, ‘The given key was not present in the dictionary’ when instantiating my LUIS Dialog. Turns out that if you have an Intent in LUIS but you have not YET implemented the handler in your LUIS Dialog then this is the error you get should the user hit that Intent. I suspect there is someway of specifying a default handler but I’ve yet to find it.