How do you measure your Dialogflow bot’s accuracy
The confusion matrix
You may be familiar with the term error matrix or confusion matrix. If not, don’t worry! It is a way to measure if classification techniques work well, and it is quite appropriate in this case because under the hood, Dialogflow takes the user’s input and classifies it to the nearest matching intentBoth Dialogflow ES and Dialogflow CX have the concept of int... More.
Sample size
Consider the last 100 user messages to your bot. If you don’t have that many, get a few beta testers to try out your bot for a few minutes.
Chatbase UMM
A while back, I wrote a post which linked to Chatbase’s UMM method which provides a way to reason about your chatbot’s accuracy. While it is a good idea and I do derive some ideas from it, it is not particularly useful because there isn’t any way to measure the accuracy using the UMM method.
About this website
I created this website to provide training and tools for non-programmers who are building Dialogflow chatbots.
I have now changed my focus to Vertex AI Search, which I think is a natural evolution from chatbots.
Note
BotFlo was previously called MiningBusinessData. That is why you see that watermark in many of my previous videos.
When a take a look to theIntent Detection Confidence, i see a score of 0.83768564. I suppose there is no way i know wich intent gets fired with the 0.16231436 score from the total of 1… as dialogflow dont display such an intent…
Yes, that is correct. I wish Dialogflow would have implemented a top N intents feature. It is probably Dialogflow’s biggest shortcoming when compared to the other bot frameworks.
Hi sir,would you mind if , you know any tool to generate the trained model accuracy by a ghraps like the tensorboard
I don’t know of any, but such a tool would obviously be very helpful for those building Dialogflow chatbots.