How to measure the deflection rate in Dialogflow ES
In a previous article, I discussed how to measure the accuracy of your Dialogflow ES chatbot.
However, the accuracy usually needs to be combined with the deflection rate of your chatbot to get the full picture of its usefulness.
Note: The deflection rate is defined as the number of times the bot deflects the query from the human.
Let us take a look at the accuracy mapping diagram again.
A true positive and a false positive are both considered deflections.
Let us suppose users sent 100 messages to the bot, and the breakdown is as follows:
| True Positives | 50 |
| True Negatives | 40 |
| False Positives | 5 |
| False Negatives | 5 |
| Total Messages | 100 |
Let us consider a different breakdown
| True Positives | 70 |
| True Negatives | 20 |
| False Positives | 5 |
| False Negatives | 5 |
| Total Messages | 100 |
As you can see, the accuracy is 90% in BOTH cases, but we obviously prefer the second case because it has a higher true positive rate. In other words, the bot is able to correctly answer many more questions on its own.
You can intuitively note that while we do want as high a deflection rate as possible, it should not be because we have a lot of false positives.
Conversely, we do not want a lot of true negatives just to maintain a high level of accuracy, because it means you are not making the most of your chatbot deployment.
About this website
I created this website to provide training and tools for non-programmers who are building Dialogflow chatbots.
I have now changed my focus to Vertex AI Search, which I think is a natural evolution from chatbots.
Note
BotFlo was previously called MiningBusinessData. That is why you see that watermark in many of my previous videos.