Dialogflow ES Conversation Analytics Tips

This article provides some tips for doing Dialogflow ES conversation analytics.

You cannot do analytics if you cannot calculate accuracy

The ability to do conversation analytics in Dialogflow is directly tied to the accuracy of your bot’s intent mapping.

(Here is an article which talks about measuring your Dialogflow bot’s accuracy).

At the beginning, you will almost certainly be doing a lot of manual work analyzing your History and Training data inside Dialogflow. This will help make the intent mapping more accurate and make your conversation analytics data more useful.

Don’t use slot filling (required parameters)

Another reason to avoid slot filling (why I usually avoid slot filling) is the trouble you will have with your conversation analytics if you use Chatbase. They aren’t handled particularly intelligently in Chatbase (it uses sub-intents). This is to be expected when you realize that slot filling allows the user to send unlimited number of undetected messages before they exit the slot filling loop.

Minimize candidate intents

I recommend people use a context lifespan of 1 for their intents. This reduces the number of candidate intents at each step, and in turn makes it much easier to analyze your conversation analytics (because there are fewer unexpected intentsBoth Dialogflow ES and Dialogflow CX have the concept of int... in your Chatbase funnels).

Follow good conversation design principles

Generally speaking, you should try to make all training phrases within an intent as similar as possible (minimize intra-intent variation) while also trying to make training phrases from two different intents as unique as possible (maximize inter-intent variation). This is another way to help Dialogflow do better intent mapping.

Use Versioning

As I mentioned in a previous article, you really cannot do conversational analytics without measuring bot accuracy first (well, you can, but it will be meaningless).

And you cannot measure bot accuracy without maintaining bot versions.

I used the Search tool to convert my agent ZIP files into a CSV, which I then imported into Airtable.

Design a Flowchart

I used Mindomo to design the bot flowchart and maintain versions.

It made it much easier for me to both design the bot and to translate it into a Dialogflow ES agent.

Use Airtable for calculating accuracy

Airtable is like a swiss-army knife for Dialogflow ES bot developers.

It can be used as a content management system for your ES bot.

You can use it as a database instead of using Google Sheets

And you can use it to measure the accuracy of your bot of course.

(As a bonus it is also very user friendly)

Use a conversation-friendly integration channel if possible

At some point in the conversation, the bot asks the user what tool they use to design a flowchart for their ES bot. Most people don’t use Mindomo, so it usually recommends the tool and also links to another article on this website.

Clicking on the link inside the bot opens the article in a new page, and the old conversation history is maintained so the user can just continue talking to the bot.

This seems pretty simple isn’t it?

But you cannot do this in Dialogflow Messenger.

So this bot turned into a very clear demonstration of why Zoho SalesIQ (and comparable live chat tools) is better than using the built in Dialogflow Messenger integration.

Don’t use the Training feature (blindly)

It is very tempting to map a user utteranceAn utterance is simply what the user says to the bot. Both E... using the Training feature. In fact, you can use it in two ways. You can take a correctly mapped user utterance and add it as an additional training phrase into that intent.

Or you can take an incorrectly mapped user utterance and add it to either the correct intent or perhaps create an entirely new intent.

I recommend that you don’t do this impulsively. Adding a user utterance from the Training feature can reduce both the cohesion and the separation of your bot and make it less accurate over the long run.

Instead, give some thought to how you are designing the training phrases inside your intents.

Set up automated tests

I have created a tool to automatically generate a test script from your Dialogflow ES agent ZIP file.

You can use it in conjunction with the pytest unit testing framework inside PyCharm and set up automated testing for your ES bot. I explain how to do this in my Improving Dialogflow ES accuracy course.

Follow a systematic process

Here are the results I got for my case study chatbot by using the accuracy formula

Version	1	2
True Positives	429	347
True Negatives	251	127
False Positives	12	14
False Negatives	16	3
Total Responses	708	491
True Positive Rate	60.59	70.67
False Positive Rate	35.45	25.87
True Negative Rate	1.69	2.85
False Negative Rate	2.26	0.61
Accuracy	96.05	96.54

As you can see, Airtable is also a good place to summarize the results and share them with other stakeholders. 🙂

The accuracy might seem pretty high. While my bot has multiple turns, remember that it is also a pretty simple bot and there isn’t a lot of scopeDialogflow CX scope helps you decide what set of actions CX ... for the user to “stray” from the conversation flow.

The accuracy you get will depend on a lot of factors, and unfortunately some of them will be outside your control. So instead of using these numbers as a yardstick, I strongly suggest you follow a process and make sure your bot metrics improve over time.