Website Name Change
I have changed the name of this website from Mining Business Data to BotFlo. I am offering a 60% off discount on both my Dialogflow ES and Dialogflow CX courses till April 20th 2021 for people who can help me spread the word about my new website.
The ability to do conversation analytics in Dialogflow is directly tied to the accuracy of your bot’s intent mapping. (Here is an article which talks about measuring your Dialogflow bot’s accuracy)
This is why Chatbase is not a particularly useful tool for Dialogflow conversation analytics.
Unlike Google Analytics funnels, where you can be certain that a user click is exactly what shows up in the “funnel”, the same thing cannot be said about Chatbase funnels (read my article on Chatbase funnels if you haven’t used them before) which actually rely on Dialogflow’s intent mapping being accurate.
Even in their documentation, Chatbase recommends that you manually inspect the transcript to ensure that the correct intent was mapped. They recommend that you read the conversation transcript to spot UMM errors (what are UMM errors).
Unfortunately, this somewhat beats the whole point of using Chatbase for aggregate level analysis given that you cannot be certain the individual pieces of data are exactly what you expect.
Note: this is why I recommend to my clients to roll out their own analytics systems. Chatbase doesn’t help much with improving the actual accuracy of your intent mapping, and the rest of the features it provides are not very useful, at least for the typical client I work with. Please don’t take this to mean I am against Chatbase in general. I just think it isn’t a good fit for most of my clients.
With all this in mind, here are some tips for doing conversation analytics in Dialogflow.
Improve intent mapping accuracy manually first
At the beginning, you will almost certainly be doing a lot of manual work analyzing your History and Training data inside Dialogflow. This will help make the intent mapping more accurate and make your conversation analytics data more useful.
Don’t use slot filling
Another reason to avoid slot filling (why I usually avoid slot filling) is the trouble you will have with your conversation analytics if you use Chatbase. They aren’t handled particularly intelligently in Chatbase (it uses sub-intents). This is to be expected when you realize that slot filling allows the user to send unlimited number of undetected messages before they exit the slot filling loop.
Minimize candidate intents
I recommend people use a context lifespan of 1 for their intents. This reduces the number of candidate intents at each step, and in turn makes it much easier to analyze your conversation analytics (because there are fewer unexpected intents in your Chatbase funnels).
Follow good conversation design principles
Generally speaking, you should try to make all training phrases within an intent as similar as possible (minimize intra-intent variation) while also trying to make training phrases from two different intents as unique as possible (maximize inter-intent variation). This is another way to help Dialogflow do better intent mapping.
Avoid doing your own NLU on the backend
Suppose you used the wildcard entity to do the job of the regexp entity before the Regexp entity was introduced. This means there is a step in your intent mapping where the NLU is not transparent to Chatbase (or any other conversation analytics system). This is why I recommend people avoid reinventing the NLU wheel in backend code, and instead use the explicative approach as much as possible (even if it means more intents in your Dialogflow agent).
Use a custom integration
The custom integration gives you access to the raw request and response messages. You should log these request response pairs to your database. In fact, this is a big reason why you would be better off using a custom integration. Over time, the extra effort of implementing your own custom integration more than pays off when it comes to building maintainable and easy-to-debug Dialogflow bots.
Log conversations to a document DB
Since all the request and response messages are in JSON format, and new fields are sometimes added into this JSON object format, it is a good idea to use a document database such as Cloud Firestore to store your conversation logs. If you are going for a relational database, use one which makes it easy to query JSON objects, such as PostgreSQL.
Generally speaking, all the things you do here will also help you build better Dialogflow bots. This extra little bit of rigor in designing your conversations will make it much easier when you do conversation analytics.
Tip: My Dialogflow Conversation Design course is a good reference if you would like to design your Dialogflow ES bots to be more analytics friendly.