Home / DialogFlow ES / Dialogflow ES Automated Conversation Testing
DialogFlow ES

Dialogflow ES Automated Conversation Testing

You don’t have much control over what people say to your bot. But you do have control over setting up some conversation tests which make sure that you didn’t break something in your chatbot unexpectedly.

How to break your Dialogflow bot

Here are some ways you can break your Dialogflow ES bot.

Add a new training phrase into an intent

Action: You want to handle a new phrase in an existing intent.

Problem: You have a very similar phrase already declared as part of another intent. As a result, the old phrase now gets mapped to the one you changed, potentially breaking some functionality.

Update an existing intent

Action: You modify the user’s phrase in an existing intent.

Problem: An user phrase which used to get mapped to the current intent is now triggering the fallback intent.

Approve a phrase in the training tab

Action: You see an unmapped phrase in the training tab and believe it should be mapped to an existing intent. So you select the intent, and click on the Approve button

Increase the ML threshold

Action: To get a tighter mapping, you slightly increase the ML threshold.

Problem: You didn’t know this before, but many of your phrases were already being mapped very close to the ML threshold score. After you increase it, the old phrases are getting mapped to the fallback.

Decrease the ML threshold

Action: To get a more generous mapping (or because you don’t want to spend more time adding new user phrases), you lower the ML threshold

Problem: Even junk user phrase inputs with very poor matching starts getting mapped to your intents. As a result, phrases get matched even if an entity is not present. This leads to low quality input data getting into your system.

Dialogflow updates their platform

Now clearly, this isn’t something you can control.

But there are also occasions when your chatbot, which was humming along very well, suddenly breaks. You go to the forum and notice that people are complaining. Turns out, Dialogflow made some updates to their service which breaks the old functionality in some way.

Automated Conversation Testing

In my Improving Dialogflow ES accuracy course, I explain how you can set up Automated Conversation Testing using Python.

Here is a screenshot of the desired result. Using pytest and PyCharm, you can very easily set up automated conversation testing for your Dialogflow ES agent. As you can see in the screenshot, this allows you to quickly check if your existing intents are still working as expected.

Generating test scripts

The larger your bot, the more tedious and difficult it can be to generate suitable test phrases for your test script. This is a big reason why people don’t create automated conversation tests in Dialogflow ES.

You can use the simple “filler word” trick to get around this problem. I also go over this entire system in much more detail in the course.

Dialogflow ES Conversation Testing

Course Preview: Automated Testing Basics

Course Preview: Automated Testing Implementation using Python

<— End of article —>


This website contains affiliate links. See the disclosure page for more details. 
"The magic key I needed as a non-programmer"

The custom payload generator was the magic key I needed (as a non-programmer) to build a good demo with rich responses in DialogFlow Messenger. I've only used it for 30 minutes and am thrilled. I've spent hours trying to figure out some of the intricacies of DialogFlow on my own. Over and over, I kept coming back to Aravind's tutorials available on-line. I trust the other functionalities I learn to use in the app will save me additional time and heartburn.

- Kathleen R
Cofounder, gathrHealth
"Much clearer than the official documentation to be honest"

Thanks a lot for the advice (of buying and following your videos)! They helped a lot indeed. Everything is very clear when you explain, much clearer than the official documentation to be honest 🙂

Neuraz T
Review for Learn Dialogflow CX
"I will strongly recommend this course because even I can learn how to design chatbot (no programming background)"

I think Aravind really did a great job to introduce dialogflow to people like me, without programming background. He organizes his course in very clear manner since I have been a college professor for 20 years. It is very easy for me to recognize how great Aravind’s course is! Very use-friend and very easy to follow. He doesn’t have any strong accent when he gives the lectures. It is so easy for me to understand. Really appreciate it.

Yes, I will strongly recommend this course because even I can learn how to design chatbot (no programming background) after studying Avarind’s course, you definitely can!

Ann Cai
Review for Learn Dialogflow ES

Similar Posts