Website Name Change
I have changed the name of this website from Mining Business Data to BotFlo. I am offering a 60% off discount on both my Dialogflow ES and Dialogflow CX courses till April 20th 2021 for people who can help me spread the word about my new website.
If you are creating a large FAQ chatbot in Dialogflow, typing out all the intents one by one can be a slooooooooow and painful process.
Note: this article talks about the use case where you have already identified the Dialogflow intents. If you haven’t yet defined the intents and the training phrases inside them, check out my tool which can help you automatically extract Dialogflow intents from your chat logs.
Thankfully, Dialogflow has an agent ZIP file import feature, and it is possible to programmatically create this ZIP file.
In this article, I describe how you can type out your intent information into a spreadsheet and then use a CSV export of your spreadsheet to generate a Dialogflow agent ZIP file. I also discuss the different aspects you need to consider as you try to use this approach for more complex FAQ bots.
By following this template, you should be able to build a tool which can create a Dialogflow FAQ chatbot with the push of a button.
I also have an app which can do this for you, if you don’t want to build such a tool by yourself.
This article talks about text-only FAQ chatbots. If you would like to see how to do something similar for rich response channels such as Dialogflow Messenger, Facebook Messenger and Google Assistant check out the follow up article.
Want to build it yourself? Here is how it works
First, let us export a ZIP file for a very simple Dialogflow agent and take a look at the directory and file structure, as well as the contents of the JSON file.
I will use an agent which has exactly two intents (taken from the prebuilt smalltalk agent).
I went to the Export and Import tab, and clicked on Export the ZIP file for this simple agent. Once you uncompress the ZIP file, the following file/folder structure is revealed:
And you can inspect the smalltalk.agent.be_clever_usersays_en.json.
You see that it is an array of JSON objects, and an individual JSON object has an id, the data, isTemplate, count and updated fields.
Next, notice the response JSON object smalltalk.agent.be_clever.json
Say, you want to add your own intent.
You simply need to follow the same pattern as the other intents, and create two new JSON files, and ZIP it up and import it into Dialogflow.
So you generate a random GUID, and get all the other stuff exactly in place. In my example, I made a copy of both the smalltalk.agent.be_clever_usersays_en.json and the smalltalk.agent.be_clever.json files and added a random GUID in the copied files, and created a ZIP file and tried to do a restore.
Make sure the filenames match:
Here is what the end result looks like after the import:
In fact, an even easier option is to simply omit all the GUIDs in the intents as well as the userSays as you create your ZIP file. Dialogflow will simply see that the ID is missing and generate its own.
But there are a couple of important things to note:
- Dialogflow will not permit two intents with the exact same name – be sure to change the name
- Your files should be of the format filename.json and filename_usersays_en.json – if you don’t follow this convention, the import doesn’t work correctly
Simple question and answer FAQ chatbot
If you just want to create a simple FAQ bot with one question corresponding to one answer, it requires a simple CSV format. There is a column with a user says message (Query), and then another column with the text response (Response).
What you want to do: You upload this CSV format, and you have an agent at the end of it.
How to convert your CSV file into Dialogflow agent ZIP File
These are the steps to take your 2 Column CSV file specified in the format above and convert it into a Dialogflow agent ZIP file in BotFlo.
1 Upload your CSV file
2 Download agent ZIP file
3 Restore agent ZIP file in Dialogflow
4 Test your agent
How to add multiple training phrases per intent
Now, the simple 2 Column CSV format is not sufficient. For example,
- how can you handle multiple user says (training) phrases per intent?
- what if you want to add multiple text responses?
- what should be used as the intent’s name?
Suppose we create the following CSV column structure to accommodate the points from the previous section. You have an intent ID, followed by intent name, followed by userSays phrase, and the final column will be your text response.
For example, this is what the CSV format would look like (screenshot taken from my BotFlo app). All rows must have an intentID to identify the intent. Notice that you can have multiple queries (training phrases) and multiple responses per intent.
While this gives you the ability to use multiple userSays phrases as well as multiple text responses, it is also more complex to process this format.
How to add input and output contexts
If you would like to add input and output contexts into the CSV format, you need to consider the following:
- you can have multiple input and output contexts per intent
- every output context should have an associated context lifespan
Here is the format I came up with for this purpose (again taken from my BotFlo app):
What about entities?
Specifying entities inside this CSV format is more complex.
Why is this?
It’s because when you add a training phrase containing an entity, Dialogflow automatically parses the entity value (annotates it, that is). For example, you can see below that the word tomorrow is automatically annotated as a date entity.
If you download and inspect the JSON of this intent, it looks like this:
So adding entities via a CSV format will have two issues:
- it makes the CSV format more complex (not such a big problem)
- the annotation will not automatically work (bigger problem)
For example, you might annotate a name as @sys.given-name in your CSV file, for example. But that will not automatically be accepted by Dialogflow which might reject the entity value since its system entity cannot handle it.
(I am working on a solution to this also, and will update this post soon).
What about followup intents?
Sometime around May 2017, Dialogflow (which was then API.AI) introduced the followup intents feature. These allowed you to have a hierarchy of intents, like below.
Since the number of hierarchical levels is (theoretically) unlimited, you can imagine that it is not possible to capture this very complex structure in the CSV format at all.
So an app which can convert a CSV file into a Dialogflow agent ZIP file will NOT be able to capture followup intents.
If you want to use followup intents in your Dialogflow agent (make sure you read this article to learn about some situations where you should not use follow up intents), you cannot use this approach.