How to quickly convert an FAQ page into a Dialogflow ES bot
In this article, I will explain how you can quickly turn an FAQ page into a Dialogflow chatbot.
Let us consider this example page which has an FAQ for Shopify payments.
This would be a one-and-done FAQ chatbot. That is, usually, you do not expect follow up questions referencing the previous answer. (Answering follow-up questions would require the use of followup intentsBoth Dialogflow ES and Dialogflow CX have the concept of int... More or input and output contextsContexts are used in Dialogflow ES to manage the state of th... More, for which you need to do more work and hence spend more time).
Before we move on to the rest of the article, I will address an objection some people will have: “Why not just use the Dialogflow Knowledge Base feature?”
Here are some reasons:
| Intent Based FAQ Bot | Knowledge Based FAQ Bot | |
| How we build it | Using intents and training phrases | Using the knowledge base feature |
| Support for multiple responses for the same query | No | Yes |
| Multiple training phrases per intent | Yes | No |
| Supports entitiesBoth Dialogflow ES and Dialogflow CX support entities and th... More in training phrases | Yes | No |
| Support for contexts (and follow up) | Yes | No |
| Easy to construct rich responses (e.g. for Dialogflow Messenger) | Yes | No |
| Use term reinforcement for adding weights to specific words and phrases | Yes | Blackbox |
| Allows us to minimize intra-intent variance and maximize inter-intent variance (as recommended by Google) | Yes | Blackbox |
Except for being able to get multiple responses for a single query (highlighted), you can see that an intent based FAQ bot gives you a lot of benefits and much fewer limitations when compared to knowledge based FAQ bots.
You might still decide to use the Dialogflow Knowledge base features, but I am presenting a case for the pros and cons of using the Knowledge base feature. If you decide to just use an Intent based FAQ bot, the rest of the article explains how you can build it out quickly.
Scraping the FAQ page
The first step is to scrape the FAQ page.
You can use the free WebScraper Chrome extension to quickly scrape the questions from the page.
First scrape all the questions into a CSV file using the WebScraper extension. This should usually be straight-forward and you can do this by just following the documentation.
This is what the CSV file will look like.
Very important for the next steps: Sort the first column by web-scraper-order (ascending).
Now use the WebScraper to download the section which contains both the questions and the answers. You will usually do this by clicking on a single question, and clicking on the [p] Hotkey button to expand the selection out inside the Select tool.
This is what the CSV file will look like
Convert the CSV files into the 2 Column CSV format
I have created a tool which can merge the two CSV files and create a new CSV which has Question-Answer pairs.
On the left hand side, select the CSV file which has only the questions. On the right hand side, select the CSV file which contains both questions and answers.
Note: make sure the CSV files have been sorted by web-scraper-order. (Usually this doesn’t matter for the Full CSV file which generally has only a single row)
Click on the Merge CSV files to create a CSV file which extracts the answer for each question and constructs a CSV file which has the Question in the first column and the corresponding Answer in the second column.
Note: since this is based on the WebScraper extension, you cannot get rich formatted text (e.g. hyperlinks) in the Answer column.
Your CSV file will look like this:
Convert an FAQ page into a Dialogflow chatbot
Now you have a CSV file in the 2 Column format.
This means you can use the 2 column FAQ bot converter to convert the merged CSV file into an FAQ chatbot.
About this website
I created this website to provide training and tools for non-programmers who are building Dialogflow chatbots.
I have now changed my focus to Vertex AI Search, which I think is a natural evolution from chatbots.
Note
BotFlo was previously called MiningBusinessData. That is why you see that watermark in many of my previous videos.