Website Name Change
I have changed the name of this website from Mining Business Data to BotFlo. I am offering a 60% off discount on both my Dialogflow ES and Dialogflow CX courses till April 20th 2021 for people who can help me spread the word about my new website.
I got this question in my comments recently:
This question got me thinking about building custom integrations versus using the built-in 1-click integrations in Dialogflow.
I do understand that a lot of people choose Dialogflow mainly for its ability to do 1-click integration with multiple platforms. But my experience till date has been that almost always, you are better off building out a custom integration with Dialogflow.
First, some definitions.
What is a 1-click integration?
These are the integrations you see on your Dialogflow console.
Here is an example of a 1-click integration I built. (Sorry, I don’t have bandwidth to maintain it right now).
What is a custom integration?
When you build a custom integration, you will use Dialogflow’s REST API and build an integration similar to the 1-click integration, except that you can customize its behavior.
A good example is the 1-click web demo of my Freelance Advice Bot, which looks like this:
Contrast this with my SupportBot, which has a lot more features (clickable hyperlinks, ability to display images, buttons etc).
By using a custom integration, I got around a lot of limitations imposed by the existing 1-click web demo.
Dialogflow Integrations 101
A small detour. I have noticed that people are sometimes unclear exactly what an integration is. In that case, my mini-course Dialogflow Integrations 101 can help.
Here is one of the videos from the mini-course.
Now let us look at the reasons why custom integrations are usually better than using the 1-click integrations.
1 Number of potential integrations are increasing
As you can see from the list of integrations on the Dialogflow console, the number of potential integrations keeps increasing.
This has meant that over time, the Dialogflow team has had lesser and lesser time to allocate per 1-click integration, which has meant that the documentation for these integrations isn’t easy to find (and sometimes are non-existent).
2 The integration channels themselves are evolving quite fast
It is also important to note that many of these channels, such as Facebook Messenger and Skype, are themselves in a race with Google (and Dialogflow) to build out their own bot frameworks.
This means sometimes the existing 1-click integrations break when new updates are rolled out on those other channels.
3 It is not too difficult to build out your own custom integration
Another reason to build a custom integration is that there is a reasonable amount of documentation on how to do it. Besides, once you figure out how to do one custom integration, you possess the knowledge to create many more such custom integrations.
Nearly all custom integrations follow a template:
- you register a webhook (URL where data is sent)
- each time someone sends a message to your bot on the channel the corresponding message’s details are sent to your webhook
- the webhook collects the message information and extracts the actual text of the message from the (typically) JSON object
- the text of the message is sent to Dialogflow via its REST API so Dialogflow can detect the intent
- the response from Dialogflow (also in JSON format) has a bunch of information including
- the text response
- complex response objects (such as cards, lists etc if relevant)
- other metadata which can sometimes be useful (e.g. intent detection confidence score)
- the webhook code will relay this response from Dialogflow back to the originating channel by calling an API endpoint
- the JSON object which is relayed back to the channel is usually “massaged” so that it is in a format which can be rendered in the chat window (e.g. if it is a Telegram bot, you can display buttons by using the JSON object which corresponds to a button)
- there is usually a chatId associated with the original message, and if your response to the channel uses the correct chatId, the bot response will show up inside the chat window
On first glance, this looks quite complex. I agree that it is not trivial. But when you build one custom integration and get the hang of it, you can build more custom integrations quite a bit more easily.
4 Input preprocessing
Now we get to the really powerful stuff.
Intercepting the message coming in BEFORE sending it to Dialogflow allows you to preprocess the input. This allows some cool things such as:
- rate control
- selective response
- don’t respond to abusive messages
- change formats for better entity detection
- for example, Dialogflow is able to extract dates in MM-DD-YY format better than MM/DD/YY format when I last checked. So you can, in theory, search for these patterns using some kind of regular expression, and modify it into a date that Dialogflow can parse more easily
5 Output postprocessing
After the response comes back from Dialogflow, you can also do post-processing of the response.
A very good example of this is to do a handoff to a human operator if the intent calls a fallback twice in a row (usually an indication that your bot doesn’t yet have the ability to satisfy the user’s request).
6 Using knowledge connectors on an as-needed basis
Dialogflow recently introduced a cool new feature called Knowledge Connectors (still in BETA, as of August 2018). I will be writing a tutorial about this feature shortly.
I am concerned that the knowledge connectors feature doesn’t give you much control over when knowledge intents can be triggered (for example, it would be great to have them be invokable only if a context is set).
Instead, Dialogflow presents you with this completely opaque system of deciding when a knowledge intent should fire:
If you have an agent which has both Knowledge Connectors (i.e. can answer FAQs) and also has additional functionality (e.g. book a ticket), you can selectively choose to show a knowledge connector response only if you wish. For example, you can do so only if a specific context is active, or if the fallback intent has fired and you think the FAQ answer might actually be more helpful.
While this comes under the broader umbrella of post-processing of response, it is a very specific solution to a problem some of you may be facing.
There are also some cons for this custom integration approach.
Did you choose Dialogflow for its “1-click integrations”? Then this article spells trouble for you. 🙂
Also, you need to add the code running the integration to the “list of things you need to maintain”, which isn’t so great.
On the other hand, if you do need powerful integrations, I think you should go with custom integrations. Plus, sometimes they are the only choice (e.g. handoff to human).