(0:00 - 0:28) All right, so the first thing that I would like you to do is go and read this Dialogflow Python web tutorial I have on my site and I want you to do that in some good detail. And here is the reason why I am suggesting that. Now if you scroll down to the static response section in the tutorial, what you will notice is that there is this sort of like a small piece of code that you can use and here I am using the service called Replit.
(0:29 - 1:07) So you can just copy paste this entire code into the Replit REPL and you will have a working webhook at that point. What I have is a very small chunk of code which does the task. You know, we are trying to get it to work and I am not interested in creating like a very complex bot which goes and looks up like a big database and comes back with a bunch of information and does a whole bunch of things and finally produces this fulfillment information and gets the rich response to render into the Dialogflow messenger bot.
(1:08 - 1:30) And of course, most of the online tutorials you find would probably do that and I don't blame them. They are trying to focus on something else where they are trying to help you build a bot from end to end. But the problem there is that often you don't understand the individual components which are required to make this work.
(1:30 - 2:02) In contrast, in this tutorial that I have created, what I am doing is I am going to, I am taking you step by step through creating more and more complex webhooksYou can use webhooks to add custom business logic in both Di... More. So that way you understand what the most basic thing is like you need to know like just sending a chunk of information back from your webhook is actually sufficient to have something show up in your messenger bot or whatever you are building. And when you start from that point, it actually becomes easier for you to build other things on top of it.
(2:03 - 2:59) And in the case of rich responses for Dialogflow integrations, like for example the Dialogflow messenger or maybe you are using Slack or Skype or WhatsApp or something like that, all of them require that you understand the, of course for WhatsApp there are no rich responses, but let's say it's the Facebook messenger, there is a, you have to understand the format that you have to send the data across. And before you get into that details about the format, you should first have something which is very simple and which actually works and is able to produce some result in the chat user interface of the specific integration that you are building. And that is why I think that it's a good idea to start by creating some webhook code, which does the absolute minimum that you need to do to get it to work.
(2:59 - 3:27) And then once you do that, you can build on top of it and maybe pull the data from like a web, instead of like just generating it in the code, instead of hard coding those values into the webhook, you can instead maybe, you know, get the information from a database and do things like that. But those things have to be after the first step. And the first step is that you understand the different moving parts so that you get a good picture of what is going on.
(3:28 - 3:32) And then you will be able to do the more complex things very quickly.
Understanding the Custom Payload Part 1
(0:00 - 3:31) All right, now let's try and understand the custom payload that is used in Dialogflow and exactly what functionality that it serves because it's actually used for a lot of things and getting a basic understanding of what the custom payload is used for will actually help you not only with the Dialogflow messenger bot but even for other integrations that you might be building. Now to show as a simple example I have this thing called the course bot on my course website you if you go to learn.miningbusinessdata.com you will be able to interact with this bot. Now when you interact with the bot it may not have exactly the same information I keep you know improving and updating this bot but for now for the purposes of this demo this is sufficient.
So you can see that as soon as I say hello it comes back with this would you like to see the current deals and see that it's got a question and I'm sorry a button so I'm going to click on this button and it takes a few seconds and then it comes back with some information and then there is another button and then there's a clickable link so let's click on this button again and then it shows me four more buttons which I can click one of them that's not the important thing here the important thing is that to be able to build a bot like this I have to specify the information which is coming back from dialog flow inside the custom payload. Now in this case you can notice that what we have is a button list a list of you know choices they are single select that is you can't select multiple of these buttons you can only select one at a time and even if you noticed in the previous step as soon as I clicked on it it took the text of the button and added it into the chat window as if the user had typed that text into the chat window so how do you do this with the Zoho dialog flow integration and if I not mentioned it I want to point this out now I'm using the Zoho live chat service and I've integrated my chat bot into their service okay so let's take a look at how Zoho you know in the documentation how they mentioned that you do this now we are here at this page which explains how to do the single select option for dialog flow right and they say that you have to add this information and this JSON object that you see it actually gets pasted into the custom payload that is the specific intent which handles the whatever the user typed when it comes back with this response and shows a list set of buttons you have to use this as the custom payload that is you take copy paste you copy this from the from here and paste it into the custom payload and dialog flow then you should be able to see an output which looks like this and of course this is specific to that example but you noticed in our case that it had a different list of buttons like individual course course bundle dialog flow toolkit MBD archive and all that so I'm going to show you how you do this in dialog flow and that is I'm going to show you the JSON object corresponding to this in the inside my dialog flow console and then I'm coming I'll come back and explain how to do the you know what's the point of this and how exactly the custom payload is used for this particular example
Understanding the Custom Payload Part 2
(0:00 - 0:52) All right, so now what we'll do is we'll take a look at the custom payload which was used to generate this list of buttons in the Zoho chatbot. So I'm going back to my dialog flow intentBoth Dialogflow ES and Dialogflow CX have the concept of int... More here and this was the intent which actually fired when we got this list of buttons to show up. And notice that the custom payload here has a few fields.
The first is there is a field called platform and you can see that it has the word Zoho sales IQ. And then you have a field called input and you can see that the field type is select and it has a list of options and the options is like an array. You can see that these, you know, square brackets indicate that it's an array and it's an array of strings where each string has the text which goes into the button text.
(0:52 - 7:38) And then you also have the replies field which sort of goes on top of these buttons where it says okay which of these are you interested in and then it shows this list of options. Now let's go back to the Zoho documentation and this is where we were and remember that, you know, when you put the custom payload in the right format it's supposed to show up as the output shown similar to the output shown here which is, you know, that's exactly what we got, right. The purpose of adding this Zoho sales IQ platform field is that when the integration code sends some response from Dialogflow back to the Zoho chatbot, so they have built their own integration that is Zoho has built a one-click integration with Dialogflow and in case you're not familiar with it, I think you should go and check it out.
It's a very good integration. So and what happens is when the custom payload JSON object is sent back via the integration code, it has this object inside that is the this object comes back as part of the response and when it looks at the custom payload what Zoho is going to do is it's going to check oh is there any message here which corresponds to Zoho sales IQ and if that is then it's going to take this whole JSON object, right, it goes one level up and it takes the whole JSON object and then uses this whole information and accordingly renders the response into the live chat widget that you have for Zoho, right. So what's happening here is that the platform field is acting as a hint to the specific channel that you're using.
So in this case it's Zoho, it could also be Skype, it could be Slack and in the case of those platforms you will find that the integration code actually sends a message where it says the platform is Slack or the platform is Skype or maybe it has a unique number which is understood by those platforms. I don't remember exactly but I think that it probably says Slack and Telegram and all that but the important point is that this word, this field platform is used to distinguish. It's giving a hint to those channel integrations and saying to them like hey this is what you expected, this is the format that you wanted and this is the object that I have and you can use all this information to render the rich control into your channel as your need is, you know, as whatever your user expects.
Maybe it's a card type which goes to Slack and all of them will have this platform and then it will have a JSON object and the format of this object is usually specified by the channel. In this example, for example, Zoho decided that to be able to render this list of buttons and have something on top of the, you know, show up as some text on top of it, you need to have this format. Okay, you have to follow this format so that it is able to render the list of buttons into the live chat widget.
It will have a different format. For example, if you want to show a calendar, let's just quickly click on this calendar thing and you see that in this case the platform is still the same. The replies will have choose a date and in this case the input, you can see that the type has changed from select to calendar and then you have a label and you can see that it will render like an actual calendar inside the live chat widget.
So, the point I'm making overall is that custom payloads are used to send rich responses, even regular responses, but custom payloads specifically are used for rich responses for different channels and they are, the important thing is that all these channels should make sure that they have a field which they will select for themselves and which has to be, you know, used in the custom payload that you send so that those channels know which platform the message is intended for. You know, it's possible that you might have the same response going to different channels, like the same intent could be, you know, you have a bot in Dialogflow which is integrated with Slack, with Skype, with Zoho, with everything and you have to use the platform field to distinguish which message object is intended for which channel. And the other thing that I want to point out is, you can understand that in Dialogflow today, like if you were to go to Dialogflow and look at the console, you can see that with the plus sign you can see that these are all the built-in integrations that Dialogflow already has, right, Facebook and Slack and Telegram.
Let's just quickly go into Facebook because I want to show something. In Facebook, there are these different responses that are supported. There's a text response, image, card, quick replies and custom payload.
Now, if you were to use, let's say, the image, right, notice that here you're not actually using the custom payload, you are just typing the image URL into an actual, like, box. Now, this is under the hood also using a payload except that it's not called the custom payload, it's a different type of payload. And the problem with this is that now Facebook may be improving their channel so much that they will be keep on adding newer rich type of responses and it's not possible for the Dialogflow team to be on top of every single possible rich response that Facebook has added.
In the same way, it's not possible for Dialogflow team to, you know, know exactly which type of rich response is going to be supported by the Zoho live chat service because just like Zoho, there are so many other live chat services and many of them have, you know, rich controls like these input cards. It's not possible for the Dialogflow team to go and build the specific information into their console because there's just too many of these integrations, there's just too many of the rich responses, each of them have their own idiosyncrasies. So, that's where they introduced this idea of the custom payload and then you can just say custom payload and then you make sure that you provide this information and then as long as the format that you put into the custom payload is one which is honored by the channel integration that you're interested in.
(7:38 - 9:12) And notice that here it says Facebook and then goes something, you know, with the information. But for the Zoho, we had to actually, if you remember, we had to put platform and Zoho sales IQ and this is a sort of like a protocol that these companies have agreed amongst themselves. And if you don't have this integration supported directly inside Dialogflow, then your option would be to add this platform field and then provide whatever platform name and then based on that you can get this message object.
All right, I think that that's a fairly complex explanation, I think, but at the same time, getting a good understanding of what is going on under the hood for these rich responses and specifically with regards to custom payload will actually help you design any kind of integration, any kind of rich response. You can, if you want, you can create your own, I'll give a great example. As of today, there is no simple Android integration for Dialogflow.
There is no simple way to have an app which is built for Android, which uses rich controls and which uses Dialogflow in the backend. So let's say that you decided to build your own Android, you know, framework or whatever, you know, a library for Android, which will allow you to do this kind of stuff. Then what you will do is you will define a platform field that you want your users to use.
(9:12 - 10:17) So maybe you will call it like Android Dialogflow library or something and then that would be in the name of the platform, that is, it will be added as the platform value and then you will also declare like what fields that you support. Maybe you will have similarly like select options and all that. And then you will, based on that, you will actually render the appropriate widget into your Android apps user interface.
Now, having said all this, there is one more thing that I would like to point out. So for that, I am going to go to the integration for the Dialogflow messenger. All right.
So now I am in the Dialogflow messenger integration and I'm going to click on show button type. Remember that this also uses the custom payload, right? And here, the important thing that you have to notice is that the custom payload is prefaced with this thing called rich content. Now, this is what acts like the platform field, which we use for Zoho.
(10:18 - 11:09) And if you remember for Facebook, I can even show that right here. In the case of Facebook, if you have to add a custom payload, you notice that you have this Facebook as the actual field inside the custom payload. So if it has Facebook, it goes to Facebook messenger.
If it has rich content, it goes to Dialogflow messenger. And I think that if you go to something else like Slack and try that, yeah, see, now it says Slack. And in that case, this payload is sent across and this is used by Slack.
The point I'm making is that the Dialogflow messenger for its part uses rich content as the field name to know that that object is intended for the Dialogflow messenger integration.
Understanding the Custom Payload Part 3
(0:00 - 0:28) Okay, so to wrap up this section on custom payloads, I would like to point out just a few more things. The first thing that I would like to point out is that you can see that in the case of the Dialogflow Messenger integration, you have to put the rich content field as sort of the parent object of everything, the whole content which goes inside of that field. So that indicates that this object is meant for the Dialogflow Messenger integration.
(0:29 - 5:29) And the Zoho SalesIQ is not a built-in integration. So what you do in that case is that you have to actually have a field which specifies the platform. Okay, there's a field with the name of platform and the value will be whichever channel is supported.
Now, if you go to the Facebook integration, you can see that the custom payload has the field called Facebook as the parent object and all this information is sent across. And then for the case of Slack, you can see that the parent is called Slack. So when you are deciding how to add these custom payloads, the first thing you have to see is, is there already a built-in integration in Dialogflow? Does it show up in this list? And if it does, then you have to put the custom payload through this, like clicking on this add responses and clicking on the custom payload.
On the other hand, if there is no built-in integration, or if you are looking for the Dialogflow integration, I mean the Dialogflow Messenger integration, you have to go to this section under the default responses. And then you have to add the responses over here. That is the custom payload.
And finally, the other thing you have to realize is that for the built-in integrations, the field structure, the structure of the JSON is a little different because here you can see that the type of integration is used as the parent field name and the entire object is used to render the control, while in the case of unsupported or rather the platforms which don't have one-click integrations, you have to explicitly specify the platform name here with the field called platform, and then this object as a whole will be sent to that particular platform. Now, I keep saying that this field is sent to that platform and this other field, this Facebook payload is sent to Facebook and this payload is sent to Slack. I don't want you to understand it as if Dialogflow is doing something where it sends these messages individually to different platforms.
That's not what is happening. What is happening is under the hood. The integration code is just going to send all these responses as a single big blob, like a big object blob which is in JSON format, and then those platforms are going to search through the JSON object which comes back and check to see if there is a custom payload which has a field called Facebook.
So, the Facebook integration and the Facebook service on their end will check to see if this is available and if it is, then they will render the information into their widget, into their chat service, and then Slack similarly will check to see if the payload has this Slack field, and if it does, it's going to dump all this information into the Slack chat user interface. So, that is what, when I say that this is sent to Slack and this is sent to Facebook, it's not as if they are all sent as individual messages to different channels. It's just that the whole object blob is sent in one shot and those channels are responsible for extracting the information which is relevant to their channel.
And by the way, again, even here, it's not as if Facebook has to write some extra code just to handle this custom payload coming from the Dialogflow integration. That's not how these things work. What actually happens is Facebook provides an API and a data exchange format where they say, hey, if anybody wants to send a rich response to Facebook, which will render, let's say, some kind of a calendar control in the Facebook messenger.
Now, I don't even know if it's supported in the Facebook messenger, but let's suppose that it is. Then they'll say, make sure that you send a field like this with all this information, and then we will take care of the rest. So, that means everybody who to integrate with Facebook messenger platform, and it doesn't have to be just Dialogflow.
Even Dialogflow has competitors. It's got Amazon Lex and then Microsoft Bot Framework. There is Rasa NLU.
All these are bot frameworks which could all do the same job that Dialogflow does. Although, I personally think that Dialogflow does a better job than all the others. But the point is that all those other services like Amazon Lex will actually follow a similar payload format, and they will just make sure that the format they send across to Facebook follows this data exchange format that Facebook decided on and provided to these other companies.
And as long as the format is defined in the way that Facebook expects it, it will just work. That is, you will just see that the message shows up as an actual rich control, rich response in those specific channels.
Trigger the static response
(0:00 - 0:32) Alright, so once you understand the previous concepts, especially to do with the custom payload, to get the rich response sent from your webhook to the Dialogflow Messenger integration actually is very straightforward. So what I'm going to do is, I'm going to use the bot we have built before and I'm just going to first say, show me suggestion chips, because this will show me the suggestion chips type. You can see that it comes back with yes and no suggestion chips, which is excellent.
(0:33 - 1:10) Now this is the first step, right? What we do is, we trigger the correct type of rich response that we want from our Dialogflow Messenger integration, but this time we have just used the static response. So if you were to go to my show suggestion chip type, so this is the intent, show me suggestion chips, and you can see that it sent this custom payload, rich content, this stuff back, and that is what rendered into this chat window as yes and no.
Get response object from history
(0:00 - 0:31) All right, so once we get the static response to render properly, the next step we have to do is go to the history tab in your agent, and you will see the conversation history over here. And here is the message I typed, show me suggestion chips, and it sent a response back which rendered here as we had expected, I do not have it now, but we got the response back. And what you have to do is click on this ellipsis here and click on raw interaction log, okay.
(0:31 - 1:06) Now what you have is this is the JSON which was exchanged back and forth, this was the interaction log that actually happened for the intent which fired. And here we are interested in only exactly one thing, you see this thing here called fulfillment messages, copy the whole thing. So you have to copy the whole thing, and make sure that you do not take the comma there, just like just start from the open quote, go all the way down to the close of the square bracket and copy that information.
(1:07 - 1:25) Now I am coming back to my replit webhook. And what I will do is, this is the, I already have this filled out, but I just do it again for your sake. So the earlier response was something like return and then it will say fulfillment text and you know, with something in a string.
(1:25 - 1:45) So I just replace the whole thing. And this is what it used to look like with return and then it will be a fulfillment text colon, some actual text, and that would be enough to trigger a text response from the webhook. Instead, what we are going to do is because we want a rich response, and to paste this whole thing here, okay.
(1:46 - 2:07) So you can see that we have pasted everything from the, what we copied over from Dialogflow, we pasted the whole thing into the replit webhook. And just to make it look prettier, you can use this auto format feature and give it a few seconds, I think, and it will actually format it in a way which is, it's easier to read. There you go.
(2:08 - 2:20) So what it did is that it just took the exact content we had, and it just indented it in a better way so that it's easier for you to read. And it's all like aligned and all that. Other than that, it's the same information.
(2:21 - 2:38) So now you can see that the format has to be written, there's a field called fulfillment messages, and then there's an array. And then inside the array, there's a single object, it's called, which has a field called payload. And that has an object inside, which has a field called rich content.
(2:38 - 2:51) And all this stuff here, which is coming from this stuff here. If you notice this payload, and this, just ignore the word payload. But just think of this, this object that we have.
(2:51 - 3:05) And you will actually see that it is more or less the same. I think it is identical to what we had with the, in what we defined inside the intent. Let me just double check that.
(3:05 - 3:18) But I think that that's what it is. You have this rich content and open an array. And then there's another array open object with options, text, yes, no, type, and then close the two arrays.
(3:18 - 3:51) So if you were to come back here, you will see that in the payload and the rich content, it has these two arrays, and then open the option object, which has these two values and then type chips, and then you close the two arrays. So yes, this is the, in other words, the way you do that is create a field called fulfillment messages, open an array, open an object, in that object, there will be a field called payload. And then that contains another JSON object, which will be equivalent to what you had in your static response that you sent.
(3:51 - 4:08) Now, of course, you don't want to do this all manually in any case. I'll just suggest that you follow the step I told you, which is to copy the whole thing from the training, I'm sorry, the history tab instead. So go to history, and click on this raw interaction log.
(4:08 - 4:19) And I'm just asking you to just copy this whole thing. And not worry about like what else you should have added there and all that stuff. And of course, there's also this language and I think that also matters.
(4:20 - 4:48) So now with this information, which is saved, I'm going to stop it and I'll run it again. And now this is the information which is on the webhook. Just for clarity to distinguish from the other type we declared, I'm just going to say, instead of yes or no, I'm going to say yep and nope, which is not a big difference, but just for us to distinguish and not accidentally, you know, make some mistakes.
(4:48 - 5:07) So this is the message that is sent from the webhook, so that you can send a rich response from your webhook to your Dialogflow messenger integration. And in the next video, we will see how to complete the loop.
Send rich response from webhook
(0:00 - 0:22) All right, so now that you have understood the format of the JSON response, which should be sent back from the webhook, the next thing that we have to do is create an intent which we can actually fire so that we can get the answer in the Dialogflow Messenger chat window. So I have created a new intent called show me webhook rich response. And you can see that the training phrase is just show me webhook rich response.
(0:23 - 0:44) And one thing I've done is I've added the text response saying this is from the Dialogflow console. And then I have enabled the webhook call for this intent. So the reason I'm doing this is because if there is a timeout on the webhook or if there is some error or something like that, as you know, what Dialogflow will do is it will fall back to the text response which is already declared over here.
(0:45 - 1:00) So if the webhook call fails for some reason, we should see this message. And I want to make sure that this message is very clear that it failed. By seeing this response, we already know that the webhook call did not work for some reason.
(1:00 - 1:20) Okay, so now I have enabled the webhook call for this intent. And in the fulfillment section, if you go to the fulfillment section, you should have the URL followed by the slash webhook. Again, if you're not clear on this, go and read the Python webhook tutorial that I talked about at the beginning of this section, and it will become clear what I've put here.
(1:20 - 1:48) Now, with this intent declared, this intent called show me webhook rich response, the only thing left for us to do is go back to our messenger bot and just type the word show me webhook rich response. So this will trigger the intent that we just created. And you can see that it comes back with the yep and the nope suggestions just exactly as we expected.
(1:48 - 2:08) And just to make it even more clear, what I'm going to do is I'm going to go to another intent. So let's instead of let's go for show. Let's see.
I'm thinking of button type. Okay, so let's think of let's do the show me button. Okay, so here I'm going to say show me button.
(2:08 - 2:17) Remember that this is the static response. And it's going to come back with this button with a link, which is go to Google. And then you can go to your history.
(2:18 - 2:35) Go to this show me button interaction, open the raw interaction log, and copy everything which is inside the fulfillment messages. Go back to our replit. Delete everything inside this return object.
(2:36 - 2:50) Control V to paste everything that we have. And then what I'll do is just for the sake of formatting, I'm clicking on this auto format button over here. It takes a few seconds for it to actually do the formatting.
(2:50 - 3:00) And now it's done. So what you can see is the way it's changed is there's the fulfillment messages, the payload, rich content is all the same. But what is inside this is changed now.
(3:00 - 3:06) Now it's got a text. Go to Google. There's a type and the type indicates that it's a button control.
(3:06 - 3:15) There's an icon. And then there's finally a link. So I'm going to stop it and rerun it so that we get the latest version of this webhook code.
(3:16 - 3:36) Now the other thing I have to do is actually I don't have to do anything else because the webhook is already connected. So the only thing I need to do now is to copy this and paste it here because this will trigger the webhook intent. But instead of yep and no, we should see if everything works fine.
(3:36 - 3:51) Instead of yep and no, it should say go to Google here now. And that's what we got. And remember, I had explicitly added the message saying that this is from the Dialogflow console.
(3:51 - 4:23) That is, if the webhook call hadn't worked properly, we should have seen this message. The fact that this intent triggered and we have enabled the webhook call and this came back tells us that everything is working exactly as expected. So this is how you can use the existing information that you can glean from your Dialogflow console to construct the appropriate rich response object in your webhook code.
Note: This is my old website and is in maintenance mode. I am publishing new articles only on my new website.
If you are not sure where to start on my new website, I recommend the following article:
