Website Name Change
I have changed the name of this website from Mining Business Data to BotFlo. I am offering a 40% off discount on both my Dialogflow ES and Dialogflow CX courses till end of April 2021 for people who can help me spread the word about my new website.
To be very clear, this article is not about web design. I am not a web designer, or for that matter a designer of any sorts. I am a DialogFlow trainer/coach/consultant, and I am writing this article after a few attempts to create a web based chatbot based on DialogFlow. You can see plenty of articles on this broad topic on this site, and this article is a proposal based on my learnings till date. (And it will probably be a bit opinionated)
My first chatbot
The very first chatbot I built with DialogFlow was a sports stats chatbot for getting records about the sport of cricket. It is actually a chatbot with high utility value (by which I mean, till date, Google can’t answer the questions that my chatbot can), but I didn’t build it out fully. I am not sure where Cricinfo draws the line when it comes to using its site and data to build such a bot.
As I created that bot, I ran into a pretty annoying issue right away. The built in web demo integration from DialogFlow didn’t support clickable hyperlinks. It will just display the full URL, and you need to copy/paste it into another tab to go to the link.
Then I realized that without using the REST API exposed by DialogFlow, it is not practical to build a chatbot with hyperlinks. Unfortunately, if you wish to use the REST API, then you take responsibility for the entire web UI (including the front end design). I figured out how to do this, and even created a course based on that. (I don’t recommend the course anymore because the stuff is now obsolete and you will see why soon).
Adding Actions on Google controls for a richer experience
Even with enabling hyperlinks, the resulting chatbot was extremely limited. It couldn’t really display things like images. You could customize it to render all kinds of things, but I was looking for a more standardized way of doing things. For example, if you chose to customize it so that you handled most of the HTML rendering after the response came back from DialogFlow’s REST API, you didn’t have any kind of WYSIWYG (What You See Is What You Get) way to build out the chatbot.
At this stage, I thought of a neat ‘hack’. Enter the Google Assistant tab inside the DialogFlow console.
Now the Google Assistant tab allows you to specify buttons, list boxes, carousels and hyperlinks etc. It can be a truly rich interaction for the end user, as you can see from the video below.
At that time, I thought I had squeezed every last drop of ‘extensibility’ out of the DialogFlow system since I actually had a working system which could render rich controls and had buttons and such for interaction. The nice thing about the “Assistant” chatbot is that it is actually WYSIWYG. But it faces the same challenges that the UI for the Google Assistant faces on your Android phone (see “Removing choices” section below).
Around the same time, I also experimented with a very interesting WordPress plugin I had recently seen. The plugin allows you to easily create a chatbot and add it to your WordPress site. There is only one catch – the plugin is not build on any kind of NLP (Natural Language Processing) technology, so unlike DialogFlow, you need to manually set all the conversation paths.
I call them TapBots, because for the most part, the user is only tapping (clicking on a button) and not typing.
You might be thinking: that’s pretty dumb.
Except, the metrics told me otherwise.
User engagement and conversation flow
As you can see, I am very interested in enabling web based chatbots using these bot frameworks such as DialogFlow. Here is a question: why are you using a chatbot?
“To demonstrate how cool these technologies are”
“To show how cool our company is”
“We want our website to appeal to a web savvy audience”
These aren’t very good answers, even if sometimes you might want these things as side effects 🙂
In reality, you are building your chatbot as an alternative UI to help your users get things done. And there are actually some very nice benefits (and also some pretty nasty downsides) to using a conversational interface to help folks do their tasks.
This means you want your chatbot to have two important attributes:
- Task oriented conversation flow
- User engagement
Now these WordPress plugins are clearly very good at 1. After all, there is no way for the user to make a wrong choice. But do they help with user engagement?
It definitely got more engagement than my not-quite-as-visually-appealing but cleverer bots.
One thing which was noticeable was also the speed of the interactions. Clearly, people were clicking through much more quickly with the tapbots, as you might expect. On the other hand, the regular, rich control chatbots seemed to be causing people to slow down.
My guess, at this point, is that this is because they can sometimes have too many options. See the image below for the number of choices a user has when they are about to respond to the carousel selector in my “Assistant” chatbot.
Giving the user more options was not helping a lot. There is also another problem: the behavior of the controls were not consistent.
Now, I tried to more or less exactly mimic the behavior of the Google Assistant on a smartphone. What I noticed is that these inconsistencies are there in the Google Assistant also.
If you clicked a suggestion chip (the blue buttons), they disappeared. If you clicked on a list box choice, the list box was still on the screen. What happens when you click on the list box again from the previous turn? Google Assistant still allows you to make that choice. This is like getting the answer for the previous question. Handling these quirks is very challenging from a UI standpoint.
On the other hand, the TapBot UI is much simpler. You can either tap (click), or you could type. There wasn’t any way to do both.And you certainly can’t go back to make a selection from a previous step. I credit this ‘lack of choices’ design for the much higher user engagement. I haven’t read the book, but I am reminded of the title “Don’t make me think” by Steve Krug. 🙂
Adopting this idea to build modern web chatboxes
So my proposed idea would work like the image below. While the design is overlaid on top of a smartphone screen, I think most desktop chat boxes are also similarly shaped so the ideas still apply.
We first divide the user interface into three sections. We render the chatbot’s messages in the top section (section 1), similar to how it already works. But the important thing is that the top section will not be active or clickable (except for hyperlinks).
Section 2 and 3 will lie at the bottom, and together, shouldn’t take up more than about a third of the total height. Section 2 will contain the rich control such as a selection list box, or a date picker, or similar controls which are flat and expose additional functionality once you click the dropdown arrow.
Finally, section 3 will be used only for suggestion chips. If there are no suggestions for a given turn, we will collapse its container and expand section 1 (to let user see more of the chat interface).
Like I mentioned in the previous paragraph, the suggestion chips, which are usually used to change the conversation path, will always line up at the bottom to help the user quit the conversation or start over (as an example). If this becomes a “convention”, it will sort of help reinforce the notion in user’s minds.
The separation of rich “content” – e.g. hyperlinks, images, videos, gifs – to be shown on the top section from rich “controls” – list boxes, checklistboxes, date pickers etc. will help us define what goes into section 2.
Since we will not have a lot of space to work with, most often the controls will be similar to dropdown boxes which allow the user to make (sometimes) complex selections such as the typical datepicker.
We will display either a rich control such as a datepicker, or a textbox for free form input, in section 2 but never both.
Do you think this would be a good way to build out a web based chatbot interface? Let me know in the comments.
"The magic key I needed as a non-programmer" The custom payload generator was the magic key I needed (as a non-programmer) to build a good demo with rich responses in DialogFlow Messenger. I've only used it for 30 minutes and am thrilled. I've spent hours trying to figure out some of the intricacies of DialogFlow on my own. Over and over, I kept coming back to Aravind's tutorials available on-line. I trust the other functionalities I learn to use in the app will save me additional time and heartburn. - Kathleen R Cofounder, gathrHealth