Home / DialogFlow ES / A simple method to evaluate multiple bot frameworks
DialogFlow ES | REST API

A simple method to evaluate multiple bot frameworks

Website Name Change

I have changed the name of this website from Mining Business Data to BotFlo. I am offering a 60% off discount on both my Dialogflow ES and Dialogflow CX courses till April 20th 2021 for people who can help me spread the word about my new website.

Note: this guide is meant for programmers who are familiar with APIs

Recently I was talking to someone who has built a cross-bot-framework conversation designer – like an abstraction layer which sits on top of all the major bot frameworks and provides drag-and-drop interfaces to design your bot.

It is an interesting idea, but there is a major problem – all the bot frameworks are not the same. Dialogflow ES, Dialogflow CX, Watson Assistant, Amazon Lex, Microsoft LUIS, RASA NLU – all of them are very different from each other. In fact, they are so different from each other that any effort to unify them to build a higher level abstraction layer will suffer from a problem – you either need to choose the lowest common denominator, or you need to twist one bot framework’s methodology to suit the other.

As I was trying to explain why I don’t particularly prefer that approach, I got a comment on my YouTube channel.

And then I realized that this is the common thread which ties the whole idea together: all chatbot frameworks are built on top of “primitives” – sort of the “building blocks” of the bot framework. And primitives are not just theoretical concepts – without well designed primitives, I would say it would be hard to provide an API for the bot framework.

Which brings us to the method:

Study the API provided by the bot framework.

This gives you another way to answer the Dialogflow ES vs Dialogflow CX vs Microsoft Bot Framework vs IBM Watson Assistant vs Amazon Lex as RASA NLU question.

Dialogflow ES

When you design the API for your bot framework, you have to expose your bot framework’s “primitives” to your API’s end user. In fact, many APIs are designed using some kind of namespacing – here are Dialogflow ES’s API calls:

The namespaces give you an insight into the bot framework’s primitives

If you ignore the projects and agents namespaces, which are basically common to all API calls, the remaining namespaces calls give us a lot of insight into the building blocks of Dialogflow.

Let us expand the intents:

So these are API calls which are associated with intents. And the namespaces explain the “primitives” which are exposed by Dialogflow ES.

Dialogflow CX

Now, look at the sharp contrast with Dialogflow CX:

Dialogflow CX primitives are very different from Dialogflow ES

It is easy to see that this is a pretty major overhaul. Once again, ignore the common prefix projects.locations.agents which doesn’t give you much insight.

You now have the following primitives:

  • flows
  • pages
  • transitionRouteGroups

There are others, but these influence the behavior of your agent a lot more.

And as it turns out, Dialogflow CX is the first bot framework which incorporates an actual visual state machine builder. And the flows, pages and transitionRouteGroups are the primitives you need to build out a visual state machine. (How good or robust these primitives are – we will find out over time).

So let us do a quick tour of the APIs of the other major bot frameworks for comparison.

Amazon Lex

As I expected, Lex doesn’t even have namespaces to make it clear what the primitives are. Which means the API was probably designed as an afterthought and slapped on to “stuff which works”.

Watson Assistant

I am not sure if this is all of Watson Assistant’s REST API. If it is, that means they don’t allow the user to programmatically create and modify agents. (Can someone confirm this?).

Watson Assistant API doesn’t seem to have methods to create/update intents

Microsoft LUIS

Given their very long history of creating developer tools, designing APIs are like bread and butter for Microsoft. So it is no surprise that their API is much more organized and complete than Watson Assistant and Lex. But it is also very different than Dialogflow’s building blocks, and exposes very different primitives.

RASA NLU

RASA is an entirely different beast from all the other bot frameworks since it is open source. As a result, their APIs are also a lot more low-level – that is, it is stuff which would make sense only for people who already have a background in Natural Language Understanding (NLU), for example “pipeline components”.

This is also why you need an entirely different type of programmer if you wish to use RASA NLU when compared to the other bot frameworks. For the others, a web programmer with some training in basic NLU would be able to do the job. For RASA NLU, you need someone who can actually delve into stuff like this (recent videos from their YouTube channel):

RASA is more low-level than other bot frameworks

The obvious benefit: you can do pretty much whatever you want with RASA NLU.

The obvious downside: your programmer has to implement a big chunk of it, because the “primitives” are too low-level.

Summary

Since the time I started in Dialogflow, it has become a much, much larger subject. And so have all the competing bot frameworks.

So I doubt if there is a person who knows multiple bot frameworks and knows them quite deeply.

So clearly you need to spend some time doing your own research. Analyzing the APIs provided by these bot frameworks gives you a shortcut to see how these frameworks are built. In turn, I think it will also help you build much better chatbots no matter which framework you choose.

Similar Posts