ChatGPT vs Dialogflow CX

I have created a chapter in my Udemy course to explain this topic.

Why I created this course

(0:00 - 0:19)
So, I want to get started by explaining why I created this course. I first started publishing this course to my audience when Dialogflow CX was still in beta and this was sometime in 2021. In fact, you can even see this if you look at the video watermarks in all the videos.

(0:20 - 2:21)
The first set of videos will use my old website URL and the second set of videos will use the more recent website URL, okay. So, that is pretty much that gives you an indication of how long this course has been in development and I kept adding material over time as Dialogflow CX evolved and I kept doing it till about end of 2023, okay. And meantime, the large language models like ChatGPT, they were getting better and better and then I saw that people started believing that the GPT bots can do everything that Dialogflow CX can do and actually that is not true. 

But what I will say is that the use cases for Dialogflow CX have now gone down a lot given that the LLMs have entered the picture, okay and that does make sense because Dialogflow also had some limitations, right and LLMs are pretty good at filling those gaps, okay. And what I have done with this course is I am trying to explain the trade-offs, when it makes sense to choose LLM bots and when it makes sense to choose Dialogflow CX and I also want to point out that there actually there are some trade-offs. It is not still as of this recording which is October 2024, it is still not like a given that you will be able to solve every problem using large language models. 

I still, it could happen, I am not sure, but I still do not see a few things which I would expect that it should have been reasonably well solved given that this is now more than a year and a half since ChatGPT entered the picture. So, in other words, companies like OpenAI have had about a year and a half to solve the problems which Dialogflow CX was solving very efficiently even before the, even before ChatGPT entered the picture.

How Dialogflow CX is better than LLM bots

(0:00 - 1:47)
Now let us talk about how Dialogflow CX is better than bots built using large language models okay. So the first thing that is very evident when you use Dialogflow CX for a while is that it is much easier for non-programmers to design intentsBoth Dialogflow ES and Dialogflow CX have the concept of int... More. Now this course is going to teach you what intents are and it is going to teach you all the Dialogflow CX concepts but it is kind of very basic. 

The intent is what is the user trying to do that is what is captured by an intent and you will see as you design your bot that the Dialogflow CX has a very structured way of designing intents. It is coming from the viewpoint that you know what the user is likely to ask and you also know approximately what words they are likely to use okay. When you know these two things then it is actually easier to use Dialogflow CX to design your intents and you will see why that this is a better approach in those circumstances okay. 

And Dialogflow CX is also it makes it much easier to specify entity definitions and to monitor entity extraction and again this is also a basic concept in chatbots. EntitiesBoth Dialogflow ES and Dialogflow CX support entities and th... More are you can think of them as representing proper nouns that is you know things like names and places and all these things which have you want to get that information to be able to get the complete picture of what the user is asking okay. It is a sort of like think of it as concepts.

(1:48 - 6:22)
Now the even though it has been almost two years since ChatGPT came into the picture what I have seen is that it still does a pretty mediocre job at this task of entity extraction and there are actually lot of reasons for that and one of the reasons is that it is not designed that is ChatGPT is not designed to for the structured NLP tasks okay and in fact I will go so far as to say that you often feel like it is force fitting the structured NLP into the prompt you know we have to use prompts for getting ChatGPT to work. So it often feels like we are force fitting structured NLP into the prompt based paradigm okay that is exactly what it feels like when you use large language models for the task of entity extraction and this leads to the third point which is that Dialogflow CX it makes it much easier to manage state and if you want to build a conversational flow managing state is actually quite important and the way ChatGPT manages state is what we can say it is completely non-deterministic and in fact it is not even a comparison which you can make in any reasonable way because when you use Dialogflow CX what you are quite literally saying is that I expect the conversation to move from state A to state B where you know both A and B. In the case of ChatGPT it is more like a guess where you allow the LLM based agent to infer the state right it is not even a defined state they are inferred and you can go back and look at the conversation transcript and see if everything is fine but it does not change the fact that you probably still do not know or you cannot name the state which was being used when the conversation was happening in the case of ChatGPT. Dialogflow CX is built as a conversational state machine the state machine is a very important concept in Dialogflow CX in fact I spend a lot of time covering the state machine because without understanding it you just cannot use Dialogflow CX you have to understand state machines to be able to understand CX and there are also a few other things it is easier to simpler to update the underlying model now it is not done in a very transparent way in Dialogflow CX I will be very honest about that but what is going on under the hood is a sort of a fine tuning of the machine learning model when you use Dialogflow CX and you give it a lot of training phrases and then all these entities and you define states and then when you look at the users interaction and then you go back and add more training phrases into your intents and when you do all these things that is the task of monitoring your bots behavior and updating it accordingly what you are really doing under the hood is you are fine tuning your the machine learning model to your domain specific requirements so this is all being done with a very nice UI in the case of Dialogflow CX which is one more big point of differentiation from chat GPT like interfaces and you also have access to test cases that you can run to see if your bot is still running as you were expecting it to and you can also manage and monitor the conversation history quite easily because Dialogflow CX is built as a tool which is mainly for non-programmers okay so you the more complex the bot the more you need help from a programmer you need to have a programmer in your team but the conversation itself can be designed by non-programmers so that is one big differentiation or it is a big way that Dialogflow CX differs from chat GPT based bots okay so this is how Dialogflow CX is better than using chat GPT and similar large language models to build your chatbot

How LLM Bots are better than Dialogflow CX

(0:00 - 0:56)
Okay, so based on my previous video, you might have come to some kind of an understanding that Dialogflow CX is much better for certain tasks and the thing that I sort of did not mention there is that most of the time those tasks are not what people are building chatbots for, okay. So, that is why LLM bots are, you know, they are really taking the world by storm and that is why they are being used in so many use cases because LLMs, the LLM based bots are fundamentally different from the structured NLP bots that you build using Dialogflow CX, okay. Now, the first thing which should be pretty clear to anyone who has used Dialogflow for any period of time is that the LLM based bots are just simply far superior for FAQ style questions.

(0:56 - 2:35)
Even when you know what kind of question the user is going to ask, they could ask the same question in hundreds of different ways and one big limitation of Dialogflow CX and other chatbot frameworks is that they were not able to do a good job of anticipating the different variations for these ways that the user would ask the question. In fact, the problem which you would often is that as you kept handling more and more variations for these phases, they would start interfering with other intents that you have defined in your bot. Now, I am not going to go into that in too much detail, that is a complex topic all by itself, but let us just say that by doing better training and increasing the number of training phases and all that, you are not always guaranteed to improve your chatbot in Dialogflow CX. 

In some cases, it would actually get more worse. It could get worse because it had more intents to conflict with what you just added, that is the training phase you added. So, in this way, LLM bots were much better than Dialogflow CX. 

Even when they got started, even at the beginning of 2023, I was noticing that LLM bots, they were able to answer questions, understand the questions more easily. And the other important difference in these large language model based bots is that they already have world knowledge. So, they know a lot of things about the topics which are not being defined inside your bot.

(2:35 - 3:03)
And this is what I refer to as world knowledge, everything which is the universe which lies outside your bot's definition. And they can answer questions that you did not even know that the user was interested in. Now, that may not be like a requirement for you, but if it ever happens, you will find that the users are generally pretty satisfied and glad that the bot is able to answer something like an out of syllabus question.

(3:04 - 3:51)
And the other thing which is sort of tied to the first point in this video is that generative responses can span multiple intents. Now, when you create bots using chatbot frameworks like Dialogflow CX and even other frameworks, you have this problem where if the user asks a question which can only be answered by combining two intents, two or more intents together, then the chatbots will usually just completely fail in the case of the structured NLP bots. But in the case of generative AI bots, they are able to generate responses which can span multiple intents, that is they can.

(3:51 - 4:18)
Now, they do not even have the concept of intents in the case of LLM bots, I want to be clear about that. This is just a comparison where you imagine that they had two intents and you can see that they could combine the information in those two intents together and generate a coherent response. Now, the problem with this and why this is not always a good thing is because these bots are prone to, the large language models are prone to hallucinations.

(4:19 - 5:00)
And hallucinations are things where, in fact, this is when the most common situation is that these hallucinations happen when they try to combine multiple intents and provide answers because they combine things in ways which do not actually make sense to humans. That is why it is called a hallucination, where it is a sort of a combination answer which does not make sense to humans. The answer will be grammatically correct, it might sometimes even seem to be a bit plausible, but if you know anything at all about your domain, you will note that the answer is just flat out wrong and that is because the GPT bot is hallucinating.

(5:00 - 6:39)
Just something to remember. And the other way that LLM bots are very good is they are very good at inferring high level phrases which begin conversations. Now, I forgot to add that some of these high level phrases could be very short and the short words and short phrases are extremely challenging to handle if you are using the structured NLP bots, the frameworks, the old chatbot frameworks which were like building structured NLP bots, they used to struggle a lot on extremely short conversation starter questions. 

The problem is, as you might imagine, is that just the way those bots work is it is based on looking at a pattern of words which are together, it is looking at a pattern of words together and the shorter the sentence, the fewer the words, the harder it is for your bot to infer what the user is saying. So, this used to be a big problem with the structured NLP bots and the LLM bots have, I will say that they almost solved the problem. And again, they have a large community of users now, given that they have become so popular, there are a lot of people using it and which means that there is a lot of documentation available for these kind of bots. 

And then the LLM bots, there is a rapid pace of innovation, things are improving on a daily basis or a weekly basis and definitely on a monthly basis and you can see that even when some things did not work as well as you might imagine at the beginning, over time they get much better.

When to use Dialogflow CX

(0:00 - 1:35)
Alright, so let us talk about when to use Dialogflow CX instead of GPT-based bots or LLM-based bots. When you know the states in your conversation and you want to manage those states, Dialogflow CX is still a far better choice because it allows non-programmers to visualize those states and also see how the conversation flows between multiple states, okay. This is very difficult to do in LLM-based bots. 

The other use case where you might prefer to choose Dialogflow CX is for bots which make heavy use of structured entities. And when I say structured entities, what I mean is the entity has a data type, for example, a date, okay, and you want the information which is extracted to be parsed as a date, which means that you should be able to access the day and the month and the year and things like that. And this should all be possible just using the chatbot frameworks, you know, the features that it provides out of the box, right. 

So, in the case of having these kind of structured entities where you have to get the correct value in the correct data type to be able to perform the next actionYou define an action in Dialogflow ES intents to tell your w... More or the next step, in those cases, you will find that Dialogflow CX is still superior to using LLM bots. Now, I can actually go into this in a lot more detail, probably not here, but you know, it could be a part of a different course. But getting structured information using the LLM bots is actually a pretty big challenge.

(1:35 - 2:56)
And I will also say that it is being addressed now, it is improving quite rapidly, the ability to do that is improving quite rapidly. But it is still not a very well understood field on the whole, especially if you have like nested and very complex entities where it is like two or three different properties combined to form the larger entity. A date is a really good example, because a date has three parts, right, it has to have the day, it has to have the month, it has to have the year. 

And we know that the year is an integer. We also know that the date is some number between 1 and 31, right. And then you know that the month is a number between 1 and 12. 

But sometimes you need access to it in the form of a string, like you want to get the name of the month rather than the number, right. So, these are all like, what I am saying is, this is just an entity, it is called date, but it is complex and it has sub attributes, which you need to know and you need to be able to extract it correctly to be able to do the next step, especially dates, which if you are using dates in your bots, you have to make sure that you get the correct value, right. So, for those kind of scenarios, you will see that Dialogflow CX is still better.

(2:57 - 4:33)
And if you have a good idea of the majority of questions that the user might ask, you might still prefer to use the Dialogflow CX bots, especially if it ties well with the other requirements that you already have, ok. Now, this might be a, you know, when you have a bot, which is, you know, deployed to your intranet, which is like internal facing, it is not facing the public. This is a good example of a very structured environment. 

It is a controlled environment rather, but the requirements are also like more, it is more similar to what you will have for structured NLP bots, right, because the user are employees. So, you kind of expect them to ask questions in a certain way or at least you can train them to ask questions in a certain way and that will make it easier for you to not only build your bot, but also go and look at what the users are interested in, so that you can keep improving the bot over time and do it in a more structured way, right. So, these are the kind of use cases where Dialogflow CX still makes sense compared to GPT bots, but as you might have noticed, this is a fairly restrictive set of use cases and you will probably also see that the more common use case for bots is the FAQ style bots, which is also why, you know, if you are building a pure FAQ bots, I do not think that it makes a lot of sense to use Dialogflow anymore and you are probably going to be better off using a GPT based bot.

Integrating LLM bots with Dialogflow CX

(0:00 - 0:30)
Okay, so now we are going to talk about a topic where I have a somewhat strong opinion and I think that it is also kind of different from the opinion of others but I am not very sure about that. So this is, this has to do with integrating the LLM bots into a Dialogflow CX bot, okay. So what are people trying to do when they say that they want to integrate the large language model bots with the Dialogflow CX bots, okay.

(0:31 - 1:14)
They are saying if I simply forward the query which a user has provided to Dialogflow CX, you just forward it to ChatGPT for example or the, you know, the API which is provided by ChatGPT and you use Dialogflow CX essentially as a user-friendly UI wrapper, okay. It is not, it is not, it is not using Dialogflow CX features but it is just using Dialogflow CX's user-friendly, you know, interface, right. And can I build a bot which they think that this is offering the best of both worlds when people come and ask me they believe that this is offering the best of both worlds, okay.

(1:14 - 1:34)
So let us just define the LLM aspect of it as, you know, prompt-based bots and then let us define the Dialogflow CX aspect of it as structured NLP bots. You might have seen that I have referred to these two in this chapter, you know. The reason that I do not recommend this is because, actually there are a few.

(1:34 - 2:05)
The first is that you pay for both the CX response and the LLM response and because these are still fairly small amounts, I think there will be a lot of people who say that they do not mind that as a big objection, they do not think of it as a big objection, okay. And yes, I understand that. But the other thing is that when you combine your CX bot with your LLM bot like this, you actually get very little insight into the entity extraction and the state management.

(2:06 - 3:12)
Given that the big differentiating factor of Dialogflow CX is its ability to manage state as well as its ability to do a good job of entity definition and extraction, I think that this is kind of wasteful to use your, to combine your Dialogflow CX with your LLM bots in this way. And the other reason is that when you are using Dialogflow CX and you use webhooksYou can use webhooks to add custom business logic in both Di... More, they are already pretty code heavy and you need to use webhooks to be able to do this integration which means that your bot becomes even more code heavy if you want to keep up with, let us say, some new model which has been released by GPT and you want to use that instead of your old one. When you make all these updates, you need to, it becomes even more code heavy, correct? And this sort of beats the point of Dialogflow CX being this framework which is most well suited for non-programmers, right? So these are some reasons why I do not think that it is a good idea to combine your CX bot with GPT API.

(3:12 - 3:36)
And generally speaking, I do not even recommend using generative AI tools inside Dialogflow CX for the same reasons. It has its own generative AI built into Dialogflow CX now and I do not recommend using it for pretty much the same reasons. They are very different paradigms, the prompt-based bots and the structured NLP bots and it is probably not a good idea to mix them.

(3:37 - 4:33)
Now I will also say that I keep an open mind on this topic for now and I will change my view if I see some compelling use cases where it makes sense to combine the structured conversational flows that you have in Dialogflow CX with these unstructured generative AI responses. So but as of today which is October 2024 is when I am doing this recording, I still think that it is better to have these paradigms as separate, okay? And the other thing that I will also add is that if you are very interested in doing this, right? If you are very keen on combining the prompt-based and the structured NLP-based aspects, it might even be better if you just start with the vertex AI search, okay? Or vertex AI agents which is not Dialogflow. It has some overlaps but it is not the same thing and the vertex AI stuff is not covered in this course.

(4:33 - 4:56)
So it could be an entirely different course or you might have to go and look at the documentation to understand what is going on there. But what I am trying to say here is that you have to think about your use case and what really you are trying to accomplish. And you might notice that you will be able to do everything using just the vertex AI search and just add like follow-up questions ability.

(4:56 - 5:34)
It is already out of the box in vertex AI. That is vertex AI search already gives you the ability to ask follow-up questions as if it was a conversation, okay? So as I say here, it is an interesting combination of a custom search engine which is a bit like your site-specific search on Google and then you also have these facets which is a bit like Algolia if you have heard of Algolia and it incorporates these chat features like follow-up questions based on the previous search results. So you might want to just directly go into this vertex AI search instead of trying to combine generative AI aspects into your dialogue flow CX parts.

Note: This is my old website and is in maintenance mode. I am publishing new articles only on my new website. 

If you are not sure where to start on my new website, I recommend the following article:

Is Dialogflow still relevant in the era of Large Language Models?