When NOT to use GPT API for your chatbot

On the bottom of this website, there is a demo Dialogflow CX bot. It is a good example of a chatbot which would be very hard to build using the GPT API, and much easier to build using Dialogflow CX.

The bot allows you to choose between GPT API and Dialogflow CX for your bot.

What about the Assistants API?

While it is still early days, even people on the OpenAI forum are quite clear that the technology is still too unpolished for production use, especially for no-code and low-code scenarios.

Why Dialogflow CX is a much better choice for task-oriented bots

A task oriented bot is one which guides the user through multiple conversation turns to help them achieve a task.

This is in contrast to pure FAQ bots which just answer questions and usually don’t have to answer followup questions.

Here are some reasons why Dialogflow CX is much better suited for task-oriented bots

To summarize, there are a lot of benefits you get from using Dialogflow CX instead of the GPT API for task oriented chatbots, especially if you are not a programmer

How to decide if you need to use Dialogflow CX for your bot

In my view, the simplest way to decide if you need Dialogflow CX is to

a) write out 5-10 conversation scripts (these should be idealized conversation flows, also called the “happy path” in chatbot terminology)

b) convert them into a Mindomo flowchart (this allows you to visualize the conversation flow without putting too much effort into learning Dialogflow CX).

How to use Mindomo to design your Dialogflow chatbot

c) if your flowchart is multiple levels deep, I would recommend using Dialogflow CX. If it is wide and shallow (which usually means some kind of an FAQ bot), the GPT API might be a better choice

An example

Here is the Mindomo flowchart which I use when the user clicks on the “Choose between GPT and Dialogflow” button in the demo bot:

Note: The red border indicates a fallback response, while the green border indicates an end-of-conversation response.

In this case, these are the idealized conversation scripts:

Script 1Script 2Script 3Script 4
User: <triggers the start of the flow>
Bot: What kind of bot are you building?
User: faq bot
Bot: Do you have your entire FAQ in a single page?
User: yes
Bot: use the GPT API
User: <triggers the start of the flow>
Bot: What kind of bot are you building?
User: faq bot
Bot: Do you have your entire FAQ in a single page?
User: no
Bot: What is the source of your FAQ (e.g. knowledge base, website etc)?
User: Knowledge Base
Bot: Use Dialogflow CX with custom data stores
User: <triggers the start of the flow>
Bot: What kind of bot are you building?
User: faq bot
Bot: Do you have your entire FAQ in a single page?
User: no
Bot: What is the source of your FAQ (e.g. knowledge base, website etc)?
User: website
Bot: Use GPT API and retrieval augmented generation
User: <triggers the start of the flow>
Bot: What kind of bot are you building?
User: faq bot
Bot: Do you have your entire FAQ in a single page?
User: no
Bot: What is the source of your FAQ (e.g. knowledge base, website etc)?
User: something else
Bot: Create a stateful generative prompt in the fallback response based on user’s request as well as current state

There are actually quite a lot of takeaways from this fairly simple flowchart:

  • I have a flowchart which does have multiple followup conversation turns, and maintaining the conversation state properly for such a bot using the GPT API is very hard
    • in theory, the Assistants API can help with this, but it is extremely code heavy and not really suited for non-programmers
  • Dialogflow CX now has generative features, which means the fallback response does not have to be “sorry, I don’t know the answer to that”. Instead, you can ask Dialogflow CX itself to provide a reasonable response based on user’s requirements by constructing a suitable prompt
  • You can construct stateful generative prompts – these are prompts where you take the current state of the conversation (see Script 4) and ask Dialogflow CX to provide a suitable response
  • You can easily turn fallback responses into their own intents (provided it makes sense). Just turn Dialogflow CX’s response into an intent.
    • In case you are wondering what would be the benefit of doing that, it is because you now have a way to measure all the conversations where the user provided that specific type of input
  • Simpler conversation analytics. Everything in Dialogflow CX is already state-based, so it is much easier to filter conversation flows based on user intents.
  • It is much easier to measure and improve the accuracy of your Dialogflow CX bot because you actually know the bot’s exact response in all the non-generative scenarios


About this website

BotFlo1 was created by Aravind Mohanoor as a website which provided training and tools for non-programmers who were2 building Dialogflow chatbots.

This website has now expanded into other topics in Natural Language Processing, including the recent Large Language Models (GPT etc.) with a special focus on helping non-programmers identify and use the right tool for their specific NLP task.

1 BotFlo was previously called MiningBusinessData. That is why you see that name in many videos

2 And still are building Dialogflow chatbots. Dialogflow ES first evolved into Dialogflow CX, and Dialogflow CX itself evolved to add Generative AI features in mid-2023