On the bottom left of this page, there is a Dialogflow ES chatbot which integrates with the GPT API.
If you tell it what kind of tasks your bot should do, it will give you a recommendation – whether to use Dialogflow CX, Dialogflow ES and GPT API.
It is also a demo of a fine-tuned GPT-3.5-turbo model which answers the user’s question if the Dialogflow ES bot goes to the Fallback intent.
Here is how it works:
- if the ES bot can identify the user’s task, it will provide a recommendation based on the existing intents in the bot (so this will not call the GPT API)
- if the ES bot cannot identify the user’s task, it will go to the Default Fallback Intent. The Default Fallback Intent calls the GPT API where I have created a fine-tuned model, and this model returns the best recommendation based on user input
- Dialogflow ES then relays the response from the GPT API to the user
Is the GPT Finetuning API too slow for practical chatbots?
Using regular prompts based on GPT-3.5-Turbo is usually too slow for practical chatbot use cases.
Fine-tuned models are supposed to be much faster than regular prompting. In fact that is one of the reasons mentioned in the GPT documentation for using fine-tuning.
From what I have seen, the GPT fine-tuning API is too slow (that is, the latency is too high) for practical chatbots.
In other words, when I check the Raw Interaction Log inside the Dialogflow ES console I see this output:
In other words, the webhook code took more than 5 seconds to send back its response, and as a result the webhook call itself timed out.
You can try it and see for yourself. I will also be monitoring the results and keep updating this article every few weeks.