When NOT to use GPT

As I mentioned in the previous lesson, GPT is very well suited for extractive question answering.

So you should use ChatGPT for such tasks, and it will very likely perform much better than other options in those scenarios.

But there are also some situations where you should not use GPT.

Reasoning OUTSIDE the code layer

An example of this is to use ChatGPT for answering questions about the Dialogflow Conversation Layer.

Dialogflow has four layers, and one of them – the conversation layer – does not involve writing any code. In fact, it is one of the strongest points of Dialogflow ES, and the conversation layer has become even better in Dialogflow CX with the introduction of System Functions.

I expect LLMs will eventually be able to do this sooner rather than later and be able to reason about and answer such questions, but they are not there yet.

Using it at the wrong level of abstraction

An example of a Wrong Abstraction is using the GPT API to create a task oriented multi-turn conversation chatbot with multiple entity types and complex state management. (Click here for an example of such a bot).

The abstraction that the folks at OpenAI are suggesting (turn EVERYTHING into a prompt!) makes very little sense, and it will be extremely hard for you to build your bot quickly. And even after you build it out, it will be very hard to test your bot for accuracy and to debug your bot.

There is a reason why tools like Dialogflow have such a strong internal consistency between the features they offer and the APIs they expose, as this approach makes bot frameworks like Dialogflow not just easy-to-use but also easy-to-automate.

By changing the level of abstraction to “Just use this LLM and hope and pray for the best”, the OpenAI API makes it very hard for you to understand and manipulate the right abstraction layer.

Repetitive, modular NLP tasks

This is not directly related to chatbots, but I have added this here because it is a common mistake people make when they start using GPT. They think GPT is the answer to all Natural Language Processing (NLP) tasks.

An example is to use the GPT API for repetitive heavy duty NLP tasks like Named Entity Recognition.

It fails on so many fronts that it is a very good example of LLM Maximalism (using LLMs for use cases where they don’t make much sense).

This talk by spaCy founder Ines Montani explains why.