About

I am shifting focus for this website from Dialogflow to Large Language Models like Google Gemini.

Recent articles

  • How to add thinking budget to Google Gemini Flash 2.5
    You can add a “thinking budget” to make Google Gemini Flash 2.5 spend some time and resources thinking about the answer it provided. This usually provides a better answer than if you do not enable the thinking budget, and can be quite handy for tasks which do require some thinking. Python code for adding thinking…
  • Extracting senior care status from VAERS using Gemini Flash
    This task will extract information about whether a VAERS death report was for someone who was in a nursing home or in some kind of senior care. died due to cardiac arrest; This is a spontaneous report received from a contactable physician via regulatory authority with Regulatory authority report number CH-SM-2021-10701. A 94-year-old female patient…
  • How much has Google Gemini improved over the last year?
    Sundar Pichai mentioned a few interesting things in a recent interview on the All In Podcast About a year back, Google Gemini was simply not very good when compared to ChatGPT. But the LLM competition has now intensified, and you can see that Gemini has improved tremendously when compared to a year back. And this…
  • Gemini 2 Flash vs Gemini 2.5 Flash Preview for structured data extraction
    Click here to learn about the system I used for this comparison Model Accuracy Comparison Model Accuracy Comparison Comparison gemini-2.0-flash gemini-2.5-flash-preview-05-20 Age 100 100 Onset Date 100 100 Vaccination Date 100 100 Number of Vaccine Doses 100 100 Vaccine Manufacturer 100 100 Vaccine Lot Number 100 100 Had COVID-19 100 100 Was Healthy 66 99…
  • Gemini 2 Flash Lite vs Gemini 2.5 Flash Preview accuracy for structured data extraction
    Click here to learn about the system I used for this comparison Model Accuracy Comparison Model Accuracy Comparison Comparison gemini-2.0-flash-lite gemini-2.5-flash-preview-05-20 Age 100 100 Onset Date 98 100 Vaccination Date 100 100 Number of Vaccine Doses 100 100 Vaccine Manufacturer 100 100 Vaccine Lot Number 100 100 Had COVID-19 100 100 Was Healthy 90 99…
  • Gemini 1.5 Pro vs Gemini 2.5 Flash Preview accuracy for structured data extraction
    Click here to learn about the system I used for this comparison Model Accuracy Comparison Model Accuracy Comparison Comparison gemini-1.5-pro gemini-2.5-flash-preview-05-20 Age 100 100 Onset Date 100 100 Vaccination Date 100 100 Number of Vaccine Doses 100 100 Vaccine Manufacturer 100 100 Vaccine Lot Number 100 100 Had COVID-19 100 100 Was Healthy 87 99…
  • Gemini 1.5 Flash vs Gemini 2.5 Flash Accuracy for Structured Data Extraction
    Click here to learn more about the system I have used to do this calculation Model Accuracy Comparison Model Accuracy Comparison Comparison gemini-1.5-flash gemini-2.5-flash-preview-05-20 Age 99 100 Onset Date 99 100 Vaccination Date 100 100 Number of Vaccine Doses 100 100 Vaccine Manufacturer 100 100 Vaccine Lot Number 100 100 Had COVID-19 100 100 Was…
  • LLM eval: A case study using multiple Gemini models
    In my Udemy course, I build a system where I use four different Gemini models to provide an example of LLM eval – where an LLM is a judge for whether or not an answer is correct. Basic idea: This system can be extended to any dataset and to any LLM that you want to…
  • How to extract health status from VAERS using Google Gemini
    This task is to extract whether the patient was healthy before taking the vaccine In the example below patient_was_healthy will be False, because they did have a medical history. Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of…
  • How to extract number of vaccine doses from VAERS report using Google Gemini
    This task is to extract the number of COVID19 vaccine doses a patient has taken Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of v20100815 and v20100816, and also received information via System. A 47-year-old female received first…
  • How to extract vaccine lot from VAERS report using Google Gemini
    This task involves extracting the vaccine lot from VAERS Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of v20100815 and v20100816, and also received information via System. A 47-year-old female received first dose of bnt162b2 (COMIRNATY, lot number:…
  • How to extract vaccine manufacturer from VAERS using Google Gemini
    This task is to extract the vaccine manufacturer – for example Pfizer, Moderna etc – from a VAERS report. For example, in the report below, we should extract Pfizer (which made Comirnaty) Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority…
  • How to extract reporter status (is HCP and is contactable) from VAERS using Google Gemini
    This task is to find out of the person who filed the VAERS report (the reporter) was a healthcare professional. For example in the report below, the answer should be “Yes” since it was filed by a physician. Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received…
  • Extracting most recent vaccination date from VAERS using Google Gemini
    This task involves extracting the date of the most recent vaccination from a VAERS report Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of v20100815 and v20100816, and also received information via System. A 47-year-old female received first…
  • Extracting COVID19 infection status from VAERS using Google Gemini
    In this task, we want to know if the patient ever had a documented COVID19 infection Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of v20100815 and v20100816, and also received information via System. A 47-year-old female received…
  • Calculate the accuracy of all the LLMs using the gold dataset
    Once you create the gold dataset, you can do the following: Once you create a copy of the majority vote CSV file which includes all corrections, it becomes your gold dataset. Now you can use this gold dataset to calculate the accuracy of all the LLMs for the given task. Here are the results for…
  • Use an LLM Eval tool to create a “gold” dataset
    I have created a custom LLM eval tool for my example, but you can build a similar one which best matches the dataset you are interested in. This is what my custom eval tool looks like for VAERS reports The Eval tool will help you quickly provide the correct value for cases where there is…
  • Consolidate results from multiple LLMs into a CSV file
    Once you run the same data extraction (for example ONSET_DATE) using multiple Gemini models, you can consolidate all the results into a CSV file. This is what the Python code look like The basic idea is that for each VAERS_ID you will get the corresponding value extracted by each Gemini model and check to see…
  • Why Google Gemini is the best choice for LLM evals (LLM as a judge)
    In a recent article, Simon Willison wrote this In my opinion, Google Gemini models make great choices for LLM evals
  • How to extract AGE from VAERS report using Google Gemini
    We are interested in extracting the AGE information from a VAERS writeup. This must be an integer. Since there isn’t anything ambiguous about the age, we expect that this will be an easy task for the current generation of LLMs. Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable…
  • How to extract ONSET_DATE from VAERS write-ups using Google Gemini
    The field called ONSET_DATE describes the date of onset for the earliest symptom (after vaccination). Here is an example Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of v20100815 and v20100816, and also received information via System. A…
  • Getting started with Dialogflow ES Messenger
    In this tutorial, I explain how to add a chatbot to your website using the Dialogflow Messenger integration. Create the Dialogflow intents First, create a new Dialogflow agent and add all your intents. Embed the chat widget into your website Once you have created all your intents, you can add it into your website. Embedding…
  • 10 Tips to increase the accuracy of LLM Structured Outputs
    Here are some tips you can use to improve the accuracy of structured outputs from LLM. Note: you can use the Python instructor library to get structured outputs from nearly any Large Language Model. Use a more descriptive field name You can improve the accuracy of structured information extraction by providing a more descriptive field…
  • Is Dialogflow still relevant in the era of Large Language Models?
    Last updated: 17 Jan 2025 Till early 2023, Dialogflow was probably the best framework for developing chatbots. Consider these three major changes after the wide adoption of Large Language Models like ChatGPT: 1 Chatbots are now being integrated with search – that is, they do not make a lot of sense as standalone apps. Google…
  • How to handoff to live agent in Dialogflow ES
  • How to use Google Sheets with your Dialogflow ES bot
  • How to create a Quiz Bot in Dialogflow ES
  • How to create a Chatfuel style decision tree chatbot in Dialogflow
  • How to manage context from your Python webhook in Dialogflow ES
  • How to get a user’s birthdate in DialogFlow ES
  • How to use list and composite entities in Dialogflow ES
  • How to use webhook for slot filling in Dialogflow ES
  • How to confirm or update user input in Dialogflow ES
  • How to use slot filling in Dialogflow ES
  • How to save user input to an Airtable database in Dialogflow ES
  • Get your DialogFlow agent to initiate the conversation before user types a message
    First published: Aug 2017 | Last Updated: May 2022 So let us understand the question: When you work with Dialogflow you notice that it follows the “user types something -> agent replies with an answer” sequence. Suppose you wish to get the agent to say something before the user types anything, how do you do it? Example…
  • Naming conventions for Dialogflow ES intents and contexts
    Do you capture user input in your chatbot? For example, do you ask them to provide their name, or email, or other information? I have recently started using the following convention in the bots I am building (as well as advising on). Suppose you have an intent which should get user’s email address. You can,…
  • How to mimic Dialogflow CX session variables in Dialogflow ES
    In the previous article I explained how you can mimic the behavior of follow up intents to move to the next intent. One disadvantage of this approach is that you cannot access the information collected from 2 or more steps prior in the current response. The way around this is to use the concept of session variables. What…
  • How to collect user input without follow up intents in Dialogflow ES
    Sometimes I see questions like these in the Dialogflow forum. So, I got one intent working. How can I now “move” the conversation to the next intent? Answer In Intent1, declare an output context called contextA. In the next intent you want to fire, use contextA as the input context. You have now chained Intent1 and Intent2 using contextA. Is…
  • How to use follow up intents to collect user input in Dialogflow ES
    In this article, I will explain how to use follow up intents to collect user input in Dialogflow ES. Let us take the example of a simple lead capture chatbot. It captures the details of visitors who are interested in adding a chatbot to their website. We would like to get the following inputs: Here is how…
  • Dialogflow Python webhook tutorial
    One of the things that I have mentioned before is that you need to have a programmer on your team if you would like to build a non-trivial Dialogflow bot. At the same time, all Dialogflow bot makers would still benefit from understanding the basics of webhooks. This tutorial will be helpful if you are technical, but not…
  • How to export and import Dialogflow ES agent ZIP file
    Dialogflow ES provides the option to export your entire Dialogflow ES agent as a ZIP file. How to Export Dialogflow ES agent Go to Settings by clicking on the Gear icon next to the name of the agent. Choose “Export and Import” tab and click on the “Export as ZIP” button to export the agent…
  • Export large Dialogflow ES FAQ bot to Dialogflow CX
    I got this question from a BotFlo app user. There is a CSV to agent ZIP file converter in the BotFlo app, and this question relates to that tool: Could one export from ES to CX specifying the route group as the destination in CX. Unlike Dialogflow ES which provides an agent ZIP file with…
  • Slot filling using webhook in Dialogflow CX
    I got this question about the Learn Dialogflow CX course I am using slot filling to collect some data, example name, subjects,marks. After collecting data I need to do some logic to them. where should i enable the webhook to get all the parameters that i have collected in cx?  Let us take the example of…
  • What is the “Tag” field in a Dialogflow CX webhook?
    I got this question from a student I want to create a wehbook for my bot. I followed your instructions from https://botflo.com/how-to-debug-dialogflow-python-webhook-using-ngrok/ and completed all of the stages.However, step 14 differs from the tutorial. I have a “Tag” to fill and I’m not sure what to place in there. The “Tag” field is similar to the Action…
  • How to replicate contexts in Dialogflow CX?
    I got this question on my YouTube channel: I have a question about the context equivalent in CX. In ES, there was a concept of contexts. How can we achieve the same in CX? I tried one way with using condition routes with page transitions and it worked in a way. But are there other…
  • Updating Dialogflow CX parameters from webhook
    This article is a follow up to the chapter on Webhooks in my Learn Dialogflow CX course. You first need to go through those videos to be able to understand this tutorial. The main thing that you need to do is understand the exact JSON format to use so that you can add, update or…
  • ChatGPT vs Dialogflow CX
    I have created a chapter in my Udemy course to explain this topic. Why I created this course (0:00 – 0:19)So, I want to get started by explaining why I created this course. I first started publishing this course to my audience when Dialogflow CX was still in beta and this was sometime in 2021….
  • How to quickly find the differences between two Dialogflow agent ZIP files
    Managing Large Dialogflow ES Bots I have built a tool to help you do this. Steps Go to the Unpack agent ZIP file tool in BotFlo (you must be logged in to use the tool). Suppose you want to find the difference between an older version of your Dialogflow agent and the current version. First…
  • How to search across a large Dialogflow ES bot
    Managing Large Dialogflow ES Bots Have a lot of intents in your Dialogflow agent? Are you not clear exactly which intent was matched by looking at the response? Given that you can just take the agent ZIP file and unzip the file and do a search inside the files, you might be wondering why not…