Sundar Pichai mentioned a few interesting things in a recent interview on the All In Podcast About a year back, Google Gemini was simply not very good when compared to ChatGPT. But the LLM competition has now intensified, and you can see that Gemini has improved tremendously when compared to a year back. And this…
Click here to learn about the system I used for this comparison Model Accuracy Comparison Model Accuracy Comparison Comparison gemini-2.0-flash gemini-2.5-flash-preview-05-20 Age 100 100 Onset Date 100 100 Vaccination Date 100 100 Number of Vaccine Doses 100 100 Vaccine Manufacturer 100 100 Vaccine Lot Number 100 100 Had COVID-19 100 100 Was Healthy 66 99…
Click here to learn about the system I used for this comparison Model Accuracy Comparison Model Accuracy Comparison Comparison gemini-2.0-flash-lite gemini-2.5-flash-preview-05-20 Age 100 100 Onset Date 98 100 Vaccination Date 100 100 Number of Vaccine Doses 100 100 Vaccine Manufacturer 100 100 Vaccine Lot Number 100 100 Had COVID-19 100 100 Was Healthy 90 99…
Click here to learn about the system I used for this comparison Model Accuracy Comparison Model Accuracy Comparison Comparison gemini-1.5-pro gemini-2.5-flash-preview-05-20 Age 100 100 Onset Date 100 100 Vaccination Date 100 100 Number of Vaccine Doses 100 100 Vaccine Manufacturer 100 100 Vaccine Lot Number 100 100 Had COVID-19 100 100 Was Healthy 87 99…
Click here to learn more about the system I have used to do this calculation Model Accuracy Comparison Model Accuracy Comparison Comparison gemini-1.5-flash gemini-2.5-flash-preview-05-20 Age 99 100 Onset Date 99 100 Vaccination Date 100 100 Number of Vaccine Doses 100 100 Vaccine Manufacturer 100 100 Vaccine Lot Number 100 100 Had COVID-19 100 100 Was…
In my Udemy course, I build a system where I use four different Gemini models to provide an example of LLM eval – where an LLM is a judge for whether or not an answer is correct. Basic idea: This system can be extended to any dataset and to any LLM that you want to…
This task is to extract whether the patient was healthy before taking the vaccine In the example below patient_was_healthy will be False, because they did have a medical history. Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of…
This task is to extract the number of COVID19 vaccine doses a patient has taken Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of v20100815 and v20100816, and also received information via System. A 47-year-old female received first…
This task involves extracting the vaccine lot from VAERS Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of v20100815 and v20100816, and also received information via System. A 47-year-old female received first dose of bnt162b2 (COMIRNATY, lot number:…
This task is to extract the vaccine manufacturer – for example Pfizer, Moderna etc – from a VAERS report. For example, in the report below, we should extract Pfizer (which made Comirnaty) Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority…
This task is to find out of the person who filed the VAERS report (the reporter) was a healthcare professional. For example in the report below, the answer should be “Yes” since it was filed by a physician. Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received…
This task involves extracting the date of the most recent vaccination from a VAERS report Feeling cold; Chills and shivering; high sweating; This is a spontaneous report from a contactable physician, received from the Regulatory Authority, regulatory authority report number of v20100815 and v20100816, and also received information via System. A 47-year-old female received first…