Keep a list of inputs and a predefined schema, and ask the LLM to extract the structured data and measure how well it does For a sufficiently complex schema (which you will use in this course), and reasonably long text, you will be surprised at how often even the best LLMs fail to produce complete…
Here is a simple code example to get started System instruction Input text Pydantic Schema
This is the Pydantic schema used in the course. As you can see, this is a very complex schema which is used to extract structured data from a VAERS report. The complexity of this schema acts as a very good test for the quality of an LLM.
This is the system instruction class This provides one file where you can modify your system instructions to see which instruction gives you the best results
This is the input class In the get_input_text method, the input text is formatted to a) split it into sentences b) prepend a sentence number to each sentence This formatting does matter – in fact it matters quite a lot – when you are extracting structured output, since the sentence numbers can be used to…
This method lets you call the OpenRouter API, pass in a specific Pydantic schema, and get a response which should conform to the schema that you sent. There is a lot more going on in this code than what is the absolute minimum necessary, so let me focus on the main part This is the…
Which LLM to choose? How to decide on the schema? How to engineer the prompt to get the most out of your investment?
This is the basic idea Declare a Pydantic class like this The prompt will look like this Call the LLM And then you will do a bunch of post processing to check if the response is valid JSON, and whether it conforms to the provided Pydantic schema etc (with GPT5 both of these are usually…
Last updated 20 Aug 2025 Here is something you can do using GPT5 Given this input clinical narrative (sentence number prefixes were programmatically added): [1] VAERS ID: 190064 (history) [2] Form: Version 1.0 Age[3] : 1.33 Sex: Male Location: Georgia Vaccinated: 1999-06-30Onset: 0000-00-00Submitted: 2004-03-08Entered: 2002-09-12 Days after submission: 544VaccinÂation / ManuÂfacturer Lot / Dose Site…
Compare the visualizations of two outputs to see which one is better Use AI to build the tools to speed up the process
This is the most important part of the Prompt Engineering workflow You need to build a tool which allows you to easily visualize these citations. Here is an example. The citations are clickable. And clicking on them will jump to that specific sentence number in the top input box.
Add fields into the schema which asks for citation numbers for all the sentences which are relevant to the information which was extracted For example consider these classes And this is how it can be used By asking for citation numbers you can see which sentence from the input text has the information, and this…