LLM Evals for Structured Outputs

What the stats look like inside Marimo

Example of report where NUMDAYS is available Example of report where NUMDAYS is empty

Three CSV files for each year DATA CSV files has most of the relevant information about the patient, and also include a clinical narrative (free form text) called SYMPTOM_TEXT. Notice that it can hold up to 32000 characters. You will sometimes see very long reports, especially from foreign VAERS. SYMPTOMS CSV files describes a list…

You can see both mentioned in the supported parameters in OpenRouter model search page What is the difference? Response format makes a best effort to return valid JSON. In other words, you can expect JSON.parse to not throw an error. Structured Outputs, on the other hand, is a subset of response_format. It makes a best…