Keep on top of recent developments in “reasoning”

You can run this script for each LLM on OpenRouter and get a good idea of how things are evolving in terms of LLM reasoning

I use this approach in this course to evaluate the ability of many different LLMs to extract structured data (so you can just get this course if you don’t want to spend the time and effort)

LLMs vary widely in their ability to reason about structured data extraction

Sometimes even the best LLMs do make mistakes – remember, there is still a lot of hype around AI