LLMs vs NLP libraries

People will prefer LLMs over NLP libraries like spaCy for three reasons

  • they can deal with shorthand and typos much more easily
    • probably because of their “grounding”
  • they can do basic reasoning (which is getting better by the day) even as they extract structured data
  • with more open source LLMs the problems of privacy are becoming less of an issue
    • but they probably won’t go away completely, which is why there is probably a market for an enhanced version of spaCy which can run on old machines and provide elementary LLM-like capabilities for extracting structured output