NLP for Business: Transformer Use Cases and How to Ship Them

Most business value from NLP comes from five jobs: classifying and routing text, extracting fields from documents, searching by meaning, analyzing sentiment and themes in feedback, and summarizing long material. Transformers made all five practical with little labeled data. The engineering decision today is which kind of model does each job: a large language model with structured output when volume is modest or the task needs judgment, or a small encoder model or embeddings plus a simple classifier when volume is high and latency, cost and auditability matter.

Key takeaways

• Map every NLP idea to one of five jobs: classify, extract, search, analyze feedback, summarize. Each has a known best default.
• Use LLMs to prototype and to pre-label data. Serve high-volume tasks with small models and keep the LLM as the fallback for uncertain cases.
• Validate every extraction against a schema and business rules, and send failures to a review queue instead of the system of record.
• Evaluate per class on a test set held out by time, and treat confidence thresholds as a product decision about coverage versus precision.
• Most production failures come from taxonomy drift, truncation, label noise and silent prompt or model changes, not from the architecture.

What transformers changed

Before transformers, useful NLP meant hand-built features or recurrent networks that read text one word at a time and struggled with long-range context. The transformer architecture, introduced in the 2017 paper "Attention Is All You Need", uses self-attention to relate every token to every other token in parallel. That made it practical to pretrain very large models on general text and then adapt them to a specific task with a small amount of labeled data.

Three families matter for business work:

Pick the right tool for each task

The pattern that saves the most money

The most cost-effective production pattern we know is simple: use an LLM to prototype the task and pre-label data, have people correct the labels, train a small model on the corrected set, and route only the small model's low-confidence cases back to the LLM or to a person. You get LLM-level quality on the hard cases and small-model cost on the bulk of the traffic. When a small model is not enough but a frontier LLM is too expensive at your volume, fine-tuning a mid-sized LLM is the next step.

Classification and routing

Ticket routing, email triage, document type detection, content moderation, lead categorization: classification is the workhorse of business NLP. For clear categories with a few hundred examples each, embeddings plus logistic regression is a strong, boring baseline that is hard to beat on cost.

"""ticket_router.py: sentence embeddings plus a linear classifier, with a confidence floor."""
import json
 
import numpy as np
from sentence_transformers import SentenceTransformer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
 
encoder = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")  # example; compare a few
 
 
def load(path: str):
    rows = [json.loads(line) for line in open(path, encoding="utf-8")]
    return [r["text"] for r in rows], [r["label"] for r in rows]
 
 
train_text, train_y = load("train.jsonl")
test_text, test_y = load("test.jsonl")  # held out by time, not by random row
 
X_train = encoder.encode(train_text, normalize_embeddings=True, batch_size=64)
X_test = encoder.encode(test_text, normalize_embeddings=True, batch_size=64)
 
clf = LogisticRegression(max_iter=2000, class_weight="balanced")
clf.fit(X_train, train_y)
print(classification_report(test_y, clf.predict(X_test), zero_division=0))
 
CONFIDENCE_FLOOR = 0.7  # choose on validation data: coverage versus precision
 
 
def route(text: str) -> dict:
    probs = clf.predict_proba(encoder.encode([text], normalize_embeddings=True))[0]
    best = int(np.argmax(probs))
    confidence = float(probs[best])
    if confidence < CONFIDENCE_FLOOR:
        return {"label": None, "action": "llm_or_human_review", "confidence": confidence}
    return {"label": str(clf.classes_[best]), "action": "auto_route", "confidence": confidence}

Why this beats an LLM call for high-volume routing:

• Cost and latency. Encoding a short text with a small embedding model takes milliseconds on a CPU. There is no per-token bill.
• Retraining in seconds. When the taxonomy changes, relabel, refit, redeploy. No prompt archaeology.
• Probabilities you can threshold. The confidence floor turns a classifier into a router: confident cases go straight through, uncertain ones go to an LLM or a person.
• Auditability. The same input gives the same output, and a misroute can be traced to its training examples.

Two cautions. Do not read these probabilities as true frequencies: regularization and class_weight="balanced" both shift them, so pick the threshold from a validation set by plotting precision against coverage, not by intuition. And per-class results matter more than overall accuracy: a model can be right on most tickets overall while missing nearly every rare "legal threat" ticket, which is the class you most needed.

Document extraction

Invoices, purchase orders, contracts, claims, resumes, lab reports: extraction turns documents into rows in a system of record. The modern pipeline:

1. Get clean text and layout. Digital PDFs give text directly. Scans need OCR; vision-capable LLMs can also read page images directly. Keep page images and coordinates for reviewers.
2. Extract with a schema. Ask the model for structured output matching a strict schema, including a short source quote for important fields so reviewers can verify quickly.
3. Validate with business rules. Types, formats, allowed values, and cross-field checks such as totals that must add up.
4. Route failures to review. Anything that fails validation or has low confidence goes to a human queue, never straight into the ledger.
5. Learn from corrections. Reviewer corrections become evaluation cases and, eventually, training data.

The validation layer is where reliability comes from:

from datetime import date
from decimal import Decimal
 
from pydantic import BaseModel, ValidationError, field_validator, model_validator
 
 
class LineItem(BaseModel):
    description: str
    quantity: Decimal
    unit_price: Decimal
    amount: Decimal
 
 
class Invoice(BaseModel):
    supplier_name: str
    invoice_number: str
    invoice_date: date
    currency: str
    line_items: list[LineItem]
    tax: Decimal = Decimal("0")
    total: Decimal
 
    @field_validator("currency")
    @classmethod
    def known_currency(cls, value: str) -> str:
        if value not in {"USD", "EUR", "GBP", "INR"}:
            raise ValueError(f"unexpected currency {value}")
        return value
 
    @model_validator(mode="after")
    def totals_add_up(self):
        subtotal = sum((item.amount for item in self.line_items), Decimal("0"))
        if abs(subtotal + self.tax - self.total) > Decimal("0.01"):
            raise ValueError(f"lines {subtotal} plus tax {self.tax} do not match total {self.total}")
        return self
 
 
def accept_or_review(llm_json: str, review_queue) -> "Invoice | None":
    try:
        return Invoice.model_validate_json(llm_json)
    except ValidationError as exc:
        review_queue.put({"raw": llm_json, "errors": exc.errors()})
        return None

The model will occasionally read a 7 as a 1 or miss a line. The totals check catches a large class of those errors for free, and every failure becomes a test case.

Search and question answering

Keyword search fails when users and documents use different words: "cancel my plan" versus "subscription termination". Embedding-based semantic search fixes that, and hybrid search keeps exact matches for codes and names working. Our vector database guide covers embeddings and indexes, and the RAG architecture guide covers turning search results into cited answers, so we will not repeat them here.

Sentiment and voice of the customer

A single positive or negative score per review rarely changes a decision. "Love the product, the export is broken, support was slow" averages to neutral and tells you nothing. Aspect-based sentiment does: it records what the customer felt about which part of the experience.

A practical pipeline:

1. Extract aspects with structured output. For each piece of feedback, return a list of aspect, sentiment and the supporting quote. Fix the aspect list to your product areas (onboarding, billing, export, performance, support) plus an "other" bucket.
2. Discover what your list misses. Embed the "other" quotes and cluster them. New clusters are new aspects or new problems. BERTopic and plain k-means on embeddings both work; have an LLM propose names for clusters and a person confirm them.
3. Calibrate against people. Have your own team label a few hundred items, including sarcasm, domain slang and every language you receive. Measure agreement between people first: that is the ceiling for any model.
4. Report trends with examples. Show aspect sentiment over time with the underlying quotes one click away. Teams act on quotes, not on decimals.

Known weak spots: sarcasm, negation across sentences, domain words that flip meaning ("sick" in gaming feedback), and languages or dialects that are underrepresented in the model's training data. That last one is also a fairness problem when sentiment drives decisions about customers or staff.

Summarization

Summarization is now an LLM job almost by default. The risks are omission (the one important caveat left out) and unsupported claims (a detail that is not in the source). Ways to control both:

• Structured summaries. For meetings and calls, ask for decisions, action items, owners and dates as fields, not prose. Missing fields are visible.
• Cite sections. Ask for the source section or timestamp behind each point, so readers can verify in seconds.
• Summarize long material in stages. Summarize sections, then summarize the summaries, keeping section references throughout.
• Check faithfulness. For high-stakes documents, a second pass that checks each summary sentence against the source catches most invented details.
• Fix the length per use case. "Three bullets for the executive, one paragraph per section for the analyst" beats "be concise".

Running NLP in production

• Latency and cost. Batch encoder inference, cache embeddings for repeated texts, quantize models for CPU serving, and export to ONNX Runtime where it helps. Send to LLMs only the share of traffic that needs them.
• Evaluation. Keep a labeled test set drawn from recent production data, report per-class precision and recall and a confusion matrix, and re-run it on every model, prompt or data change.
• Drift monitoring. Watch the distribution of predicted labels and of confidence scores. A sudden shift usually means the input changed: a new product, a new form, a new customer segment.
• Relabeling cadence. Sample production items every month, label them, and fold them into training and test sets.

The general machinery for this, from versioning to monitoring and retraining pipelines, is covered in our MLOps guide.

What goes wrong in production

• Taxonomy drift. A new product launches and its tickets get confidently routed into the closest old category. Watch confidence distributions and the size of the low-confidence queue, and add classes deliberately.
• Label noise. Two annotators disagree on a sizable share of items and the model learns the disagreement. Write labeling guidelines with examples and measure agreement.
• Rare classes hidden by averages. Overall accuracy looks great while recall on the important rare class is poor. Report per class, and oversample or separately detect the rare ones.
• Silent truncation. The original BERT-family models stop at 512 tokens and many pipelines truncate without warning. The deciding sentence in a long email is often at the bottom. Chunk long inputs or use a longer-context model.
• OCR errors upstream. Extraction quality cannot exceed the text it receives. Track OCR confidence and route bad pages to review.
• LLM labels that move. A prompt tweak or provider model update shifts labels across thousands of items with no error raised. Pin model versions, and run the regression set before any change.
• Personal data in training sets and logs. Feedback and tickets are full of names, emails and account details. Redact before storing and before training.
• Overtrusted scores. A 0.92 confidence is not a 92 percent chance of being right unless you have calibrated it. Choose thresholds from validation data.

Frequently Asked Questions

What are the main business applications of NLP?

The applications that reliably pay off are classifying and routing text such as tickets and emails, extracting fields from documents such as invoices and contracts, searching internal content by meaning, analyzing sentiment and themes in customer feedback, and summarizing long documents, calls and threads. Most other NLP use cases are combinations of these five.

Do you still need traditional NLP models now that LLMs exist?

Yes, for high-volume, well-defined tasks. A small encoder model or embeddings plus a simple classifier can run on a CPU in milliseconds, costs very little per item and is easy to audit and retrain. LLMs are better for prototyping, low-volume work, tasks that need judgment, and as a fallback for the cases a small model is unsure about.

What is the difference between NLP, transformers and LLMs?

NLP is the field of getting computers to work with human language. Transformers are the neural network architecture, based on self-attention, that most modern NLP models use. Large language models are very large transformer models trained to generate text, while smaller encoder transformers are typically used for classification, entity extraction and embeddings.

How much labeled data do you need to train a text classifier?

With a pretrained embedding model or encoder, a few dozen examples per class can produce a usable baseline, and a few hundred per class usually gets you to a production-quality model for clear categories. Rare but important classes need deliberate collection. Consistent labeling guidelines matter more than volume.

Is BERT still useful?

BERT-style encoder models are still widely used for classification, named entity recognition and embeddings because they are small, fast and cheap to fine-tune. The original BERT models accept at most 512 tokens, so long documents must be chunked, and newer encoder models support much longer inputs. For open-ended generation and summarization, decoder LLMs have replaced them.

How do you do accurate sentiment analysis on customer feedback?

Use aspect-based sentiment, which records what the customer felt about which part of the product, instead of one score per message. Check the model against a sample labeled by your own team, including sarcasm, domain slang and every language you receive. Then act on trends per aspect over time and read real examples, rather than reacting to a single average score.

Need help putting NLP into production?

We build NLP systems that run every day on real document and ticket volumes: classification, extraction, search and summarization with the evaluation, review queues and monitoring that keep them accurate. Senior engineers only, and we stay on the pager for what we ship. If we built it, we're on the pager.

• AI and machine learning services
• Talk to an engineer