BERT & RoBERTa

What it is

Encoder-only transformers that produce rich contextual embeddings. Decade-defining models — and still the right choice when you need fast classification, NER or QA without the cost of a generative LLM.

How Vaaani uses it

• Fast intent classification on user queries (10ms inference)
• Question answering over a fixed corpus (extractive, not generative)
• Embedding generation for semantic search backends
• Fine-tuned NER for domain-specific entities (drug names, contract terms)

Why it makes the cut

When you don't need a generator and you do need 10x lower latency, BERT-family models still win. Most Vaaani classifiers are fine-tuned BERTs.

Sample code

from transformers import AutoTokenizer, AutoModel

tok = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModel.from_pretrained("bert-base-uncased")

inputs = tok("Vaaani ships AI workers", return_tensors="pt")
hidden = model(**inputs).last_hidden_state