An assistant that can say "I don't know" is the whole product

Every language model will answer every question. That is the problem. Ask a raw model about your delivery terms and it will produce something plausible, fluent, and possibly wrong. In a demo nobody checks. In production, a wrong price quoted to a customer is a refund, a dispute, or a lost account.

The fix is not a smarter model. It is retrieval grounding: the assistant answers only from your documents, quotes what it found, and carries a confidence score. Below the threshold, it does the single most valuable thing a machine can do in customer service: it says it is not sure and brings in a human.

In our deployments the assistant has three modes. Grounded answer: the response cites a specific document, price list, or policy. Clarify: the question is answerable but ambiguous, so the assistant asks one narrowing question. Escalate: confidence is low or the topic is on the restricted list (prices above a limit, medical advice, legal obligations), so the conversation is handed to a person with a summary attached.

Customers do not punish the escalation. They punish confident nonsense. A handoff that arrives with context reads as service; a wrong answer reads as a scam.

Ask what the assistant does when it does not know. If the answer involves the model's general knowledge, keep looking. The honest architecture is boring: your documents, source references, thresholds, and an escalation path. Boring is what you want answering your customers at 2 a.m.