AI evaluation framework: nếu không đo được, bạn chưa thật sự có AI product

Vì sao cần evaluation?

"It works on my laptop" không phải evidence. AI product cần metrics rõ ràng:

5 metrics cần track

1. Accuracy / Correctness

• Factual accuracy (kiểm tra output vs ground truth)
• Hallucination rate (bao nhiêu % output bịa)

2. Consistency

• Same input → same quality output? (temperature = 0)
• Edge cases handled gracefully?

3. Latency

• Time to first token (TTFT)
• Total response time
• User-perceived delay

4. Cost

• Tokens per request (input + output)
• Cost per user session
• Monthly burn rate projection

5. User satisfaction

• Task completion rate
• User feedback/ratings
• Retention/engagement metrics

Tools

• **Braintrust** — LLM eval platform
• **LangSmith** — Tracing + eval by LangChain
• **Promptfoo** — Open-source prompt testing
• **Custom**: Python script + spreadsheet vẫn tốt cho start