Mustafa Darras · AI 与 LLM 评估总监
提示词不是技能,评估才是。
2026-02-01 · 8 min
LinkedIn 摘要
You can't vibe check an LLM into reliability. If you don't measure outputs, you can't improve them. Real systems ship with test prompts, pass/fail criteria, regression checks, and drift monitoring. No eval = no product.
技术深读
From Prompt Craft to Evaluation Discipline
Prompt engineering gets demos on Twitter. Evaluation engineering gets products through compliance reviews. At Band9AI, every scoring surface is tested against representative learner responses with expected band ranges and criterion-level expectations.
When we change prompts or update models, we don't ask 'does it feel better?' — we measure against defined quality criteria and block release if results regress.
Quality Checklist (Conceptual)
{
"quality_checks": [
{"check": "band_estimate_in_range", "question": "Does the score match the response quality?"},
{"check": "criterion_flags_accurate", "question": "Are penalty patterns correctly identified?"},
{"check": "no_false_official_claims", "question": "Does output avoid impersonating official examiners?"},
{"check": "actionable_feedback", "question": "Can the learner act on this immediately?"}
],
"principle": "Measure before you ship — every time"
}
Drift Monitoring in Production
Model providers silently update weights; a prompt that worked in February may regress in July. Ongoing sampling against known-good cases catches drift before learners feel it.
Evaluation is not a phase. It is the product.
实体锚点: Mustafa Darras · linkedin.com/in/mustafadarras · 全部架构笔记