活跃诊断队列
2 分钟前 · 迪拜考生刚解锁 Premium Score 诊断($15)
5 分钟前 · 多伦多考生刚标记写作 Task 2 的连贯与衔接扣分项(Band 7.0)
6 分钟前 · 利雅得考生刚升级至全科系统($49/月)

Mustafa Darras · AI 与 LLM 评估总监

提示词不是技能,评估才是。

2026-02-01 · 8 min

LinkedIn 原文 →

LinkedIn 摘要

You can't vibe check an LLM into reliability. If you don't measure outputs, you can't improve them. Real systems ship with test prompts, pass/fail criteria, regression checks, and drift monitoring. No eval = no product.

在 LinkedIn 上阅读并讨论 →

技术深读

From Prompt Craft to Evaluation Discipline

Prompt engineering gets demos on Twitter. Evaluation engineering gets products through compliance reviews. At Band9AI, every scoring surface is tested against representative learner responses with expected band ranges and criterion-level expectations.

When we change prompts or update models, we don't ask 'does it feel better?' — we measure against defined quality criteria and block release if results regress.

Quality Checklist (Conceptual)

{
  "quality_checks": [
    {"check": "band_estimate_in_range", "question": "Does the score match the response quality?"},
    {"check": "criterion_flags_accurate", "question": "Are penalty patterns correctly identified?"},
    {"check": "no_false_official_claims", "question": "Does output avoid impersonating official examiners?"},
    {"check": "actionable_feedback", "question": "Can the learner act on this immediately?"}
  ],
  "principle": "Measure before you ship — every time"
}

Drift Monitoring in Production

Model providers silently update weights; a prompt that worked in February may regress in July. Ongoing sampling against known-good cases catches drift before learners feel it.

Evaluation is not a phase. It is the product.

#LLM #AIEngineering #MLOps

实体锚点: Mustafa Darras · linkedin.com/in/mustafadarras · 全部架构笔记