Prompting Isn't the Skill. Evaluation Is.

From Prompt Craft to Evaluation Discipline

Prompt engineering gets demos on Twitter. Evaluation engineering gets products through compliance reviews. At Band9AI, every scoring surface is tested against representative learner responses with expected band ranges and criterion-level expectations.

When we change prompts or update models, we don't ask 'does it feel better?' — we measure against defined quality criteria and block release if results regress.

Quality Checklist (Conceptual)

{
  "quality_checks": [
    {"check": "band_estimate_in_range", "question": "Does the score match the response quality?"},
    {"check": "criterion_flags_accurate", "question": "Are penalty patterns correctly identified?"},
    {"check": "no_false_official_claims", "question": "Does output avoid impersonating official examiners?"},
    {"check": "actionable_feedback", "question": "Can the learner act on this immediately?"}
  ],
  "principle": "Measure before you ship — every time"
}

Drift Monitoring in Production

Model providers silently update weights; a prompt that worked in February may regress in July. Ongoing sampling against known-good cases catches drift before learners feel it.

Evaluation is not a phase. It is the product.