Model Evaluation

Kind: activity

Layer: Correctness Core

Record: architecture:model-evaluation

Severity: mandatory

Scope: model, model-backed feature, pipeline

Canonical: Ontology

The activity of measuring a model against datasets, metrics and slices, and comparing the results with acceptance thresholds.

Listed in Architecture principles, after Model Governance and before Model Inference.

Requires

• Dataset
• Metrics
• Acceptance Criteria

Reinforces

• Model Safety
• Correctness

Enables

• Model Selection/Regression Detection

Conflicts with

• Untested Model Deployment

In tension with

• Metric Completeness

Tensions

• Model Evaluation / Metric Completeness

Violated by

• Untested Model Deployment

Refactored by

• Evaluation Suite

Severity

• mandatory

Category

• Model Architecture

Refactors

• Model-Driven Architecture

Reinforced by

• Artificial Intelligence Architecture
• Prompt Engineering

Required by

• Model Drift Monitoring

Linked from

• Systems built around a model
• Correctness / Determinism / Verification
• Metaprogramming / Language-Oriented Architecture
• Model Architecture
• Core Vocabulary
• Model Architecture
• Severity levels
• The resolutions