Technical integrity
Decode success, corruption, resolution, duration, audio intelligibility, schema conformance, checksum verification, and exact or near-duplicate rates.
Quality that can be challenged
A score can summarize. It cannot replace evidence.
Every published revision should carry a versioned quality report with a method, denominator, threshold, result, evidence time, and implementation version for each rule. Unknown remains unknown. Rights remain a separate fail-closed gate.
Six dimensions a buyer can inspect
Human context
Coverage of purpose, initial state, cause, action sequence, intervention, measurement, uncertainty, immediate outcome, and delayed outcome.
Annotation evidence
Contributor-confirmed versus machine-suggested fields, reviewer tier, agreement, evidence anchors, contradiction checks, and audit pass rate.
Dataset coverage
Task, environment, contributor, device, language, time, success/failure, rare-case, and follow-up distributions—with missingness shown by subgroup.
Provenance
Collection and selection method, source representation, transformations, verification events, timestamps, code versions, and known chain-of-custody gaps.
Rights compatibility
Purpose-by-purpose permission, ownership basis, subject or third-party evidence, sensitive-class review, expiry, geography, withdrawal, and prohibited-use gates.
Every metric needs a receipt
| Field | Example meaning |
|---|---|
| Method | Exact calculation, sampling procedure, or review rubric |
| Population and denominator | What was examined, excluded, missing, or sampled |
| Threshold | The acceptance rule fixed before finalization |
| Result and confidence | Measured value plus uncertainty where appropriate |
| Evidence time | When the source and result were current |
| Code and policy version | A reproducible implementation and decision rule |
Standards informing the design
Dataset Cards and Datasheets inform transparent documentation; Schema.org and MLCommons Croissant inform machine-readable discovery and loading; finalized revisions follow the dataset–revision–asset separation used by established data exchanges. These standards improve interoperability but do not prove quality or legal compliance.