sdvm-audit-1 rates a sample and returns one verdict per check; sdvm-fix-1 fixes it. The Python SDK reaches them through three clients: Auditor, Fixer, and Refinery, which runs both in one call.
Key features
sdvm-audit-1rates per-sample quality: shape, content, grammar, coherence, label correctness, distractor quality and more, one verdict per check.sdvm-fix-1fixes what the audit found, behind a never-worse guarantee: a sample is never made worse than its input.- Refine runs
sdvm-audit-1,sdvm-fix-1andsdvm-audit-1again in one call, so you can see whether the fix helped. - Fix-or-flag contract: anything the fix cannot fully resolve comes back with
flagged=Truefor review instead of a silent partial edit. - Dataset conventions: declare uniform lowercasing, markup tokens or intentional truncation so the audit does not mistake them for defects.
- Voted verdicts:
AuditorConfig(votes=N)audits each item N times and keeps the majority per dimension. - Local aggregate checks such as answer-position bias, computed on your machine without a request.
- Async twins of every client, and a helper to publish results to the Hugging Face Hub.
Where to go next
Quickstart
Install the SDK and fix your first samples in under five minutes.
Core concepts
Samples, the audit, fix and reaudit blocks and the never-worse contract.
Audit
Measure quality per sample and across a whole dataset.
Refine
Audit, fix and re-audit in one server-side call.
SDK reference
Every public class and function, generated from the package.
HTTP API
Call the audit, fix and refine endpoints from any language.