Quickstart

Prerequisites

Python 3.12 or newer.
An SDVM account. Create one at sdvm.ai and add credits on your profile page.

Installation

pip install sdvm

uv add sdvm

To run the dataset walkthroughs in examples/, add the extra: pip install "sdvm[examples]".

Get an API key

Create a key on your profile page. It is shown once, so store it right away. The examples read it from an environment variable.

export SDVM_API_KEY=sdvm_...

Keep keys out of source control. Every client accepts api_key= directly, but reading it from the environment is the pattern used throughout these docs.

Fix your first samples

Wrap each string as a TextSample, pass the list to Fixer.run, and get fixed samples back.

import os

from sdvm import Fixer
from sdvm.types import TextSample

fixer = Fixer(api_key=os.environ["SDVM_API_KEY"])

fixed = fixer.run([
    TextSample(text="she sels sea shells by the sea shor"),
    TextSample(text="i wanna no what da weather is gonna b like tmrw in nyc"),
])
for sample in fixed:
    print(sample.text)

# she sells sea shells by the sea shore
# i wanna know what da weather is gonna b like tmrw in nyc

sels, shor and no were corrected because they are genuine misspellings, while the missing capitals, wanna, da, b and tmrw stayed: they are how the author writes, not mistakes. See what a fix will not do. Up to 100 samples go in one request.

Audit before you fix

For multiple-choice data, run the Auditor first. Each sample comes back with its verdicts on .audit, and Auditor.aggregate gives a local, dataset-level view.

from sdvm import Auditor
from sdvm.types import MultipleChoiceQuestionAnswerSample

samples = [
    MultipleChoiceQuestionAnswerSample(
        context="What is the capital of France?",
        choices=["Berlin", "Paris", "Madrid", "Rome"],
        answer_index=1,
    ),
]

audited = Auditor(api_key=os.environ["SDVM_API_KEY"]).run(samples)
print(audited[0].audit)
# {'label_correct': True, 'single_valid_answer': True, 'distractors_discriminating': True, ...}

print(Auditor.aggregate(audited)["answer_position"])

Run the whole loop

Refinery runs sdvm-audit-1, sdvm-fix-1 and sdvm-audit-1 again in one call. Each sample comes back with .audit (the verdict before the fix), .fix (what changed) and .reaudit (the verdict after it, which tells you whether the fix helped).

from sdvm import AuditorConfig, Refinery

refined = Refinery(api_key=os.environ["SDVM_API_KEY"], audit=AuditorConfig(votes=3)).run(samples)
for sample in refined:
    print(sample.audit, sample.fix, sample.reaudit)

flagged = [s for s in refined if s.fix["flagged"]]  # your review queue

Next steps

Core concepts

What the audit, fix and reaudit blocks mean.

Multiple choice

Completion stems versus questions, and why the type matters.

Dataset conventions

Stop the audit flagging formatting that is a convention, not a defect.

Denoising with votes

Majority-vote the verdicts where one has to be stable.

Error handling

The exception hierarchy and what each status code means.

Examples

HellaSwag and MMLU walkthroughs that ship with the SDK.

Get started

Sample types

Models

Guides

SDK

HTTP API

Pricing and limits

Prerequisites

Installation

Get an API key

Fix your first samples

Audit before you fix

Run the whole loop

Next steps

Core concepts

Multiple choice

Dataset conventions

Denoising with votes

Error handling

Examples

​Prerequisites

​Installation

​Get an API key

​Fix your first samples

​Audit before you fix

​Run the whole loop

​Next steps

Core concepts

Multiple choice

Dataset conventions

Denoising with votes

Error handling

Examples

Prerequisites

Installation

Get an API key

Fix your first samples

Audit before you fix

Run the whole loop

Next steps