Dataset conventions - SDVM

Some datasets have formatting that looks like a defect but is a deliberate convention: uniform lowercasing, markup tokens, contexts truncated by design. Without being told, the quality audit flags such formatting as a grammatical problem. conventions is a free-text description that tells the audit what to leave alone.

import os

from sdvm import Auditor, Fixer

WIKIHOW = (
    "Bracketed markers like [header] [title] [step] are section labels, not errors; the "
    "text is uniformly lowercased, do not flag lowercasing; the context may end "
    "mid-sentence by design, do not flag it as incomplete."
)

audited = Auditor(api_key=os.environ["SDVM_API_KEY"]).run(samples, conventions=WIKIHOW)
fixed = Fixer(api_key=os.environ["SDVM_API_KEY"]).run(audited, conventions=WIKIHOW)

On HellaSwag’s wikiHow text this takes the grammatical false-alarm rate from roughly 40% down to about 12%.

Where it applies

Auditor.run(..., conventions=...) for the quality audit.
Fixer.run(..., conventions=...) for the routing audit, so a convention does not trigger a needless fix.
Refinery(conventions=...) forwards it to every audit and fix stage.

Writing a good description

Be self-contained: the text is the only context the audit gets about your dataset.
Name the convention and say what not to flag, as in the example above.
Keep it to the formatting. Conventions are not a place to change what counts as a wrong answer.

Last modified on September 18, 2026