sdvm-fix-1

TextThinking

sdvm-fix-1 reads an audited sample and returns it fixed, with a fix: changes, flagged when something could not be resolved, attempts. A sample never comes back worse than it arrived. Pass it what sdvm-audit-1 returned. A sample without an audit is audited first.

import json
import os

from openai import OpenAI

client = OpenAI(
    base_url="https://api.sdvm.ai/v1",
    api_key=os.environ["SDVM_API_KEY"],
)

audited = client.chat.completions.create(
    model="sdvm/audit-1",
    messages=[{"role": "user", "content": json.dumps({"data": samples})}],
)

fixed = client.chat.completions.create(
    model="sdvm/fix-1",
    messages=[{"role": "user", "content": audited.choices[0].message.content}],
)
for sample in json.loads(fixed.choices[0].message.content):
    print(sample["fix"])

With the SDK, Fixer(api_key=...).run(audited) attaches it to each sample’s .fix. [s for s in fixed if s.fix["flagged"]] is the review queue.

A real request and response

The audited sample from sdvm-audit-1, as returned.

{
  "data": [
    {
      "task_type": "multiple_choice",
      "style": "continuation",
      "context": "A woman sits at a bathroom mirror with a tube of mascara. She unscrews the wand and leans in close to the glass. she",
      "choices": [
        "drives off.",
        "exits.",
        "carefully applies the mascara to her upper and lower lashes, blinking between coats.",
        "mixes a bowl."
      ],
      "answer_index": 2,
      "audit": {
        "num_choices": 4,
        "has_multiple_choices": true,
        "context_non_empty": true,
        "choices_non_empty": true,
        "valid_answer_index": true,
        "no_length_shortcut": false,
        "no_mojibake": true,
        "no_pii": true,
        "ctx_grammatical": false,
        "ctx_coherent": true,
        "ctx_fluent": true,
        "ctx_complete": null,
        "choice_grammatical": true,
        "choice_coherent": false,
        "choice_fluent": true,
        "choice_complete": true,
        "choice_defects": [0, 3],
        "label_correct": true,
        "single_valid_answer": true,
        "distractors_discriminating": true
      }
    }
  ]
}

{
  "fixed_data": [
    {
      "task_type": "multiple_choice",
      "style": "continuation",
      "context": "A woman sits at a bathroom mirror with a tube of mascara. She unscrews the wand and leans in close to the glass. She",
      "choices": [
        "scratches the fogged glass with her fingernail to clear a small patch.",
        "checks her phone reflection to ensure the lighting looks flattering for her selfie.",
        "carefully applies the mascara to her upper and lower lashes, blinking between coats.",
        "adjusts the curlers in her hair while staring intently at her reflection."
      ],
      "answer_index": 2,
      "audit": {
        "num_choices": 4,
        "has_multiple_choices": true,
        "context_non_empty": true,
        "choices_non_empty": true,
        "valid_answer_index": true,
        "no_length_shortcut": false,
        "no_mojibake": true,
        "no_pii": true,
        "ctx_grammatical": false,
        "ctx_coherent": true,
        "ctx_fluent": true,
        "ctx_complete": null,
        "choice_grammatical": true,
        "choice_coherent": false,
        "choice_fluent": true,
        "choice_complete": true,
        "choice_defects": [0, 3],
        "label_correct": true,
        "single_valid_answer": true,
        "distractors_discriminating": true
      },
      "fix": {
        "changes": [
          "ctx: Capitalized 'she' to 'She'",
          "Rewrote choice 0 to describe a plausible mirror-related action that contradicts applying mascara.",
          "Rewrote choice 1 to describe a plausible phone-related action that contradicts applying mascara.",
          "Rewrote choice 3 to describe a plausible hair-related action that contradicts applying mascara."
        ],
        "flagged": false,
        "attempts": 1
      }
    }
  ],
  "original_count": 1,
  "fixed_count": 1,
  "cost_usd": 0.01,
  "input_tokens": 1088,
  "output_tokens": 4023
}

ctx: the lowercase she was capitalised. The wording of the stem is unchanged.
The distractors were rewritten to the length of the correct choice, which is untouched.
audit is the one that came in, so nothing was audited again. sdvm-fix-1 does not re-audit; Refine does.

Options

int

default:"1"

Tries at an accepted fix, 1 to 5. A flagged sample is tried again on the original with what went wrong as feedback, stopping at the first accepted fix. On FixerConfig; on the HTTP API, config: {"max_attempts": 3}.

str | None

default:"None"

Dataset conventions the model must not read as defects. See Dataset conventions.

run takes a list of samples, up to 100 per request.

from sdvm import Fixer, FixerConfig

fixer = Fixer(
    api_key=os.environ["SDVM_API_KEY"],
    config=FixerConfig(max_attempts=3),
)
fixed = fixer.run(audited)

Get started

Sample types

Models

Guides

SDK

HTTP API

Pricing and limits

A real request and response

Options

​A real request and response

​Options

A real request and response

Options