What to do when an answer gets a low ClearRun score

Treat a low score as a flag. The answer's stated confidence is ahead of the support in its reasoning. Then choose one next step: retry, ask for support, or send it to a person.

Last updated

ClearRun does not block or change the answer. You decide what happens next.

Steps

  1. Look at the issues

    The verdict lists the issues it found and a short summary. Start there.

  2. Retry with a better prompt

    Ask the model to give its reasons, name its sources, or say what it is unsure about. Then score the new answer.

  3. Ask for a hedge

    If the answer can't be supported, ask the model to state it with less certainty.

  4. Send it to a person

    For answers that matter, route low scores to someone who can check the facts.

What a low score tells you

The answer's confidence is not backed by what it shows. It may state things as certain without reasons, give exact numbers with no source, or contradict itself.

It does not tell you the answer is false. A low-scoring answer can be right. A high-scoring answer can be wrong.

Set the rule in your own code

const { verdict } = await client.score({ responseText: answer });
if (verdict.score == null || verdict.score < 70) {
  // retry, or send to a person
}

The number 70 is an example. See How to choose a ClearRun score threshold.

Keep your own fact checks

ClearRun does not check facts. If being right matters, check the claims against sources too.