How to score an answer with its prompt (client.evaluate)

Call client.evaluate({ responseText, prompt }). It sends both to POST /api/evaluate, which scores the answer with the question that produced it.

Last updated

Steps

  1. Send the answer and the prompt

    const { verdict } = await client.evaluate({
  prompt: 'Where is the Eiffel Tower?',
  responseText: 'The Eiffel Tower is definitely in Berlin.',
});

  2. Tag the call if you like

    await client.evaluate({
  prompt,
  responseText,
  modelId: process.env.OPENAI_MODEL,
  sessionId: 'thread-123',
});

Limits

responseText must be 10 to 100,000 characters. Shorter returns 400 Content is too short to score (CONTENT_TOO_SHORT). Longer returns 400 Content too long (CONTENT_TOO_LONG).

score or evaluate?

Use evaluate when you have the prompt. See Why send the prompt with the answer (evaluate vs score).