langchain.evaluation.schema.StringEvaluator¶

class langchain.evaluation.schema.StringEvaluator[source]¶

Bases: _EvalArgsMixin, ABC

Grade, tag, or otherwise evaluate predictions relative to their inputs and/or reference labels.

Methods

__init__()

aevaluate_strings(*, prediction[, ...])

Asynchronously evaluate Chain or LLM output, based on optional input and label.

evaluate_strings(*, prediction[, reference, ...])

Evaluate Chain or LLM output, based on optional input and label.

Attributes

evaluation_name

The name of the evaluation.

requires_input

Whether this evaluator requires an input string.

requires_reference

Whether this evaluator requires a reference label.
async aevaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict[source]¶

Asynchronously evaluate Chain or LLM output, based on optional input and label.

Parameters

  • prediction (str) – The LLM or chain prediction to evaluate.

  • reference (Optional[str], optional) – The reference label to evaluate against.

  • input (Optional[str], optional) – The input to consider during evaluation.

  • **kwargs – Additional keyword arguments, including callbacks, tags, etc.

Returns

The evaluation results containing the score or value.

Return type

dict

evaluate_strings(*, prediction: str, reference: Optional[str] = None, input: Optional[str] = None, **kwargs: Any) → dict[source]¶

Evaluate Chain or LLM output, based on optional input and label.

Parameters

  • prediction (str) – The LLM or chain prediction to evaluate.

  • reference (Optional[str], optional) – The reference label to evaluate against.

  • input (Optional[str], optional) – The input to consider during evaluation.

  • **kwargs – Additional keyword arguments, including callbacks, tags, etc.

Returns

The evaluation results containing the score or value.

Return type

dict

property evaluation_name: str¶

The name of the evaluation.

property requires_input: bool¶

Whether this evaluator requires an input string.

property requires_reference: bool¶

Whether this evaluator requires a reference label.

Examples using StringEvaluator¶