langchain.smith.evaluation.string_run_evaluator.LLMStringRunMapper

class langchain.smith.evaluation.string_run_evaluator.LLMStringRunMapper

Bases: StringRunMapper

Extract items to evaluate from the run object.

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

__call__(run: Run) → Dict[str, str]

Maps the Run to a dictionary.

map(run: Run) → Dict[str, str]

Maps the Run to a dictionary.

serialize_chat_messages(messages: List[Dict]) → str

Extract the input messages from the run.

serialize_inputs(inputs: Dict) → str

serialize_outputs(outputs: Dict) → str

to_json() → Union[SerializedConstructor, SerializedNotImplemented]

to_json_not_implemented() → SerializedNotImplemented

property lc_attributes: Dict

Return a list of attribute names that should be included in the serialized kwargs. These attributes must be accepted by the constructor.

property lc_namespace: List[str]

Return the namespace of the langchain object. eg. [“langchain”, “llms”, “openai”]

property lc_secrets: Dict[str, str]

Return a map of constructor argument names to secret ids. eg. {“openai_api_key”: “OPENAI_API_KEY”}

property lc_serializable: bool

Return whether or not the class is serializable.

property output_keys: List[str]

The keys to extract from the run.

model Config

Bases: object

extra = 'ignore'