Ask a question to get started
Enter to send•Shift+Enter new line
StringEvaluator()
RunEvaluator
BaseModel
Evaluate an example asynchronously.
Recursively generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
Generates a JSON string representing this model as it would be received from or sent to the API (but with indentation).
Usage docs: https://docs.pydantic.dev/2.4/concepts/serialization/#modelmodel_dump
Usage docs: https://docs.pydantic.dev/2.4/concepts/serialization/#modelmodel_dump_json
The name evaluation, such as 'Accuracy' or 'Salience'.
'Accuracy'
'Salience'
The key in the run inputs to extract the input string.
The key in the run outputs to extra the prediction string.
The key in the example outputs the answer string.
Function that grades the run output against the example output.
The single key this evaluator emits, when its name is set.
Evaluate a single run.
Grades the run's string input, output, and optional answer.
.. deprecated:: 0.5.0
StringEvaluator is deprecated. Use openevals instead: https://github.com/langchain-ai/openevals
