Use embedding distances to score semantic difference between a prediction and reference.

Hierarchy

  • StringEvaluator
    • EmbeddingDistanceEvalChain

Implements

Constructors

Properties

distanceMetric: EmbeddingDistanceType = "cosine"

The distance metric to use for comparing the embeddings.

outputKey: string = "score"
requiresInput: boolean = false
requiresReference: boolean = true
embedding?: EmbeddingsInterface

The embedding objects to vectorize the outputs.

evaluationName?: string = ...

The name of the evaluation.

memory?: BaseMemory
skipInputWarning?: string = ...
skipReferenceWarning?: string = ...

Accessors

Methods

  • Parameters

    • inputs: ChainValues[]
    • Optional config: (RunnableConfig | CallbackManager | (BaseCallbackHandler | BaseCallbackHandlerMethodsClass)[])[]

    Returns Promise<ChainValues[]>

    ⚠️ Deprecated ⚠️

    Use .batch() instead. Will be removed in 0.2.0.

    This feature is deprecated and will be removed in the future.

    It is not recommended for use.

    Call the chain on all inputs in the list

  • Parameters

    • values: ChainValues & {
          signal?: AbortSignal;
          timeout?: number;
      }
    • Optional config: RunnableConfig | CallbackManager | (BaseCallbackHandler | BaseCallbackHandlerMethodsClass)[]
    • Optional tags: string[]

      Deprecated

    Returns Promise<ChainValues>

    Deprecated

    Use .invoke() instead. Will be removed in 0.2.0.

    Run the core logic of this chain and add to output if desired.

    Wraps _call and handles memory.

  • Check if the evaluation arguments are valid.

    Parameters

    • Optional reference: string

      The reference label.

    • Optional input: string

      The input string.

    Returns void

    Throws

    If the evaluator requires an input string but none is provided, or if the evaluator requires a reference label but none is provided.

  • Evaluate Chain or LLM output, based on optional input and label.

    Parameters

    • args: StringEvaluatorArgs
    • Optional config: BaseCallbackConfig | CallbackManager | (BaseCallbackHandler | BaseCallbackHandlerMethodsClass)[]

    Returns Promise<ChainValues>

    The evaluation results containing the score or value. It is recommended that the dictionary contain the following keys:

    • score: the score of the evaluation, if applicable.
    • value: the string value of the evaluation, if applicable.
    • reasoning: the reasoning for the evaluation, if applicable.
  • Invoke the chain with the provided input and returns the output.

    Parameters

    • input: ChainValues

      Input values for the chain run.

    • Optional options: RunnableConfig

    Returns Promise<ChainValues>

    Promise that resolves with the output of the chain run.

  • Parameters

    • inputs: Record<string, unknown>
    • outputs: Record<string, unknown>
    • returnOnlyOutputs: boolean = false

    Returns Promise<Record<string, unknown>>

  • Parameters

    • input: any
    • Optional config: RunnableConfig | CallbackManager | (BaseCallbackHandler | BaseCallbackHandlerMethodsClass)[]

    Returns Promise<string>

    Deprecated

    Use .invoke() instead. Will be removed in 0.2.0.

Generated using TypeDoc