The SageMakerEndpoint class is used to interact with SageMaker Inference Endpoint models. It uses the AWS client for authentication, which automatically loads credentials. If a specific credential profile is to be used, the name of the profile from the ~/.aws/credentials file must be passed. The credentials or roles used should have the required policies to access the SageMaker endpoint.

Hierarchy

  • LLM<BaseLLMCallOptions>
    • SageMakerEndpoint

Constructors

Properties

client: SageMakerRuntimeClient
endpointName: string
streaming: boolean
endpointKwargs?: Record<string, unknown>
modelKwargs?: Record<string, unknown>

Generated using TypeDoc