langchain.embeddings.llamacpp.LlamaCppEmbeddings

class langchain.embeddings.llamacpp.LlamaCppEmbeddings(*, client: Any = None, model_path: str, n_ctx: int = 512, n_parts: int = - 1, seed: int = - 1, f16_kv: bool = False, logits_all: bool = False, vocab_only: bool = False, use_mlock: bool = False, n_threads: Optional[int] = None, n_batch: Optional[int] = 8, n_gpu_layers: Optional[int] = None)

Bases: BaseModel, Embeddings

llama.cpp embedding models.

To use, you should have the llama-cpp-python library installed, and provide the path to the Llama model as a named parameter to the constructor. Check out: https://github.com/abetlen/llama-cpp-python

Example

from langchain.embeddings import LlamaCppEmbeddings
llama = LlamaCppEmbeddings(model_path="/path/to/model.bin")

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

param f16_kv: bool = False

Use half-precision for key/value cache.

param logits_all: bool = False

Return logits for all tokens, not just the last token.

param model_path: str [Required]

param n_batch: Optional[int] = 8

Number of tokens to process in parallel. Should be a number between 1 and n_ctx.

param n_ctx: int = 512

Token context window.

param n_gpu_layers: Optional[int] = None

Number of layers to be loaded into gpu memory. Default None.

param n_parts: int = -1

Number of parts to split the model into. If -1, the number of parts is automatically determined.

param n_threads: Optional[int] = None

Number of threads to use. If None, the number of threads is automatically determined.

param seed: int = -1

Seed. If -1, a random seed is used.

param use_mlock: bool = False

Force system to keep model in RAM.

param vocab_only: bool = False

Only load the vocabulary, no weights.

embed_documents(texts: List[str]) → List[List[float]]

Embed a list of documents using the Llama model.

Parameters

texts – The list of texts to embed.

Returns

List of embeddings, one for each text.

embed_query(text: str) → List[float]

Embed a query using the Llama model.

Parameters

text – The text to embed.

Returns

Embeddings for the text.

validator validate_environment  »  all fields

Validate that llama-cpp-python library is installed.

model Config

Bases: object

Configuration for this pydantic object.

extra = 'forbid'

Examples using LlamaCppEmbeddings

• Llama-cpp

• Llama.cpp