langchain.embeddings.xinference.XinferenceEmbeddings

class langchain.embeddings.xinference.XinferenceEmbeddings(server_url: Optional[str] = None, model_uid: Optional[str] = None)[source]

Bases: Embeddings

Wrapper around xinference embedding models. To use, you should have the xinference library installed: .. code-block:: bash

pip install xinference

Check out: https://github.com/xorbitsai/inference To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers .. rubric:: Example

To start a local instance of Xinference, run
$ xinference

You can also deploy Xinference in a distributed cluster. Here are the steps: Starting the supervisor: .. code-block:: bash

$ xinference-supervisor

Starting the worker: .. code-block:: bash

$ xinference-worker

Then, launch a model using command line interface (CLI).

Example: .. code-block:: bash

$ xinference launch -n orca -s 3 -q q4_0

It will return a model UID. Then you can use Xinference Embedding with LangChain.

Example: .. code-block:: python

from langchain.embeddings import XinferenceEmbeddings

xinference = XinferenceEmbeddings(

server_url=”http://0.0.0.0:9997”, model_uid = {model_uid} # replace model_uid with the model UID return from launching the model

)

Methods

__init__([server_url, model_uid])

aembed_documents(texts)

Asynchronous Embed search docs.

aembed_query(text)

Asynchronous Embed query text.

embed_documents(texts)

Embed a list of documents using Xinference.

embed_query(text)

Embed a query of documents using Xinference.

Attributes

client

server_url

URL of the xinference server

model_uid

UID of the launched model
async aembed_documents(texts: List[str]) → List[List[float]]

Asynchronous Embed search docs.

async aembed_query(text: str) → List[float]

Asynchronous Embed query text.

embed_documents(texts: List[str]) → List[List[float]][source]

Embed a list of documents using Xinference. :param texts: The list of texts to embed.

Returns

List of embeddings, one for each text.

embed_query(text: str) → List[float][source]

Embed a query of documents using Xinference. :param text: The text to embed.

Returns

Embeddings for the text.

client: Any
model_uid: Optional[str]

UID of the launched model

server_url: Optional[str]

URL of the xinference server

Examples using XinferenceEmbeddings