langchain.chains.combine_documents.map_reduce.MapReduceDocumentsChain

class langchain.chains.combine_documents.map_reduce.MapReduceDocumentsChain(*, memory: Optional[BaseMemory] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, callback_manager: Optional[BaseCallbackManager] = None, verbose: bool = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, input_key: str = 'input_documents', output_key: str = 'output_text', llm_chain: LLMChain, reduce_documents_chain: BaseCombineDocumentsChain, document_variable_name: str, return_intermediate_steps: bool = False)

Bases: BaseCombineDocumentsChain

Combining documents by mapping a chain over them, then combining results.

We first call llm_chain on each document individually, passing in the page_content and any other kwargs. This is the map step.

We then process the results of that map step in a reduce step. This should likely be a ReduceDocumentsChain.

Example

from langchain.chains import (
    StuffDocumentsChain,
    LLMChain,
    ReduceDocumentsChain,
    MapReduceDocumentsChain,
)
from langchain.prompts import PromptTemplate
from langchain.llms import OpenAI

# This controls how each document will be formatted. Specifically,
# it will be passed to `format_document` - see that function for more
# details.
document_prompt = PromptTemplate(
    input_variables=["page_content"],
     template="{page_content}"
)
document_variable_name = "context"
llm = OpenAI()
# The prompt here should take as an input variable the
# `document_variable_name`
prompt = PromptTemplate.from_template(
    "Summarize this content: {context}"
)
llm_chain = LLMChain(llm=llm, prompt=prompt)
# We now define how to combine these summaries
reduce_prompt = PromptTemplate.from_template(
    "Combine these summaries: {context}"
)
reduce_llm_chain = LLMChain(llm=llm, prompt=reduce_prompt)
combine_documents_chain = StuffDocumentsChain(
    llm_chain=reduce_llm_chain,
    document_prompt=document_prompt,
    document_variable_name=document_variable_name
)
reduce_documents_chain = ReduceDocumentsChain(
    combine_documents_chain=combine_documents_chain,
)
chain = MapReduceDocumentsChain(
    llm_chain=llm_chain,
    reduce_documents_chain=reduce_documents_chain,
)
# If we wanted to, we could also pass in collapse_documents_chain
# which is specifically aimed at collapsing documents BEFORE
# the final call.
prompt = PromptTemplate.from_template(
    "Collapse this content: {context}"
)
llm_chain = LLMChain(llm=llm, prompt=prompt)
collapse_documents_chain = StuffDocumentsChain(
    llm_chain=llm_chain,
    document_prompt=document_prompt,
    document_variable_name=document_variable_name
)
reduce_documents_chain = ReduceDocumentsChain(
    combine_documents_chain=combine_documents_chain,
    collapse_documents_chain=collapse_documents_chain,
)
chain = MapReduceDocumentsChain(
    llm_chain=llm_chain,
    reduce_documents_chain=reduce_documents_chain,
)

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

param callback_manager: Optional[BaseCallbackManager] = None

Deprecated, use callbacks instead.

param callbacks: Callbacks = None

Optional list of callback handlers (or callback manager). Defaults to None. Callback handlers are called throughout the lifecycle of a call to a chain, starting with on_chain_start, ending with on_chain_end or on_chain_error. Each custom chain can optionally call additional callback methods, see Callback docs for full details.

param document_variable_name: str [Required]

The variable name in the llm_chain to put the documents in. If only one variable in the llm_chain, this need not be provided.

param llm_chain: LLMChain [Required]

Chain to apply to each document individually.

param memory: Optional[BaseMemory] = None

Optional memory object. Defaults to None. Memory is a class that gets called at the start and at the end of every chain. At the start, memory loads variables and passes them along in the chain. At the end, it saves any returned variables. There are many different types of memory - please see memory docs for the full catalog.

param metadata: Optional[Dict[str, Any]] = None

Optional metadata associated with the chain. Defaults to None. This metadata will be associated with each call to this chain, and passed as arguments to the handlers defined in callbacks. You can use these to eg identify a specific instance of a chain with its use case.

param reduce_documents_chain: BaseCombineDocumentsChain [Required]

Chain to use to reduce the results of applying llm_chain to each doc. This typically either a ReduceDocumentChain or StuffDocumentChain.

param return_intermediate_steps: bool = False

Return the results of the map steps in the output.

param tags: Optional[List[str]] = None

Optional list of tags associated with the chain. Defaults to None. These tags will be associated with each call to this chain, and passed as arguments to the handlers defined in callbacks. You can use these to eg identify a specific instance of a chain with its use case.

param verbose: bool [Optional]

Whether or not run in verbose mode. In verbose mode, some intermediate logs will be printed to the console. Defaults to langchain.verbose value.

__call__(inputs: Union[Dict[str, Any], Any], return_only_outputs: bool = False, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, include_run_info: bool = False) → Dict[str, Any]

Execute the chain.

Parameters

• inputs – Dictionary of inputs, or single input if chain expects only one param. Should contain all inputs specified in Chain.input_keys except for inputs that will be set by the chain’s memory.

• return_only_outputs – Whether to return only outputs in the response. If True, only new keys generated by this chain will be returned. If False, both input keys and new keys generated by this chain will be returned. Defaults to False.

• callbacks – Callbacks to use for this chain run. These will be called in addition to callbacks passed to the chain during construction, but only these runtime callbacks will propagate to calls to other objects.

• tags – List of string tags to pass to all callbacks. These will be passed in addition to tags passed to the chain during construction, but only these runtime tags will propagate to calls to other objects.

• metadata – Optional metadata associated with the chain. Defaults to None

• include_run_info – Whether to include run info in the response. Defaults to False.

Returns

A dict of named outputs. Should contain all outputs specified in

Chain.output_keys.

async acall(inputs: Union[Dict[str, Any], Any], return_only_outputs: bool = False, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, include_run_info: bool = False) → Dict[str, Any]

Asynchronously execute the chain.

Parameters

• inputs – Dictionary of inputs, or single input if chain expects only one param. Should contain all inputs specified in Chain.input_keys except for inputs that will be set by the chain’s memory.

• return_only_outputs – Whether to return only outputs in the response. If True, only new keys generated by this chain will be returned. If False, both input keys and new keys generated by this chain will be returned. Defaults to False.

• callbacks – Callbacks to use for this chain run. These will be called in addition to callbacks passed to the chain during construction, but only these runtime callbacks will propagate to calls to other objects.

• tags – List of string tags to pass to all callbacks. These will be passed in addition to tags passed to the chain during construction, but only these runtime tags will propagate to calls to other objects.

• metadata – Optional metadata associated with the chain. Defaults to None

• include_run_info – Whether to include run info in the response. Defaults to False.

Returns

A dict of named outputs. Should contain all outputs specified in

Chain.output_keys.

async acombine_docs(docs: List[Document], token_max: Optional[int] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → Tuple[str, dict]

Combine documents in a map reduce manner.

Combine by mapping first chain over all documents, then reducing the results. This reducing can be done recursively if needed (if there are many documents).

async ainvoke(input: Dict[str, Any], config: Optional[RunnableConfig] = None) → Dict[str, Any]

apply(input_list: List[Dict[str, Any]], callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None) → List[Dict[str, str]]

Call the chain on all inputs in the list.

async arun(*args: Any, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any

Convenience method for executing chain.

The main difference between this method and Chain.__call__ is that this method expects inputs to be passed directly in as positional arguments or keyword arguments, whereas Chain.__call__ expects a single input dictionary with all the inputs

Parameters

• *args – If the chain expects a single input, it can be passed in as the sole positional argument.

• callbacks – Callbacks to use for this chain run. These will be called in addition to callbacks passed to the chain during construction, but only these runtime callbacks will propagate to calls to other objects.

• tags – List of string tags to pass to all callbacks. These will be passed in addition to tags passed to the chain during construction, but only these runtime tags will propagate to calls to other objects.

• **kwargs – If the chain expects multiple inputs, they can be passed in directly as keyword arguments.

Returns

The chain output.

Example

# Suppose we have a single-input chain that takes a 'question' string:
await chain.arun("What's the temperature in Boise, Idaho?")
# -> "The temperature in Boise is..."

# Suppose we have a multi-input chain that takes a 'question' string
# and 'context' string:
question = "What's the temperature in Boise, Idaho?"
context = "Weather report for Boise, Idaho on 07/03/23..."
await chain.arun(question=question, context=context)
# -> "The temperature in Boise is..."

combine_docs(docs: List[Document], token_max: Optional[int] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, **kwargs: Any) → Tuple[str, dict]

Combine documents in a map reduce manner.

Combine by mapping first chain over all documents, then reducing the results. This reducing can be done recursively if needed (if there are many documents).

dict(**kwargs: Any) → Dict

Dictionary representation of chain.

Expects Chain._chain_type property to be implemented and for memory to be

null.

Parameters

**kwargs – Keyword arguments passed to default pydantic.BaseModel.dict method.

Returns

A dictionary representation of the chain.

Example

..code-block:: python

chain.dict(exclude_unset=True) # -> {“_type”: “foo”, “verbose”: False, …}

validator get_default_document_variable_name  »  all fields

Get default document variable name, if not provided.

validator get_reduce_chain  »  all fields

For backwards compatibility.

validator get_return_intermediate_steps  »  all fields

For backwards compatibility.

invoke(input: Dict[str, Any], config: Optional[RunnableConfig] = None) → Dict[str, Any]

prep_inputs(inputs: Union[Dict[str, Any], Any]) → Dict[str, str]

Validate and prepare chain inputs, including adding inputs from memory.

Parameters

inputs – Dictionary of raw inputs, or single input if chain expects only one param. Should contain all inputs specified in Chain.input_keys except for inputs that will be set by the chain’s memory.

Returns

A dictionary of all inputs, including those added by the chain’s memory.

prep_outputs(inputs: Dict[str, str], outputs: Dict[str, str], return_only_outputs: bool = False) → Dict[str, str]

Validate and prepare chain outputs, and save info about this run to memory.

Parameters

• inputs – Dictionary of chain inputs, including any inputs added by chain memory.

• outputs – Dictionary of initial chain outputs.

• return_only_outputs – Whether to only return the chain outputs. If False, inputs are also added to the final outputs.

Returns

A dict of the final chain outputs.

prompt_length(docs: List[Document], **kwargs: Any) → Optional[int]

Return the prompt length given the documents passed in.

This can be used by a caller to determine whether passing in a list of documents would exceed a certain prompt length. This useful when trying to ensure that the size of a prompt remains below a certain context limit.

Parameters

docs – List[Document], a list of documents to use to calculate the total prompt length.

Returns

Returns None if the method does not depend on the prompt length, otherwise the length of the prompt in tokens.

validator raise_callback_manager_deprecation  »  all fields

Raise deprecation warning if callback_manager is used.

run(*args: Any, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → Any

Convenience method for executing chain.

The main difference between this method and Chain.__call__ is that this method expects inputs to be passed directly in as positional arguments or keyword arguments, whereas Chain.__call__ expects a single input dictionary with all the inputs

Parameters

• *args – If the chain expects a single input, it can be passed in as the sole positional argument.

• callbacks – Callbacks to use for this chain run. These will be called in addition to callbacks passed to the chain during construction, but only these runtime callbacks will propagate to calls to other objects.

• tags – List of string tags to pass to all callbacks. These will be passed in addition to tags passed to the chain during construction, but only these runtime tags will propagate to calls to other objects.

• **kwargs – If the chain expects multiple inputs, they can be passed in directly as keyword arguments.

Returns

The chain output.

Example

# Suppose we have a single-input chain that takes a 'question' string:
chain.run("What's the temperature in Boise, Idaho?")
# -> "The temperature in Boise is..."

# Suppose we have a multi-input chain that takes a 'question' string
# and 'context' string:
question = "What's the temperature in Boise, Idaho?"
context = "Weather report for Boise, Idaho on 07/03/23..."
chain.run(question=question, context=context)
# -> "The temperature in Boise is..."

save(file_path: Union[Path, str]) → None

Save the chain.

Expects Chain._chain_type property to be implemented and for memory to be

null.

Parameters

file_path – Path to file to save the chain to.

Example

chain.save(file_path="path/chain.yaml")

validator set_verbose  »  verbose

Set the chain verbosity.

Defaults to the global setting if not specified by the user.

to_json() → Union[SerializedConstructor, SerializedNotImplemented]

to_json_not_implemented() → SerializedNotImplemented

property collapse_document_chain: langchain.chains.combine_documents.base.BaseCombineDocumentsChain

Kept for backward compatibility.

property combine_document_chain: langchain.chains.combine_documents.base.BaseCombineDocumentsChain

Kept for backward compatibility.

property lc_attributes: Dict

Return a list of attribute names that should be included in the serialized kwargs. These attributes must be accepted by the constructor.

property lc_namespace: List[str]

Return the namespace of the langchain object. eg. [“langchain”, “llms”, “openai”]

property lc_secrets: Dict[str, str]

Return a map of constructor argument names to secret ids. eg. {“openai_api_key”: “OPENAI_API_KEY”}

property lc_serializable: bool

Return whether or not the class is serializable.

model Config

Bases: object

Configuration for this pydantic object.

arbitrary_types_allowed = True

extra = 'forbid'