langchain.document_loaders.readthedocs.ReadTheDocsLoader¶

class langchain.document_loaders.readthedocs.ReadTheDocsLoader(path: Union[str, Path], encoding: Optional[str] = None, errors: Optional[str] = None, custom_html_tag: Optional[Tuple[str, dict]] = None, **kwargs: Optional[Any])[source]¶

Bases: BaseLoader

Loads ReadTheDocs documentation directory dump.

Initialize ReadTheDocsLoader

The loader loops over all files under path and extracts the actual content of the files by retrieving main html tags. Default main html tags include <main id=”main-content>, <div role=”main>, and <article role=”main”>. You can also define your own html tags by passing custom_html_tag, e.g. (“div”, “class=main”). The loader iterates html tags with the order of custom html tags (if exists) and default html tags. If any of the tags is not empty, the loop will break and retrieve the content out of that tag.

Parameters

  • path – The location of pulled readthedocs folder.

  • encoding – The encoding with which to open the documents.

  • errors – Specify how encoding and decoding errors are to be handled—this cannot be used in binary mode.

  • custom_html_tag – Optional custom html tag to retrieve the content from files.

Methods

__init__(path[, encoding, errors, ...])

Initialize ReadTheDocsLoader

lazy_load()

A lazy loader for Documents.

load()

Load documents.

load_and_split([text_splitter])

Load Documents and split into chunks.
lazy_load() → Iterator[Document]¶

A lazy loader for Documents.

load() → List[Document][source]¶

Load documents.

load_and_split(text_splitter: Optional[TextSplitter] = None) → List[Document]¶

Load Documents and split into chunks. Chunks are returned as Documents.

Parameters

text_splitter – TextSplitter instance to use for splitting documents. Defaults to RecursiveCharacterTextSplitter.

Returns

List of Documents.

Examples using ReadTheDocsLoader¶