Document Loader
The document loader can be used to load data from various data sources. The data loaded from the source is stored in the langchain as a Document object, representing a document. The Document object contains a piece of text and related metadata.
The document loader exposes a "load" method to load data from the configured data source. They can also choose to implement "lazy load" for conveniently loading data into memory at a later time.
Loading Text
The simplest loader is to load the text data of a file into a Document.
from langchain_community.document_loaders import TextLoader
loader = TextLoader("./index.md")
loader.load()
[
Document(page_content='---\\nsidebar_position: 0\\n---\\n# Document loaders\\n\\nUse document loaders to load data from a source as `Document`\\'s. A `Document` is a piece of text\\nand associated metadata. For example, there are document loaders for loading a simple `.txt` file, for loading the text\\ncontents of any web page, or even for loading a transcript of a YouTube video.\\n\\nEvery document loader exposes two methods:\\n1. "Load": load documents from the configured source\\n2. "Load and split": load documents from the configured source and split them using the passed in text splitter\\n\\nThey optionally implement:\\n\\n3. "Lazy load": load documents into memory lazily\\n', metadata={'source': '../docs/docs_skeleton/docs/modules/data_connection/document_loaders/index.md'})
]