Building RAG Apps with LangChain

RAG Architecture Overview

RAG combines retrieval and generation: store documents in a vector database, retrieve relevant documents on query, then let the LLM generate answers.

Implementation

from langchain.document_loaders import TextLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter

loader = TextLoader("docs/")
documents = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000)
chunks = text_splitter.split_documents(documents)

See RAG Technology Survey for background and Vector Database: Pinecone vs Milvus for database selection.