A practical intro to RAG systems

Why RAG?

The most practical fix for the "knowledge cutoff" problem in LLMs. Instead of retraining, we feed external sources into the model at runtime.

Building the pipeline

1. Chunking — Split documents into 512–1024 token chunks that preserve meaning.
2. Embedding — Models like text-embedding-3-small or bge-base.
3. Vector store — pgvector or Qdrant, depending on scale.
4. Retrieval + Reranking — Pull top-K, then re-rank with a cross-encoder.
5. Generation — Add the context to your prompt and call the model.

Verification

Make sure the produced answers cite their sources. In our version every sentence is tagged with a <doc-id> and the UI shows the source on hover.