RAG-Based Document Assistant

Problem

Large enterprise documents (e.g., PDI user guide) are difficult to navigate. Users struggle to find precise answers quickly.

Solution

Built a Retrieval-Augmented Generation (RAG) system that:

• Extracts and chunks documents
• Stores embeddings in a vector database
• Retrieves relevant context
• Uses LLM to generate accurate answers

Approach

• Document parsing using Unstructured

• Chunking with overlap

• Embeddings using SentenceTransformers (MiniLM)

• Vector storage using ChromaDB

• Custom reranking logic (keyword + phrase + penalty)

• Dynamic context selection:

  • Narrow context for general queries
  • Wider context for table/option queries

Key Features

• Context-aware answer generation
• Strict grounding (no hallucination)
• Query-type based retrieval tuning
• Fallback when answer not found

Challenges

• Incorrect chunk retrieval (fixed with reranker tuning)
• Context noise affecting answers (fixed with dynamic neighbour window)
• Handling similar concepts (import vs export)
• Table data spread across chunks

Improvements Implemented

• Phrase-based reranking
• Opposite-term penalty (import vs export)
• Conditional context expansion
• Prompt tuning for precise answers

Tech Stack

• Python
• ChromaDB
• SentenceTransformers
• Llama (local via Ollama)
• Unstructured (document parsing)

How to Run

1. Run ingestion script to index documents
2. Run CLI (smart_run.py)
3. Enter query and view response

Future Improvements

• Better embedding models
• Larger LLM (e.g., 40B class)
• UI layer (optional)
• Evaluation framework