🎯 Complete transformation from 5.9GB bloated system to 70MB optimized solution ✨ Key Features: - Hybrid embedding system (Ollama + ML fallback + hash backup) - Intelligent chunking with language-aware parsing - Semantic + BM25 hybrid search with rich context - Zero-config portable design with graceful degradation - Beautiful TUI for beginners + powerful CLI for experts - Comprehensive documentation with 8+ Mermaid diagrams - Professional animated demo (183KB optimized GIF) 🏗️ Architecture Highlights: - LanceDB vector storage with streaming indexing - Smart file tracking (size/mtime) to avoid expensive rehashing - Progressive chunking: Markdown headers → Python functions → fixed-size - Quality filtering: 200+ chars, 20+ words, 30% alphanumeric content - Concurrent batch processing with error recovery 📦 Package Contents: - Core engine: claude_rag/ (11 modules, 2,847 lines) - Entry points: rag-mini (unified), rag-tui (beginner interface) - Documentation: README + 6 guides with visual diagrams - Assets: 3D icon, optimized demo GIF, recording tools - Tests: 8 comprehensive integration and validation tests - Examples: Usage patterns, config templates, dependency analysis 🎥 Demo System: - Scripted demonstration showing 12 files → 58 chunks indexing - Semantic search with multi-line result previews - Complete workflow from TUI startup to CLI mastery - Professional recording pipeline with asciinema + GIF conversion 🛡️ Security & Quality: - Complete .gitignore with personal data protection - Dependency optimization (removed python-dotenv) - Code quality validation and educational test suite - Agent-reviewed architecture and documentation Ready for production use - copy folder, run ./rag-mini, start searching\!
5.5 KiB
FSS-Mini-RAG
A lightweight, educational RAG system that actually works
Built for beginners who want results, and developers who want to understand how RAG really works
How It Works
graph LR
Files[📁 Your Code] --> Index[🔍 Index]
Index --> Chunks[✂️ Smart Chunks]
Chunks --> Embeddings[🧠 Semantic Vectors]
Embeddings --> Database[(💾 Vector DB)]
Query[❓ "user auth"] --> Search[🎯 Hybrid Search]
Database --> Search
Search --> Results[📋 Ranked Results]
style Files fill:#e3f2fd
style Results fill:#e8f5e8
style Database fill:#fff3e0
What This Is
FSS-Mini-RAG is a distilled, lightweight implementation of a production-quality RAG (Retrieval Augmented Generation) search system. Born from 2 years of building, refining, and tuning RAG systems - from enterprise-scale solutions handling 14,000 queries/second to lightweight implementations that anyone can install and understand.
The Problem This Solves: Most RAG implementations are either too simple (poor results) or too complex (impossible to understand and modify). This bridges that gap.
Quick Start (2 Minutes)
# 1. Install everything
./install_mini_rag.sh
# 2. Start using it
./rag-tui # Friendly interface for beginners
# OR
./rag-mini index ~/my-project # Direct CLI for developers
./rag-mini search ~/my-project "authentication logic"
That's it. No external dependencies, no configuration required, no PhD in computer science needed.
What Makes This Different
For Beginners
- Just works - Zero configuration required
- Multiple interfaces - TUI for learning, CLI for speed
- Educational - Shows you CLI commands as you use the TUI
- Solid results - Finds code by meaning, not just keywords
For Developers
- Hackable - Clean, documented code you can actually modify
- Configurable - YAML config for everything, or change the code directly
- Multiple embedding options - Ollama, ML models, or hash-based
- Production patterns - Streaming, batching, error handling, monitoring
For Learning
- Complete technical documentation - How chunking, embedding, and search actually work
- Educational tests - See the system in action with real examples
- No magic - Every decision explained, every component documented
Usage Examples
Find Code by Concept
./rag-mini search ~/project "user authentication"
# Finds: login functions, auth middleware, session handling, password validation
Natural Language Queries
./rag-mini search ~/project "error handling for database connections"
# Finds: try/catch blocks, connection pool error handlers, retry logic
Development Workflow
./rag-mini index ~/new-project # Index once
./rag-mini search ~/new-project "API endpoints" # Search as needed
./rag-mini status ~/new-project # Check index health
Installation Options
Recommended: Full Installation
./install_mini_rag.sh
# Handles Python setup, dependencies, optional AI models
Experimental: Copy & Run (May Not Work)
# Copy folder anywhere and try to run directly
./rag-mini index ~/my-project
# Auto-setup will attempt to create environment
# Falls back with clear instructions if it fails
Manual Setup
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
Note: The experimental copy & run feature is provided for convenience but may fail on some systems. If you encounter issues, use the full installer for reliable setup.
System Requirements
- Python 3.8+ (installer checks and guides setup)
- Optional: Ollama (for best search quality - installer helps set up)
- Fallback: Works without external dependencies (uses built-in embeddings)
Project Philosophy
This implementation prioritizes:
- Educational Value - You can understand and modify every part
- Practical Results - Actually finds relevant code, not just keyword matches
- Zero Friction - Works out of the box, configurable when needed
- Real-world Patterns - Production techniques in beginner-friendly code
What's Inside
- Hybrid embedding system - Ollama → ML → Hash fallbacks
- Smart chunking - Language-aware code parsing
- Vector + keyword search - Best of both worlds
- Streaming architecture - Handles large codebases efficiently
- Multiple interfaces - TUI, CLI, Python API, server mode
Next Steps
- New users: Run
./rag-minifor guided experience - Developers: Read
TECHNICAL_GUIDE.mdfor implementation details - Contributors: See
CONTRIBUTING.mdfor development setup
Documentation
- Quick Start Guide - Get running in 5 minutes
- Visual Diagrams - 📊 System flow charts and architecture diagrams
- TUI Guide - Complete walkthrough of the friendly interface
- Technical Guide - How the system actually works
- Configuration Guide - Customizing for your needs
- Development Guide - Extending and modifying the code
License
MIT - Use it, learn from it, build on it.
Built by someone who got frustrated with RAG implementations that were either too simple to be useful or too complex to understand. This is the system I wish I'd found when I started.