RAG Document Q&A — AI Assistant with Cited Answers
Page-level cited document Q&A with semantic search, similarity thresholding, and $0 infrastructure cost.
Verified answer synthesized from uploaded PDF context with strict page-level source attribution and automatic model fallback.
Built and deployed a document Q&A app that returns page-level cited answers and declines out-of-scope questions through semantic search with a similarity threshold, using pgvector, Gemini embeddings, and Groq at $0 infrastructure cost.
Operational Challenge
Standard LLM chat interfaces hallucinate answers when queried against private PDFs and fail to tell users which exact page a claim came from or when a question falls outside the uploaded documents.
Intended Audience & Use Case
Training teams, researchers, analysts, and internal operations staff who need verifiable, page-cited answers from technical manuals and PDFs.
System Design & Data Flow
Next.js Route Handlers ingest PDFs into page-tagged text chunks, generate vector embeddings via Gemini, store vectors in Neon PostgreSQL (pgvector), and retrieve top-k chunks above a cosine similarity threshold before synthesizing cited responses via Groq.
Important Technical Decisions
- —Enforced a strict cosine similarity threshold so out-of-scope queries return an explicit refusal instead of hallucinated prose.
- —Implemented resumable batch embedding and automatic model fallback to keep ingestion and inference reliable under free-tier rate limits.
- —Stored rate-limiting counters in PostgreSQL (3 questions per minute per visitor) to avoid extra Redis/cache infrastructure cost.
Features Implemented
- Built and deployed a document Q&A app that returns page-level cited answers and declines out-of-scope questions, through semantic search with a similarity threshold, using pgvector, Gemini embeddings, and Groq at $0 infrastructure cost
- Kept the app reliable within free-tier API limits through resumable batch embedding, automatic model fallback, and Postgres-backed rate limiting (3 questions per minute per visitor), using Next.js route handlers and Neon
- Shipped with 30+ unit tests, an evaluation harness that scores answer correctness, citation accuracy, and refusals, and CI on GitHub Actions (lint, type-check, tests, build, Docker image)
Testing & Engineering Notes
- —30+ unit tests covering chunking, similarity filtering, citation formatting, and refusal guards.
- —Automated evaluation harness scoring answer correctness, citation accuracy, and out-of-scope refusals.
- —GitHub Actions CI pipeline running lint, TypeScript checks, tests, production build, and Docker image verification.