A streaming RAG powered knowledge search tool that lets users ingest their own documents, runs paragraph level embeddings, and returns grounded answers with source citations in real time. Built with OpenAI embeddings, GPT-4o-mini, a custom in-memory vector store, and a Next 16 API layer hardened with rate limits and basic abuse protections.
RAG Knowledge Search is a small AI lab project that I built to show what applied AI engineering looks like when it is treated as a real product instead of a toy.
It lets users:
Under the hood it uses:
This project lives at /ai/rag and powers the first tile in my AI lab.
Most AI demos show a model that answers questions out of thin air. That is fine for a quick prototype, but it is not how teams actually ship reliable AI features in production.
I wanted a small, end-to-end example that solves a more realistic problem:
Given a set of arbitrary documents, let a user ask questions and get answers that are grounded in those documents, with sources and safety limits, in a way that feels like a real product.
The constraints I set for myself:
I split the project into four parts:
A central module that tracks documents and their embedded chunks in memory. This isolates all retrieval logic from the API surface and the UI.
A dedicated endpoint that accepts a title and body text, validates input, chunks it into segments, embeds those segments, and stores them in the shared vector store.
An endpoint that embeds the user query, runs cosine similarity against all chunks (both default and user-provided), selects the top matches, and constructs an LLM prompt that includes the question and retrieved context.
A React page that lets users ingest docs, see a list of what they have added, and talk to the system through a streaming chat interface. The UI tracks conversation history and highlights which source chunks were used.
The page is split into:
AI responses stream token-by-token into the last assistant message for a natural feel.
All RAG logic lives under /api/ai/rag:
A shared module that holds docs and chunks in memory. This makes it easy to support both default knowledge and user-uploaded content, and gives me a clean seam for swapping in a real database or vector service later.
This project is still a lab, but I treated the API like something that might be exposed to real users:
Each RAG route is limited per IP (roughly 20-30 requests per minute, adjustable) to prevent abuse.
Titles and content are length-limited. Questions are capped at a reasonable character count. Empty or malformed payloads are rejected early.
Each IP can only add a small number of documents into the in-memory store. This avoids unbounded growth and makes it harder to spam.
The RAG endpoints enforce origin constraints and add strict response headers through Next configuration.
This is not a full security appliance, but it already follows some good habits I would apply in a client context.
Small UX choices help the project feel like something I would actually ship to a client:
This project is small on purpose, but it captures most of what an AI engineer does in day-to-day work:
It also gives me a concrete, live example I can point to when talking about RAG, instead of hand-waving through it. Future work on this lab will involve swapping out the in-memory store for a persistent vector database and experimenting with lightweight evaluation of retrieval quality.
From sound engineering to AI systems—discover the journey that shaped how I build technology.