What a RAG Chatbot Over Your Own Documents Really Costs
A chatbot that answers from your own documents costs roughly $500 to $2,500 to build, depending on whether it is a single internal bot or a multi-tenant product, and typically $20 to $200 a month to run for a small business. The build is three to ten working days. Most of the cost is not the chatbot, it is the accuracy work: chunking the documents properly, forcing citations, and making the bot say it does not know instead of inventing an answer.
The request arrives in almost identical words every week. A company has eighty PDFs, product guides, SOPs and FAQs. Support answers the same questions over and over. They want a chatbot that answers from those documents, cites its sources, and says it does not know rather than inventing something.
Here is what that costs and where the money goes.
What it costs to build a RAG chatbot
| Scope | Price | Timeline |
|---|---|---|
| Proof of concept, one document set, no auth | $100 to $400 | 1 to 3 days |
| Internal bot over fixed documents, with citations | $500 to $1,000 | 3 to 5 days |
| Customer-facing bot on your site, with an admin area | $1,000 to $1,800 | 1 to 2 weeks |
| Multi-tenant product, customers upload their own docs | $1,500 to $2,500 | 2 to 3 weeks |
Do not put a proof of concept in front of customers. It will have no rate limiting, no logging of what it got wrong, and no handling for questions it cannot answer.
The jump to multi-tenant roughly doubles the price, and it is not the chat part. It is tenant isolation, document upload, usage limits, and an admin surface so someone can see what the bot has been saying.
What it costs to run each month
| Line item | Small business | Notes |
|---|---|---|
| LLM API calls | $10 to $120 | Depends on traffic and model |
| Embeddings | $1 to $10 | One-off per document |
| Vector storage | $0 to $70 | $0 with pgvector |
| Hosting | $0 to $25 | Often free tier at this size |
| Total | $20 to $200 | A few hundred conversations |
Two decisions move this more than anything else.
Use pgvector unless you have a reason not to. If you already run Postgres, pgvector adds vector search for free. No new service, no new bill. Pinecone and Qdrant are good above roughly a million chunks or when you need fast metadata filtering, but below that they buy you very little.
Pick the model per question, not per project. Most support questions are easy once the right passage is retrieved. Route those to a small model and keep the large one for questions spanning several documents. This commonly halves the API bill with no visible quality drop.
Where the work actually goes
The chat interface is a day. Retrieval is a day. The remaining three to eight days are accuracy, and accuracy is the product.
Chunking decides whether it works
A chunk is the slice of document the system retrieves and hands to the model. Chunk badly and no prompting saves you.
The tutorial default of 1,000 characters with 200 overlap cuts tables in half and separates headings from their text. For real company PDFs:
- Split on document structure first, so headings, sections and tables stay intact
- Keep tables whole even when large. Half a table is worse than none
- Attach document name, page number and section heading as metadata. You need these for citations and they improve retrieval
- Target 300 to 800 tokens for prose, larger for tables
Scanned PDFs need OCR first, and OCR quality caps everything downstream. If your sources are scans of faxes, say so before anyone quotes.
Citations are not decoration
Every answer should name its document and page. The obvious reason is trust. The better reason is that requiring a citation gives you something to check: if the model returns an answer with no citation, reject it before the user sees it. The model cannot cite a passage it was never given.
Making it say "I don't know"
A bot that admits ignorance 20 percent of the time is safe. One that answers everything and invents 5 percent is not, because nobody can tell which 5 percent.
- Check retrieval before generating. If no chunk clears your relevance threshold, do not call the model at all. Most invented answers are retrieval failures, not generation failures.
- Instruct and constrain. Tell the model to answer only from the provided passages. Keep it short and specific.
- Reject uncited answers in code. Confirm a citation is present and points at a chunk you actually supplied.
The first and third are code, not prompting. Rules that matter get enforced around the model, not requested inside it.
Evaluation, cheaper than it sounds
Write 15 to 20 real questions with known answers and run them automatically. An afternoon of work, and it separates a bot you can launch from one you are hoping about.
Build the set from real support tickets: the ten most common questions, three whose answer sits in a table, three spanning two documents, and three the documents genuinely do not answer, where the correct response is "I don't know". That last group is the one everyone forgets.
Expect 85 to 95 percent on a clean document set. Below 80, the problem is chunking or OCR, not the model.
Log the questions it could not answer
Clients increasingly ask for this by name: every unanswered question logged in one place where a non-technical person can read it, write the correct answer, and have the bot use it from then on.
Half a day of work, and it is the difference between a bot that decays as your product changes and one that improves without a developer.
A stack that works
- Ingestion: Python with a structure-aware PDF parser, plus OCR for scans
- Embeddings and storage: pgvector in Postgres
- Retrieval: vector search plus keyword search, then re-ranking. Hybrid beats pure vector on documents full of product codes, because exact terms matter and embeddings blur them
- API: FastAPI with one
/chatendpoint - Frontend: an embeddable widget, or a page in your existing app
- Evaluation: a script and a CSV of questions
LangChain and LlamaIndex will save you time on ingestion. They are not required, and for one well-understood pipeline, writing retrieval yourself is often less code than configuring theirs.
Hiring someone to build it
What to search for. "RAG developer", "AI chatbot development company", "LangChain developer", or "AI integration agency". A general backend developer can learn this, but they will learn it on your budget. Ask for a RAG project they have shipped and what they did to raise answer accuracy.
What to ask for in the quote:
- Source citations with document name and page in every answer
- A stated behaviour for questions the documents do not cover
- An evaluation run against questions you supply, with the score shared
- Your own repository, your own API keys, your own database
- A documented re-ingestion process for when documents change
- A log of unanswered questions a non-technical person can review
The citation requirement and that last point separate a bot still running in six months from one quietly switched off.
Related
We build these as AI integration work, and AI agents and automation covers the version that takes actions rather than only answering. The AI Client Qualifier is our own product running this pipeline, and the AI WhatsApp and call assistant is the same retrieval over a different channel.
If the bot is going into an app built with Lovable or Claude Code, read the production readiness checklist first. Adding a chatbot to an app with open database permissions widens the problem.
Common questions
- How much does it cost to build a RAG chatbot?
- Around $500 to $1,000 for an internal bot over a fixed set of documents, delivered in three to five days. Around $1,500 to $2,500 for a multi-tenant version where customers upload their own documents and get an embeddable widget, delivered in two to three weeks. A throwaway proof of concept can be done for a few hundred dollars but should not be put in front of customers.
- How much does a RAG chatbot cost to run per month?
- For a small business with a few hundred conversations a month, expect $20 to $200. The three line items are the LLM API calls, the vector database, and hosting. Using pgvector inside a Postgres database you already pay for removes the vector database cost entirely, which is why it is the sensible default below roughly a million chunks.
- How do you stop a RAG chatbot from hallucinating?
- Three things, in order of impact. Instruct the model to answer only from the retrieved passages and to say it does not know otherwise. Require a citation for every claim, naming the document and page, and reject the answer if no citation comes back. Then check that retrieval actually returned relevant passages before calling the model at all, because most hallucinations are retrieval failures rather than generation failures.
- Which vector database should I use for a RAG chatbot?
- Use pgvector if you already run Postgres, which covers most projects and adds no new service or bill. Use Qdrant or Pinecone when you need more than roughly a million chunks, need metadata filtering at speed, or want a managed service with no database administration. The choice rarely affects answer quality; chunking and retrieval strategy affect it far more.
- Who can build a RAG chatbot for my business?
- Look for a RAG developer, an AI chatbot development company, or an AI integration agency, and ask for a retrieval system they have already shipped plus what they did to raise its answer accuracy. A general backend developer can learn this, but they will learn it on your budget. The quote should include citations with document name and page, an evaluation run against questions you supply, and your own repository and API keys.
- How accurate should a RAG chatbot be before launch?
- Test against 15 to 20 real questions with known correct answers before anyone else sees it, and expect to reach somewhere around 85 to 95 percent on a clean document set. The number matters less than the failure mode: a bot that answers 80 percent and clearly says it does not know for the rest is safe to launch, while one that answers 95 percent and invents the other 5 percent is not.
Want this looked at properly?
Tell us what you're working with. We'll give you a straight answer in a 30-minute call, free.