- RAG = find the relevant passages in your documents, then have the model answer only from them.
- The hard part is not the model, it is retrieval: chunking, embeddings, thresholds.
- A good assistant cites its sources and refuses when the corpus does not cover the question — and that refusal gets tested.
- Across French and Arabic, similarity thresholds must be tuned per language. We learned that on El Maître.
The problem: general models make things up
A large language model learned from the web. Ask it something specific about your policies, your catalogue or a national legal code and it will produce a plausible answer — sometimes right, sometimes wrong, always in the same confident tone. For business use that is the worst case: you cannot tell when to be suspicious.
How RAG works
- Your documents are split into passages and turned into vectors (embeddings) stored in a database — we use PostgreSQL with pgvector.
- When someone asks a question, the closest passages are retrieved.
- If nothing is close enough, the assistant says so instead of answering.
- Otherwise the model gets the question and those passages, with instructions to answer only from them and cite the source.
Your documents stay in your database. The model does not learn them; it reads them at answer time.
A real build: El Maître, 8,000 legal articles in French and Arabic
El Maître is a Tunisian legal platform. We scraped and ingested over 8,000 articles across 16 codes, in French and Arabic, into pgvector, then built an assistant that answers with the article it relied on and declines when the corpus is silent.
The main difficulty was not where we expected it. Arabic matches consistently scored lower than equivalent French ones. With a single threshold, the assistant wrongly refused Arabic questions the corpus answered perfectly well. Tuning thresholds per language removed the largest source of false refusals.
Lesson: test the refusal path as seriously as the good answers. An assistant that refuses too much is as useless as one that invents.
Where it earns its keep
- Customer support: answer from product documentation and terms.
- Internal: find a procedure, clause or HR rule without digging through twenty PDFs.
- Legal and compliance: query regulations and get the exact reference.
- Sales: answer technical questions about a product catalogue.
What to plan for
- Clean, current documents: the assistant is never better than its corpus.
- A test set of questions, with expected answers and questions it must refuse.
- API costs that scale with usage, usually small per question; hosting the database is a fixed cost.
- An interface: a widget on your site, an internal tool, or an integration into an existing app.