RAG over a company's own content is a solution in which the assistant answers questions exclusively on the basis of your documents – policies, price lists, manuals, contracts, minutes, and a knowledge base – instead of answering from what it learned during training. It is used in two ways: internally, as an aid for employees, and externally, as an assistant for customers on the website. The principle is simple. Documents are split into smaller chunks, which are converted into a numerical representation and stored. When a question is asked, the most relevant chunks are found, inserted into the model's context, and the model formulates an answer from them with a reference to the source document. The result therefore stands or falls on the quality of the source material, not on the choice of model, a finding that surprises most companies. The first condition for success is order in the documents. If a company has three versions of the same policy in existence and no one knows which one is valid, the assistant will answer one way one time and another way the next, with both answers looking equally credible. Before deployment, therefore, an inventory should be carried out: what is valid, what has been revoked, who owns which document, and how often it is updated. The second condition is format. Scanned documents without recognised text, tables inserted as images, and presentations where the essential information is in the note under the slide are unusable for the system. Long documents without headings are equally harmful, since they split poorly into meaningful chunks and the answer then mixes unrelated passages. The third condition is access rights. An internal assistant must respect who is allowed to see which document – payroll records, personnel files, and contracts must not end up in the answer given to just anyone who asks. This is the most commonly underestimated risk in internal deployments, and it is addressed already at the design stage, not afterwards. The fourth condition is updating. A knowledge base that is not refreshed will, after six months, start answering according to invalid prices and conditions. Operation should therefore include a process that triggers reprocessing of a document when it changes, and regular checks of a sample of answers. The fifth condition is handling the situation where the answer is not present in the source material. The assistant should say that it does not have the information and refer the person to a human – not make one up. This behaviour is deliberately configured and tested, using questions for which no answer exists. In external deployment, there is an added obligation to clearly indicate that this is automated communication, along with the option to switch to a human. It is likewise advisable to record the questions the assistant could not answer – it is the cheapest list of missing website content a company can have. In terms of cost, it is good to know where the money is actually going. The model calls themselves tend to be a smaller item than preparing the source material, integrating with existing systems, and the time internal people spend testing and checking answers. Operating costs rise with the volume of questions and the length of the inserted context, so a more precise selection of a smaller number of passages both saves money and improves the answers. For a pilot deployment, therefore, set a success criterion expressed as a number in advance, limit the scope to a single process or a single team, and set a deadline by which a decision on continuation will be made. Without a criterion, the pilot turns into endless trialling. And finally: an assistant built over documents does not replace order in the documents. Companies that expect it to resolve their confusion over policies will find that it merely exposes it. Lastly, a practical note on selection: in most company deployments, it is not the choice of model that matters, but the quality of preparing the source material and the way documents are split into chunks.
See also: Content chunking, Internal AI usage policy, Content gap in retrieval.
