RAGFlow, the problem that starts before retrieval

RAGFlow, the problem that starts before retrieval

Most tools for searching your own documents assume a document is text. In practice a document is a PDF with two columns, a table split across two pages, a header repeating in the footer, and a chart whose caption sits beside it rather than below.

Ordinary text extraction from such a file yields a string where columns interleave, a table becomes a jumble of numbers, and a page number appears mid sentence. Search over that material fails whatever the embedding model's quality.

RAGFlow starts precisely at that problem. Document parsing recognises layout before anything is split into fragments.

Document layout parsing

The layer responsible for understanding documents does three things ordinary text extraction does not.

Layout recognition separates headings from body text, footnotes from paragraphs, and captions from figure descriptions. Columns therefore stay separate, and repeating footers and page numbers never reach the fragments.

Table structure recognition detects table boundaries, headers, and cell membership. That means a table enters the index as a table, preserving the link between a value and its column header, rather than as a run of numbers without context.

Optical character recognition handles scans and files where text is an image. Without that step scanned documents are invisible to search, and in many organisations they make up a substantial share of the archive.

The effect shows on questions about table values. A question about a limit for a particular plan in a comparison table stands a chance of a correct answer only if the table was read as a structure rather than as text.

That layer is part of the application and you cannot lift it out into a pipeline of your own. When you need only the conversion of files into typed elements, Unstructured fills the role, accepting dozens of formats and returning a list of elements for you to chunk. Choosing between the two comes down to whether you want a finished application or a library plugged into your own code.

Chunking templates

The second distinguishing feature is that how a document splits into fragments depends on its kind rather than being uniform for everything.

Technical documentation splits by headings, since a section is the natural unit of an answer. A contract splits by clauses, since that is how it is built. A table stays whole or splits by rows with the header repeated. A presentation splits by slide. A résumé splits by section.

That approach solves a problem uniform splitting every five hundred characters cannot avoid. A fragment starting mid sentence and ending mid table answers no question, and naive splitting produces many such fragments.

Practical advice: review the generated fragments after first indexing before building anything further. Fifteen minutes of reading usually reveals that the chosen template does not fit some documents, and fixing that changes answer quality more than any prompt tuning.

Citations pointing at the source

The third distinguishing feature is that an answer points not only at a document but at a specific place within it.

Since parsing records a fragment's position on the page, the interface can show the original document with the source of the answer highlighted. The user then sees context rather than a text excerpt alone.

That matters more than it appears to on documents where context decides meaning. A table value without a visible column header and without the footnote below the table can mislead, and pointing at the location lets that be checked in a second.

The second benefit concerns trust. An answer verifiable in one click is treated differently from one without a source, particularly in organisations where somebody answers for a decision taken on its basis.

Self hosting

The project is open source under a permissive licence and runs on your own infrastructure, which with internal documents is a condition rather than a preference.

git clone https://github.com/infiniflow/ragflow.git
cd ragflow/docker
docker compose up -d

The set covers a server, a document store, a vector store, and an interface. That means starting up is simple and hardware requirements are markedly higher than for a library added to your own application.

Models can be connected externally or run locally. A fully local variant, with a model served through Ollama or an inference server such as vLLM, allows work on a network cut off from the internet, which in some institutions is a formal requirement.

Price the resources before deploying. Document parsing with image recognition and table structure detection is computationally expensive, so indexing a large archive needs time and power rather than disk space alone.

Retrieval and reranking

On the answering side the solution combines two mechanisms whose combination delivers more than either alone.

Hybrid search merges semantic results with keyword matching. That matters on documents holding contract numbers, product codes, and identifiers a vector cannot capture, since they mean nothing semantically.

Reranking through a separate model scores the question and fragment pair together, judging more accurately than vector similarity alone. The reranking model can be connected externally or run locally, which matters when working on a disconnected network.

Measure each mechanism's effect separately before considering the matter closed. On documentation full of proper names the keyword component delivers most; on continuous prose little; and reranking works the other way round.

The third parameter worth attention is how many fragments reach the answer. Twenty is usually too many, since the model loses its way in the excess and the answer loses precision. Five apt fragments beat twenty average ones and cost less along the way.

Beyond search

Newer versions add things reaching past answering questions.

Agent flows let you build a process covering retrieval, a tool call, and a decision, rather than a single query. That helps on tasks where an answer requires combining information from documents and from an external system.

Knowledge graph search, covered more broadly in the piece on GraphRAG, handles corpus wide questions that fragment retrieval alone cannot answer.

Data source connectors pull documents incrementally from systems where they already live rather than requiring manual upload. That decides whether a deployment survives a month, since a manually updated archive stops being updated.

Chat channel integrations expose search where people already ask questions rather than requiring a visit to a separate site. That detail noticeably changes how often it actually gets used.

Permissions and document access

Search over company documents faces a question needing an answer before deployment: who may see what.

A knowledge base model lets you split documents into sets and grant access per set. That suffices when the split is coarse: public documentation, internal, personnel. At individual document level it gets harder.

An unpleasant point deserves stating. A fragment that reached an answer also reached the prompt, so any access restricting mechanism must act before retrieval rather than after. Filtering answers afterwards is not a safeguard, since the content has already left the store.

The practical consequence: if your permission model is complex, check whether the tool can express it before uploading documents. Rebuilding the knowledge base split after indexing an archive means indexing again.

The second thing is an access log. With sensitive documents you usually need to know who asked what and what they received. Check that when choosing the tool, since adding it yourself means reaching into an application you did not write.

Keeping the archive current

Documents change, and a search tool answering with year old content is worse than none, since it sounds credible.

Source connectors pull documents incrementally, so a changed file reaches the index without manual work. That decides whether a deployment survives, since an archive requiring manual upload stops being updated within weeks.

Deletion is a separate matter. A document withdrawn from the source must disappear from the index too, otherwise an answer rests on content that no longer exists. That sounds obvious and gets skipped, since adding is visible and removing is not.

The third thing is versions. On regularly changing documents, keep the date in metadata and show it beside the answer. A user seeing that information comes from a document two years old judges it differently from one without a date.

Set up a periodic consistency check: how many documents sit in the source, how many in the index, and whether those numbers agree. Drift caught automatically is a technical problem; drift caught by a user is a trust problem.

RAGFlow against the alternatives

The key difference comes down to whether your documents are hard. A set of text files or documentation pages needs no advanced layout parsing, so lighter tools give the same result for less effort.

An archive of PDFs with scans, tables, and multi column layout is an entirely different situation. There, document parsing is the hardest part of the whole task, and a ready solution saves weeks of work nobody planned for.

A simple way to settle the question: take the ten hardest documents from your collection and run them through ordinary text extraction. If the result is readable, lighter tools will do. If the columns interleave and the tables turn into a string of numbers, you have your answer.

Common mistakes

The first is skipping the review of generated fragments. A chunking template mismatched to the documents yields useless fragments, and that shows only once you look at them.

The second is assuming optical character recognition works perfectly. Poor quality scans produce errors, so on documents where value accuracy matters, check a sample by hand.

The third is deploying without pricing resources. Parsing with image and table recognition loads the processor far more than plain text extraction, so indexing a large archive can take a day.

The fourth is uploading documents by hand on a corpus that changes. Without a source connector the archive stops being current within a quarter and nobody notices until somebody receives a stale answer.

The fifth is not measuring quality. The measures covered in the piece on Ragas apply here identically, and without them every configuration change is a bet.

The sixth is treating this as a library. It is a complete application with its own interface and storage, so wiring it into an existing product means using its programming interface rather than importing it as a dependency.

FAQ

How does RAGFlow differ from other document search tools?

By focusing on parsing documents before indexing. It recognises page layout, table structure, and text in images, so it copes with complex PDFs and scans where ordinary text extraction fails.

Is it free?

Yes, the project is open source under a permissive licence and runs on your own infrastructure. You pay for hardware and for model calls, provided you use an external model rather than a locally hosted one.

Can it work without internet access?

Yes, with locally hosted models everything runs on a network cut off from the world. That is one of the main reasons this solution appears in institutions with requirements about data processing.

What are the hardware requirements?

Markedly higher than for a library added to an application, since you run several services and parsing with image recognition loads the processor. With locally hosted models a graphics card joins that, so price it before deciding.

When is something else the better pick?

When documents are simple, documentation pages or text files for instance, since advanced layout parsing adds nothing there. When building an application with its own interface, a library such as Haystack gives more control with less overhead.

Documentation and code sit in the project repository, and the document parsing layer is described in a separate chapter.