PixelRAG: UC Berkeley reinvents RAG with a system that does not read text but looks at pages as images

Ever since RAG systems — Retrieval-Augmented Generation — became one of the most widely adopted architectural patterns in applied AI, the developer community has always taken one fundamental assumption for granted: to find information inside a document, you must first extract its text. Parse the document, split it into chunks, index the chunks, retrieve the relevant ones, pass them to the model, get the answer. It is a logical, consolidated pipeline supported by dozens of mature frameworks. But it hides a structural problem that anyone who has ever built a RAG system on real-world documents knows well: the world is not made of text alone. Tables, charts, complex layouts, images with embedded data, PDFs with elaborate formatting — all of this gets systematically degraded or lost in the text extraction process. And when the information you are looking for lives inside a table or a chart, the traditional RAG system fails. UC Berkeley decided to address the problem at its root, with an approach that completely flips the paradigm: instead of extracting text from pages, PixelRAG treats every page as an image — and searches for information by looking, not reading.

PixelRAG is a next-generation RAG system developed and released by the University of California, Berkeley that replaces the traditional text extraction pipeline with an approach based on screenshot rendering and visual retrieval. The core principle is elegant in its simplicity: rather than parsing the document and converting it to text — inevitably losing structural, graphical and tabular information along the way — PixelRAG renders each page of the document as a high-resolution image and works directly on those images for both retrieval and answering. The direct consequence is that PixelRAG sees exactly what a human sees when looking at the page: tables with their visual structure intact, charts with their values, complex layouts with their visual hierarchy, images with embedded text, mathematical formulas, diagrams. Nothing is lost in the parsing process.

PixelRAG's architecture differs from the traditional RAG pipeline at every stage. The first stage is page rendering as screenshot tiles: instead of extracting text with parsers like PyPDF2 or pdfminer, PixelRAG uses a rendering engine to convert every page into a high-resolution image, high enough to preserve every visually relevant detail — readable text, table values, chart labels. Pages with particularly dense layouts are split into overlapping tiles, so no visually relevant portion is clipped at the edges. This approach works on any type of visually renderable document: native PDFs, scanned PDFs, web pages, PowerPoint presentations, Excel sheets, Word documents.

The second stage is visual indexing with multimodal embeddings. Once the screenshots are generated, PixelRAG indexes them using models capable of representing images as vectors in the same semantic space used for text. This is the technical core of the innovation: instead of creating embeddings of text chunks, PixelRAG creates embeddings of entire page images. These embeddings capture not only the semantic content of the text on the page, but also the visual structure, the layout, the relative position of elements — information that traditional text embeddings completely miss. The models used, based on architectures like CLIP and its more recent variants, are optimized so that text queries can be compared to images in a semantically meaningful way: a question like "how many shots on target did Inter Milan record in the 2010 UCL final?" produces an embedding that gets close to the visually relevant pages, even if those pages contain tables of numbers rather than running text.

The third stage is visual retrieval: when a query arrives, PixelRAG converts it into an embedding and searches for the most visually relevant pages in the index, exactly the way a traditional RAG system searches for the most relevant text chunks, but operating on image representations instead of text ones. The result is a set of page screenshots — not text fragments — that the system has identified as potentially containing the answer. The fourth stage is the multimodal Reader Model: the retrieved pages are passed to a Vision Language Model capable of seeing the images and answering questions about them. The model visually analyzes the pages, locates the relevant information within the page structure and generates the final answer. This is where PixelRAG shines compared to traditional systems: the Reader Model can read a table by seeing it as a table — not as a poorly extracted sequence of text — and pull specific values with a precision that text parsers cannot guarantee.

The diagram in Berkeley's paper illustrates PixelRAG's advantage perfectly with a concrete example. The question: "How many shots on target did Inter Milan record in the 2010 UCL final against Bayern?". With traditional RAG, the system extracts the text from the page, splits it into chunks, retrieves the relevant chunk — but the answer lived inside a table, and the table lost its structure during extraction. The retrieved text chunk is a confused sequence of numbers and words that the model cannot interpret correctly. The result: "The answer cannot be determined from the given context." A wrong answer, even though the information was in the document. With PixelRAG, the system retrieves the screenshot of the page containing the table, the Reader Model sees it visually, reads the structure the way a human would, identifies the cell corresponding to Inter's shots on target and answers with precision: "7". This is not an artificial edge case: it is exactly the kind of situation that anyone who has worked with RAG systems on corporate, scientific or legal documents has hit dozens of times.

The experimental results published in the paper show PixelRAG outperforming text-based RAG systems on every benchmark that involves documents with structured visual content. On document Visual Question Answering benchmarks — DocVQA and similar — PixelRAG beats text-based pipelines by significant margins, with especially large improvements on questions that require reading tables and charts. On scanned documents the advantage is even sharper: traditional RAG systems depend on OCR quality, and OCR on medium-to-low quality documents produces error-ridden text that heavily degrades retrieval and answering. PixelRAG is structurally immune to the problem because it works directly on the image. Documents with multi-column layouts, sidebars, footnotes and marginal annotations — all scenarios where text parsers behave unpredictably — are handled natively without degradation. The same is true for web content: instead of parsing HTML while stripping tags, CSS and JavaScript, PixelRAG renders the page the way a browser does and works on the resulting image.

PixelRAG is not a universally superior solution in every scenario, but it is specifically transformative for categories of documents where structured visual content is critical. The use cases where adoption produces the clearest benefits are financial documents and corporate reports (balance sheets, financial statements, annual reports dense with numerical tables); legal documents and contracts with tables of terms, payment schedules and structured attachments; scientific literature with tables of results, performance charts and figures with data; PowerPoint or PDF presentations, notoriously hard for traditional RAG systems because the text is fragmented and organized visually rather than semantically; technical manuals with diagrams, wiring schematics and specification tables; web dashboards and sites with structured data where HTML extraction would have made the structure unreadable.

An honest evaluation also requires discussing the limits. Computational cost is higher: working with images is more expensive than working with text, image embeddings are larger, page rendering requires extra resources, and the Vision Language Models used as Readers are generally heavier than equivalent text-only models. Retrieval latency is higher than with equivalently sized text embeddings, a relevant issue for real-time applications. For documents made exclusively of running text with no structured visual elements — novels, transcripts, plain text — the advantage disappears and the extra cost is not justified: in these scenarios a traditional text RAG system is still the right choice. PixelRAG also depends on rendering fidelity: on very old, poorly formatted or non-standard-encoded documents it can struggle. Finally, since it is a semantic search system on visual embeddings, it is less suited to searches requiring exact string matches (codes, IDs, precise numbers), where a full-text text index remains superior.

Berkeley's paper was published with open source code available on GitHub, a choice that immediately drew the attention of the AI developer community and racked up hundreds of stars in the first hours after publication. The main dependencies are mature open source components: a document rendering engine (pdf2image, Playwright for the web), a multimodal embedding model from the CLIP or ColPali family, a standard vector store (Qdrant, Weaviate, LanceDB) and a Vision Language Model as the Reader (Claude, GPT-4o, Qwen2-VL or self-hosted LLaVA). Integration into an existing pipeline is designed to be as simple as possible: wherever today you have a text parser followed by an embedding model, you swap in PixelRAG's renderer followed by the multimodal embedding. The rest of the pipeline — vector store, orchestration, reader prompt — stays untouched. For anyone building RAG products on business documents, PixelRAG is not yet a drop-in replacement for every existing pipeline, but it is certainly the architectural pattern to evaluate for every new project where documents contain tables, charts or non-trivial layouts. It is one of those releases that, six months from now, will likely have changed the very definition of "best practice" for RAG.