Colibrì: the tiny C engine that runs GLM 5.2's 744 billion parameters on your home PC without a GPU

Colibrì's tagline is disarmingly bold: «Tiny engine, immense model». And that is exactly what it delivers: running GLM 5.2, a Mixture of Experts model with 744 billion parameters, on a normal home PC with roughly 25 GB of RAM — no dedicated GPU, no cloud, no external dependencies. For years the fundamental assumption of the AI world was that frontier models required enterprise hardware — GPU clusters costing hundreds of thousands of euros, rack servers in climate-controlled data centers, cloud infrastructure with significant monthly bills. A 744-billion-parameter model seemed categorically out of reach for any consumer hardware. Colibrì proves that assumption wrong, or at least wrong in the terms it was framed. And it does so with a technical approach of rare elegance: an engine written in pure C, with zero external dependencies, that uses a disk-streaming technique for experts to keep in memory only the fraction of the model needed at any given moment.

To understand Colibrì's innovation you first have to understand why large models normally need so much memory, and why the Mixture of Experts architecture is the key that unlocks everything. A traditional language model — a dense architecture like GPT or Llama — keeps all its parameters active during every single forward pass. A 70-billion-parameter model in float16 occupies about 140 GB of memory, and all 140 GB must be simultaneously accessible during inference. No part of the model can be skipped for any token. That was the constraint that made large models inaccessible on consumer hardware: there was no way to page the weights without latency becoming unacceptable.

GLM 5.2 is not a dense model: it is a Mixture of Experts. In an MoE architecture the model is made up of many experts — specialized sub-networks — and for each token processed only a small subset of these experts is activated, selected by an intelligent routing mechanism. GLM 5.2 has 744 billion total parameters, but only activates a fraction of them per token, typically the equivalent of a 20-50-billion-parameter active model. The other experts stay unused for that particular token. This architectural property has a revolutionary implication for local inference: not all weights need to be in memory at once. If you manage to load from disk only the experts required for each token, fast enough not to block generation, you can run a huge model with a fraction of the memory that would seem necessary. That is exactly what Colibrì does.

The core mechanism of Colibrì is what its developers call «streaming experts from disk»: a technique that leverages the MoE architecture to dynamically load only the needed experts from mass storage (NVMe SSD) into system RAM, precisely when they are required. At startup Colibrì loads into RAM only the components shared across every token — embedding layer, non-MoE attention layers, routing mechanism — which are relatively lightweight compared to the total model mass. The result, as shown in Simone Rizzo's video, is an initial RAM footprint of just 9.9 GB, an infinitesimal fraction of the 140-200 GB the model would require in traditional dense form.

During generation of each token the GLM 5.2 routing mechanism determines which experts are needed to process the current token. Colibrì intercepts this decision, checks which experts are already in the RAM cache, loads the missing ones from disk — directly from NVMe storage where they have been pre-arranged in a format optimized for fast sequential reads — runs the forward pass on the selected experts and manages the cache in LRU fashion, keeping the most recently used experts in RAM and evicting the least used ones as the cache approaches its limit. The critical point is disk read speed: if loading is too slow, generation stalls waiting for data. Colibrì addresses this with three key optimizations. Predictive prefetching: based on routing patterns observed in previous tokens, the engine starts loading in advance the experts the router will likely select next. Optimized binary format: weights are stored on disk in a compact layout designed for fast sequential reads on NVMe, minimizing I/O operations. Int4 quantization: parameters are stored in 4 bits instead of the standard 16 or 32, reducing by 4-8x both the data to read from disk and the RAM needed to keep experts cached.

One of the most interesting technical choices in Colibrì is the decision to write it entirely in pure C, with no external dependencies: no PyTorch, no CUDA, no MKL, no framework of any kind. It looks like technical purism, but the practical implications are very concrete. Absolute portability: a pure-C program with zero dependencies runs on any operating system and any hardware architecture without changes — Linux, macOS, Windows, ARM, x86, RISC-V — with no drivers, frameworks or compatibility libraries to install. That drops the installation barrier to nearly zero, a huge advantage over llama.cpp or Ollama which, while excellent, still require non-trivial setup for non-technical users. Full I/O control: streaming experts from disk is extremely latency-sensitive, and frameworks like PyTorch add unavoidable overhead. By writing in C, Colibrì can use low-level system calls such as mmap, pread and io_uring on Linux to optimize I/O in ways no high-level framework would allow. Minimal footprint: the compiled binary weighs a few megabytes, needs no Python environment, downloads no gigabytes of dependencies — you download it, compile with a single command, and run.

The most important practical question for anyone who wants to use Colibrì is: how fast is it, how good is it? Colibrì loads GLM 5.2 in about 32 seconds on standard consumer hardware: for a 744-billion-parameter model that is remarkable — most equivalent systems would take minutes or tens of minutes. Generation speed depends critically on NVMe storage: on mid-range SSDs (sequential reads around 3-5 GB/s) you get 2-5 tokens per second; on high-end NVMe PCIe 4.0/5.0 SSDs you climb to 5-12 tokens per second; on systems with 64 GB or more of RAM, which allow a much larger expert cache and reduce disk reads, you reach 15-25 tokens per second. These are lower speeds than those you can get with dedicated GPUs, but for many non-real-time use cases — document analysis, text generation, coding assistance — they are perfectly usable. And they are obtained on hardware anyone has or can afford, not on servers costing tens of thousands of euros. Thanks to int4 quantization the output quality shows minimal degradation compared to the full-precision model, under 3-5% on standard benchmarks: in day-to-day practice the difference is only noticeable on tasks that demand extreme numerical precision.

System requirements are more accessible than you might expect. Minimum workable configuration: 25 GB of available RAM (32 GB installed recommended for operational headroom), NVMe SSD with at least 400 GB free for quantized weights and sequential reads above 2 GB/s, any modern 64-bit CPU (x86_64 or ARM64), no GPU required, Linux, macOS or Windows as the operating system. Optimal configuration for speed: 64 GB or more of RAM, NVMe SSD PCIe 4.0 or 5.0 with reads above 7 GB/s, processor with many cores to parallelize I/O operations. The real bottleneck of Colibrì is neither RAM nor CPU: it is storage speed. Anyone with a slow NVMe SSD — under 1 GB/s in sequential reads — will find generation frustrating. Anyone with a fast SSD will have a far more satisfying experience. Before installing Colibrì it is worth checking your storage specs.

How does Colibrì compare to other widely used local inference tools? Compared to Ollama, which is the easiest tool to use for local AI but is built on llama.cpp and does not natively support MoE disk streaming, Colibrì offers something Ollama cannot: huge MoE models on limited hardware. For manageable model sizes — up to 70 billion dense parameters — Ollama remains the most convenient choice. Compared to llama.cpp, which is the pioneer of local inference in C/C++ with support for a vast range of models and more mature hardware optimizations, Colibrì is specifically optimized for MoE disk streaming: for GLM 5.2 in particular it produces better results. Compared to Salvatore Sanfilippo's DwarfStar, which tackles a similar problem but with an approach oriented toward consumer GPUs with RAM offloading and custom CUDA kernels, the two tools are complementary rather than competing: DwarfStar excels on systems with powerful GPUs, Colibrì excels on CPU-only systems with good NVMe storage.

The implications of Colibrì go well beyond the single project and trace a clear direction for local AI in the coming years. First: the democratization of frontier models — if a 744-billion-parameter model runs on a home PC, the gap between what an enterprise user with unlimited budget can do and what a single developer or small business can do shrinks dramatically. Second: data sovereignty — running a frontier model locally means never sending a byte to third-party services, a requirement that for regulated sectors such as healthcare, legal, finance and European public administration is not an advantage but an obligation. Third: economics — those who automate repetitive AI workloads at home fully avoid recurring cloud API costs and replace them with a one-off investment in an NVMe SSD and RAM. Fourth: resilience — an AI infrastructure that depends on no cloud provider keeps working when providers change prices, deprecate models or suffer outages.

What should you do today if you are an SMB or a developer trying to understand how to integrate this technology? Practical advice in three steps. First: check your hardware — open the specs of your main PC and check installed RAM and sequential read speed of the SSD. If you have 32 GB of RAM and an NVMe PCIe 3.0 or higher you are already in the useful window to experiment with Colibrì. Second: identify a non-real-time use case — document analysis, report generation, meeting summaries, ticket categorization, code review — where 5-15 tokens per second are more than enough and where data sovereignty has real value. Third: make an honest comparison between cloud and local costs on your real workload, including latency, privacy, vendor lock-in and price predictability. If you are evaluating how to bring part of your AI infrastructure in-house, or how to pragmatically combine a local model like GLM 5.2 with cloud APIs for tasks that require maximum capability, book a discovery call: we will analyze your current stack together and design a hybrid strategy that optimizes cost, control and performance. Colibrì is not just a curious technical project: it is a clear signal that the era of serious local AI has finally begun.