A developer gets Qwen3.5 running inside an FPGA, with no GPU, though at only 2.5 tokens per second

By Zendoric · October 5, 2026.

A developer who signs as Nero7991 has published on GitHub an inference engine for language models that uses neither GPU nor CPU: all the computation happens directly in the reconfigurable circuits of an FPGA (Field-Programmable Gate Array, a chip whose internal wiring can be redefined after it has been manufactured). The repository, called llm.vhdl, implements in VHDL —a language for describing digital circuits, not code that runs on a processor— the architecture of Qwen3.5, Alibaba's family of open models: from quantizing the weights to INT4 (4 bits per parameter, to save memory and bandwidth) to the attention mechanism, by way of the sequencer that decides, token by token, which block of the model is activated.

Today the design works, genuinely, on a niche FPGA card: the SQRL FK33, with a Xilinx UltraScale+ chip and 8 GiB of HBM (High Bandwidth Memory, stacked very-high-bandwidth memory soldered next to the chip). According to the project's documentation, a single card holds the complete 9-billion-parameter model, and two cards connected in a pipeline —each with a portion of the layers— double the performance of a single card, up to around 2.5 tokens per second measured on real silicon. The outputs are validated layer by layer against a reference run in llama.cpp, and the author describes the tokenizer and the chat template as “bit-exact” against that reference.

The project itself admits that it is running at half throttle. The FK33 cards have only been tested at 75 MHz; each block of the design, synthesized separately, closes between 150 and 207 MHz on that same card, so there is still untapped headroom on speed. And the most ambitious goal —a 27-billion-parameter version, which the repository labels Qwen3.8, split across two chips on a different card, the Jungle Cat— does not even generate text yet: the two chips can be programmed and detected separately, but the high-speed link that should join them does not come up because a 156.25 MHz clock oscillator has yet to be soldered onto the board, and there is no fast path to load the roughly 13.5 GB of weights of the large model, because those cards have no PCIe connection and depend on a JTAG/Ethernet bridge with a capacity of barely 1 MB/s.

None of this competes with a GPU on speed: 2.5 tokens per second is, at a minimum, an order of magnitude slower than any modern graphics card serving the same quantized model. And that is precisely what strikes us as noteworthy: not the speed, but that a single developer, with no access to a chip fab or any commercial agreement, has been able to take the architecture of an open model —including Gated DeltaNet, the linear attention mechanism with which, according to the project's documentation, Qwen3.5 makes processing cheaper in most of its layers— and translate it, block by block, into a dedicated circuit that runs no software: it is the model, wired directly into reconfigurable silicon.

That possibility exists only because Qwen's weights and architecture are open. In general, AI inference in 2026 is concentrated in Nvidia GPUs and in the proprietary accelerators of the big cloud providers, a field where experimenting with hardware demands budgets that almost no one outside a handful of companies has. Niche projects like this one —with 6 stars on GitHub and zero comments on Hacker News, to make clear that we are talking about a technical curiosity and not a product— are the laboratory of those who do not have that budget: slow, built with a soldering iron and with more than fourteen hundred commits behind them, but the same kind of artisanal work from which, historically, the dedicated accelerators that do reach production end up emerging, once someone shows that the circuit works and reproduces the exact result of the reference model. Our underlying thesis —that abundant compute will come not only from mega data centers, but also from alternative routes around the GPU monopoly— finds here a tiny but real example of where it might start to be built.