How to read this. This is my understanding of a machine I don't own and can't yet benchmark, built from public materials, three of the company's patents, and the architecture literature — so every NextSilicon performance number below is a vendor claim, marked as such, and every place the vendor discloses nothing I say so and reason from first principles. Two of my own working assumptions turned out wrong; I've left the corrections in, because watching a mental model get fixed is more useful than pretending it arrived correct.
What I was trying to understand
The pitch for a reconfigurable dataflow processor is easy to state and hard to believe: compile your existing C/Fortran/CUDA once, and a grid of ALUs will physically rewire itself at runtime to run your hot loops — continuously, from live telemetry, without you touching the code. NextSilicon's Maverick-2 makes exactly this claim, and my first honest reaction was that it sounded like magic.
The way I get a new architecture to stop sounding like magic is to keep asking one question about each part of it: what does this remind me of? A machine that seems alien usually turns out to be a familiar idea wearing new clothes, and the fastest way to really understand it — and to tell what's genuinely new from what's just renamed — is to find the thing you already know that it's standing on. So I broke the fabric into the four questions I couldn't answer on sight:
- How does it run so many "threads"? (it reminded me of a barrel processor)
- How does it touch memory fast enough to feed the grid? (that's decoupled access/execute)
- How does it survive branches — dataflow's classic weakness? (if-conversion, plus a punt to a normal CPU)
- What does it cost to rewire, and when does that pay off? (it's a tracing JIT, in hardware)
Each of those connections is a section below. And once I had all four handholds, the machine resolved into a single sentence I could actually hold in my head: Maverick-2 is a Little's-Law latency-tolerance machine whose one genuinely new trick is the telemetry-driven runtime remap; everything else has an ancestor I already knew. That isn't a knock — welding known ideas into a chip that runs unmodified HPC code is the hard part. It's the opposite: naming the ancestors is how I convinced myself I understood it, instead of just repeating the brochure.
1. The threads — the connection that made it click was a barrel processor
The launch talk's headline is arresting: a CPU core runs ~2 threads, a GPU 32–64 per warp, but "a mill core can support hundreds of threads at once." What is a "thread" here, and how does one core hold hundreds?
The everyday picture that unlocked it for me is a short-order cook. A bad cook puts bread in the toaster and stands there watching it — one order at a time, most of the time spent waiting. A good cook drops the toast, and while it's toasting starts the next order's eggs, comes back when the toast pops. The cook isn't faster at any one task; the cook is never idle. That's the whole idea, and in computer architecture it has a name — a barrel processor — and once I saw that name in NextSilicon's patent, the "hundreds of threads" number stopped being marketing and became arithmetic.
I pulled the patent that the reference diagram cites, US11875153B1, expecting it to be the famous self-optimizing-telemetry patent. It isn't — and that correction is the first grounded finding.
US11875153B1 — "Executing concurrent threads on a reconfigurable processing grid" (inventors Elad Raz, the CEO, and Ilan Tayari; filed 2023-07-05, granted 2024-01-16). Its independent claims describe hardware fine-grained multithreading:
- Each concurrent thread is a dataflow graph of operations; the grid has a context storage holding many such threads' runtime state.
- When a thread stalls on a long-latency operation on some logical element, the hardware saves that thread's context and runs a second thread's operation on the very same logical element.
- When the stalled operation completes, it restores the context and resumes.
That is context-switch-on-stall to keep the ALU busy — Little's Law implemented in silicon. The reason a mill core carries "hundreds of threads" is a hardware context store that interleaves independent data items onto shared cells the instant any one of them waits on memory. No instruction stream, no OS threads — just saved dataflow-graph contexts swapped onto physical cells.
The honest framing: this is a barrel processor — the idea behind the Denelcor HEP, the Tera/Cray MTA, GPU warp switching, Sun's Niagara. Switch to another ready context on a stall. NextSilicon's twist is doing it on a spatial dataflow grid, at dataflow-graph granularity, rather than on a scalar pipeline. Old idea (latency-hiding via multithreading); new substrate (per-thread graphs on shared reconfigurable cells). Credit both halves.
Correction logged. My earlier notes guessed US11875153B1 was "the telemetry patent." Wrong. Self-optimization is covered by a separate patent — US10817309B2, "Runtime optimization of configurable hardware" (Elad Raz, 2017 priority, NextSilicon's founding patent), pulled and covered in §5 below. The patent family here — US12340221B2, pending US20250321741A1, plus EP/CN/JP counterparts — is the multithreading line, not the optimization line.
2. The memory path — decoupled, distributed address generation
A throughput-via-space fabric is worthless if memory can't feed it. NextSilicon's answer is the Memory Entry Point (MEP). From the one substantive briefing on it: an MEP "generates memory access requests on the memory bus, then directs completion responses to the reservation station." Crucially, each MEP has its own address translation — the ICA uses MMUs and a TLB, but "sparingly, and only when an ALU calls for specific data."
So the von-Neumann memory plane isn't merely shrunk to a side-column (an earlier insight of mine that's true but incomplete) — it's distributed and demand-driven: many small per-MEP translation caches, each serving one access pattern, instead of one central MMU funnel.
None of this is new, and naming the lineage is the honest move:
- Decoupled Access/Execute (Jim Smith, ISCA 1982): split a program into an access stream (address generation + loads) and an execute stream, coupled by FIFOs, so the access side runs ahead and hides latency. MEP-feeds-reservation-station is DAE in silicon — MEP = access engine, ALU = execute engine, staging buffer = FIFO.
- Plasticine (ISCA 2017) put a scalar datapath dedicated to address computation in its Pattern Memory Unit, feeding the compute pipeline. Its direct descendant, SambaNova, ships the same idea as the AGCU (Address Generation & Coalescing Unit). MEP is that decoupled address generator under a proprietary name.
Why this wins the benchmark it's built to win. Memory-level parallelism is capped by how
many requests you keep outstanding — Little's Law again: sustained BW = outstanding
requests ÷ latency. On a CPU the cap is the MSHR file (miss-status holding registers),
which is content-associative and area-hungry, so CPUs carry only ~12–64 of them. Tiny
MLP → random-access throughput runs ~10× below peak bandwidth → latency-bound. A GPU beats a
CPU ~10× here by manufacturing misses through thread oversubscription. NextSilicon's
bet is to get MLP from many independent per-MEP address generators instead — spatial
MLP, not thread-oversubscription MLP. That is the architectural reason the GUPS 32.6 @
460 W figure (≈22× a CPU, ≈6× a GPU — vendor claim) is even plausible: GUPS is a pure
random-access benchmark, i.e. a near-direct MLP/MSHR test.
The edge of the win, stated honestly. Decoupling only runs ahead if the next address
is known without waiting for the previous load's result. Pointer-chasing (a = A[a])
collapses it — Loss-of-Decoupling — and even MEPs stall. So the fabric beats
random-but-independent access (GUPS), not a true serial pointer-chase. That's the precise
boundary of the "low-arithmetic-intensity win."
Undisclosed, flagged. The actual outstanding-request depth per MEP or per chip — the real MLP number — is not published. GUPS 32.6 is the only empirical proxy, and it's a vendor claim. "Memory Entry Points" is itself a briefing-level term. There is a dedicated NextSilicon memory patent — US11269526B2, "Interconnected memory grid with bypassable units" (2022) — which is the natural place the MEP/outstanding-request mechanism is claimed; it's the next pull if we want this grounded to claim level. For now the grounded claim is the distributed per-MEP translation; the depth is inference.
3. Control flow — the fabric sidesteps branches, it doesn't beat them
Dataflow machines love long, predictable, straight-line loops and hate data-dependent branches. So how does Maverick-2 handle branchy code? NextSilicon's honest answer is: it mostly doesn't — it routes around the problem.
Their own words: "There is no speculation or prediction, just fetching." The fabric doesn't outdo a CPU's branch predictor; it avoids the mispredict-and-rollback machinery entirely (whose revert cost, they correctly note, is a real CPU tax). Genuinely branchy, control-dominated code is instead offloaded to Arbel, the companion RISC-V core, whose "Elite TAGE branch predictor" is the serial half of the pairing. This is the same hot/cold split the fat binary makes: hot straight-line loops to the fabric, branchy serial code to the CPU.
A naming correction. I had earlier called the fabric's dispatch front-end "Tomasulo-style." The public material describes the reservation station only as a data-staging buffer ("temporarily stages data before an ALU calls for it"), and the company explicitly disclaims speculation. There is no evidence of Tomasulo-style speculative out-of-order renaming. The reservation station is a dataflow operand-gathering buffer — fire when inputs are present — which is the static dataflow firing rule, not Tomasulo.
What about branches inside a mapped hot region? Here the fabric patent (US10817344B2) gives us a grounded primitive that vendor marketing didn't: its logical elements can "conditionally execute based on comparisons," and data is steered between cells through configurable routing junctions. So the on-grid branch mechanism is predicated / conditional-LE execution plus junction steering — the if-conversion family, not a mispredicting predictor. The full spec (predication vs steering, nested branches, cost) is still undisclosed, but the literature gives us the menu it must be drawn from:
- If-conversion / predication — map both sides of the branch onto the fabric, run both, and a select/φ node picks the live value at the merge. The dominant CGRA approach — and the source of the structural cost: the not-taken cells sit idle, wasting silicon and power. This is precisely why branchy code underutilizes a spatial fabric. Refinements exist (partial vs full predication; state-based full predication parks untaken cells with sleep/wake ops instead of re-nullifying them each pass), but the waste is fundamental.
- Steer (ρ) vs select (φ) — WaveScalar's operators: steer routes a token down only the taken path (avoiding the dead computation, unlike predication) at the cost of a data-dependent route; select merges. Tagged-token dataflow (Arvind / Monsoon) did the same with switch/gate actors — control is data.
- Block-atomic predication (TRIPS/EDGE) — inside a ≤128-instruction hyperblock, all branches are if-converted; prediction happens only at block granularity, reducing how often a control decision must be made — a latency-tolerance trick for distributed execution.
The synthesis: a spatial fabric earns its efficiency by amortizing a fixed place-and-route over long, predictable streaming. Data-dependent control is its natural enemy — it either wastes cells (predication) or churns the mapping (re-steer/re-map). NextSilicon's answer is systemic, not a fabric trick: telemetry watches which paths actually run and remaps the hot ones; the genuinely branchy code runs on Arbel. Consistent, honest, and exactly why the machine is pitched at loop-heavy HPC kernels, not irregular control code.
4. Reconfiguration — there's a roofline for this too
The most beautiful result of this investigation is that the "when does rewiring pay off?" question has a roofline, structurally identical to the arithmetic-intensity roofline this project already leans on.
From "The Configuration Wall" (ASPLOS '26, Van Delm et al.): define an
operation-to-configuration intensity I_OC = useful ops / configuration bytes
and plot it against a configuration-bandwidth ceiling BW_config (bytes/cycle the host
can push into the fabric). Below the ridge point I_OC = P_peak / BW_config the kernel is
configuration-bound — you're paying for place-and-route you never amortize; above it,
you're compute- or memory-bound as usual. In their worked Gemmini example a 64³ matmul
attains only ~41% of peak (26.8% with packing overhead) purely because it hits this wall.
This is the formal name for the "configuration residency" second-intensity this project
had been gesturing at.
The granularity ladder for load cost (grounded, orders of magnitude apart):
| Substrate | Reconfiguration granularity | Load cost | Price paid |
|---|---|---|---|
| FPGA | bit-level bitstream | ms–seconds (94 KB partial ≈ 209 ms over JTAG) | slowest switch |
| CGRA (multi-context) | word-level config, per-cycle switch | single cycle once resident | config memory ≈ 40% of chip power in one study |
| NextSilicon Mill Core | pre-built fabric image, swapped in | "nanoseconds" (claimed, no number) | on-chip image storage |
Coarse granularity is why a CGRA context load is orders cheaper than an FPGA bitstream; NextSilicon claims a further step down with pre-built images. The nanosecond claim isn't free — it's bought with the on-chip image store.
Why a configuration gets reused enough to pay for itself: CGRAs pipeline inner loops via
modulo scheduling — prologue / steady-state kernel / epilogue, where the kernel config
repeats every II (initiation interval) cycles for the whole iteration count. Loops
with many iterations are dominated by II, so the config built once is reused thousands of
times. Loops with few iterations can't amortize — the same break-even, seen from the loop
side.
And the cleanest analogy — which is NextSilicon's own framing — is the JIT compiler. They call the ICA "a JIT compiler for hardware," and the fit is exact: tracing JITs use hot-path counters (incremented on backward branches); once a counter crosses a warm-up threshold (Dynamo used 50; typical 100–1000) the region is "hot" and gets compiled. Fixed build cost, payoff proportional to subsequent executions. JITs hide the compile on a separate thread — the software analog of concurrent, shadow-register reconfiguration and of NextSilicon building mill cores "in the background without slowing the workload." So the telemetry loop is a hardware tracing-JIT, and its unpublished "hotness threshold" is the direct analog of Dynamo's 50.
Undisclosed, flagged. The Mill-Core load time in nanoseconds, the image size in bytes, and the hotness threshold are all vendor claims with no published values. The Configuration Wall roofline is exactly the yardstick to hold the "ns reconfig" claim against — which is how you'd test it in an independent evaluation.
5. The telemetry — the founding patents, and where disclosure actually stops
The one genuinely novel layer — the runtime remap that welds the other three mechanisms together — is a pair of 2017 patents granted the same day. The first is US10817309B2, "Runtime optimization of configurable hardware" (Elad Raz, sole inventor; priority 2017-08-03, granted 2020-10-27) — the patent NextSilicon was founded on. It grounds the tracing-JIT reading of §4; its companion fabric patent (below) grounds the telemetry hardware. Together they push "where disclosure stops" much further than either alone.
Its claimed method is spare and telling. Paraphrasing claim 1: receive many function calls; identify a pattern among them — the worked example is "two or more functions occurring in association above a predetermined threshold"; then manipulate the grid to compute those functions on the fabric. Two details matter: the threshold is dynamically adjusted "based on statistical analysis," and the manipulation is "in real-time, simultaneous to the operation."
That is a hot-path detector with an adaptive warm-up threshold — structurally a tracing JIT, where the "trace" is a spatial mill-core mapping instead of a compiled code trace. So the JIT-for-hardware framing in §4 isn't my analogy anymore; it's the founding patent's own logic. Detect hot co-occurring calls above a self-tuning threshold, reconfigure live — that is the ICA in one sentence.
This optimization patent works at the call-pattern abstraction — it says what is detected and that reconfiguration happens live, but on its own it doesn't say how the telemetry is gathered. Its companion 2017 patent does — and this is the payoff of one more pull.
The fabric patent — US10817344B2, "Directed and interconnected grid dataflow architecture" (Raz + Tayari, priority 2017-09-13, granted the same day as the optimization patent) — describes the grid itself: logical elements (unary/binary/ternary ops, lookup tables, conditional execution) wired by configurable routing junctions, with a compute DAG projected onto them and flow-control (Ack/Pause) semantics synchronizing operand movement. And crucially, it claims the telemetry mechanism verbatim:
"Hardware counters at LEs, ports, junctions, or the interconnect collect runtime metrics — path frequency, flow-control parameters — [and] the grid determines likely compute paths statistically and reconfigures itself at runtime" by relocating operations near critical resources.
That is, almost word-for-word, the first-principles reconstruction I'd flagged as a guess: distributed passive counters on cells, ports, and links, measuring path frequency and backpressure, feeding a runtime remap that pulls hot operations toward memory. Pulling the fabric patent turned my reconstruction into a grounded reading. What's still genuinely undisclosed is narrower than before: the explicit no-interrupt / zero-overhead guarantee, which core runs the optimizer (the control-plane wiring), and the counter widths and sampling rate. The existence and placement of the counters is no longer in question.
With this, the NextSilicon patent map has four grounded pillars (and the memory line finally located):
| Patent | Pillar | Priority |
|---|---|---|
| US10817309B2 | runtime-optimization method (telemetry → hot-pattern → reconfigure) | 2017-08-03 |
| US10817344B2 | grid dataflow architecture (LEs + junctions + ports + counters) | 2017-09-13 |
| US11875153B1 | concurrent-threads hardware multithreading | 2023-07-05 |
| US11269526B2 | "Interconnected memory grid with bypassable units" — the likely MEP patent (not yet pulled) | ~2020 |
The two founding patents (2017, granted the same day) already claim a flow-controlled dataflow grid with distributed hardware telemetry counters and a hot-pattern-detection → runtime-reconfiguration method. So NextSilicon's "self-optimizing hardware" isn't a later marketing layer — it's the 2017 core IP, and Maverick-2 is its productization (Arbel, HBM3e, the mill-core naming, the reservation-station front-end).
What I understand now, in five connections
Here's where the machine landed for me — each row is the familiar thing that made the new thing make sense:
- Threads (Q2): US11875153B1 is hardware fine-grained multithreading — context-switch-on-stall on a dataflow grid, the silicon behind "hundreds of threads per mill core." A barrel processor on a spatial fabric — the short-order cook. Not the telemetry patent.
- Memory (Q4): MEPs are decoupled, distributed per-MEP address generators (the DAE → Plasticine → SambaNova-AGCU lineage). MLP from independent address engines, not thread oversubscription — the GUPS lever. Depth undisclosed.
- Branches (Q6): the fabric sidesteps them (no speculation), offloads branchy code to Arbel, and remaps hot paths from telemetry. On-grid encoding grounded to a primitive: conditional-LE execution + junction steering (if-conversion family).
- Reconfiguration (Q7): rewiring has its own roofline (the Configuration Wall); amortized by modulo-scheduled reuse; it's a hardware tracing-JIT. The nanosecond, size, and threshold constants are undisclosed.
- Telemetry (the novelty): two 2017 founding patents — US10817309B2 (hot-call-pattern above a dynamic threshold → live reconfiguration; the tracing-JIT logic) and US10817344B2 (the grid's distributed hardware counters measuring path frequency + backpressure). Only the no-interrupt guarantee and control-plane wiring remain undisclosed.
How I'd put my understanding in one sentence: every part of this machine turned out to be something I already knew wearing new clothes — a barrel processor for the threads, decoupled access/execute for the memory, if-conversion for the branches, a tracing JIT for the rewiring — and the one genuinely new thing is that NextSilicon welded them together under a single telemetry-driven runtime remap and shipped it running unmodified HPC code. That's the value of the exercise: I don't understand a new architecture until I can point at the old idea under each piece, name the one part that's actually new, and be honest about the constants nobody's published yet. This is where my understanding sits today — if you know this hardware better than I do, I'd like to hear where I've got it wrong.
Sourcing: US patents US11875153B1, US10817309B2, and US10817344B2 (Google Patents); The Next Platform Maverick-2 deep-dive
(2025-10-22); NextSilicon launch materials; the CGRA / dataflow architecture literature
(Smith 1982 DAE; Plasticine ISCA 2017; SambaNova SN40L; TRIPS/EDGE; WaveScalar; the DATE
2013 SFP and ASPLOS '26 "Configuration Wall" papers). Full grounding and per-claim
flags live in dataflow-architecture/LOG.md §"Inside the fabric" and QUESTIONS.md
(Q2/Q4/Q6/Q7).