Your model may already ship a speculative decoder. Almost nothing loads it.

Speculative decoding usually means running two models. A small draft model guesses the next few tokens, the real model checks them in one pass, and you keep whatever survives. It works, but on a Mac the draft model competes for the memory you wanted to spend on the real one.

Several open models already ship the draft model inside themselves. DeepSeek, GLM and MiMo all publish a multi-token prediction head, trained alongside the body, weighing a few hundred tensors at most. It reuses the body’s hidden states, so it costs almost nothing to run. Then most runtimes load the checkpoint, skip those tensors, and decode one token at a time anyway.

MTPLX is Youssof Altoukhi’s project, and it is the one runtime I know of on Apple Silicon that treats those heads as a first-class feature. I have been contributing to it for the past week, mostly on the part that finds the head in the first place.

Why it is interesting on a Mac specifically

The head is part of the checkpoint you already loaded, so nothing extra occupies your memory. On a 16 GB machine that decides whether speculation is available to you at all.

Acceptance uses the Leviathan and Chen rejection sampling result with residual correction, so at temperature 0.6, top_p 0.95 you get what plain decoding would have given you, only sooner. That is worth checking in any local speculative decoder you try, because several of them assume greedy sampling and change what the model would have said at the settings you actually use.

Draft depth is worth different amounts on an M1 Pro and an M5 Max, since the trade depends on memory bandwidth and on how long your Mac holds its clocks. mtplx tune runs the real model at each depth with the fans pinned, keeps plain decoding as the baseline, and saves a depth only if it beat that baseline, reporting it plainly when none of them did. The project publishes 1.6× on a 16 GB M4 Mac mini and 2.24× on an M5 Max for its own packs, and its 9B model on that Mac mini goes from 14.4 to 23.0 tokens a second.

Forge, and the heads it could not find

mtplx forge takes a model off Hugging Face and turns it into a pack MTPLX can run: convert to MLX, pull the head out into a sidecar, calibrate it, then measure before and after on your hardware and tell you the verdict.

The step that pulls the head out matched two key layouts:

mtp.*                      Qwen, Gemma
language_model.mtp.*

There are two more in the wild:

model.layers.{N}.*         DeepSeek, GLM        appended after the body's layers
model.mtp_layers.{i}.*     MiMo                 its own namespace

A model in either of those groups extracted zero keys, so forge wrote no sidecar. The inspection step still reported that a head was present, because it looks at different evidence, so the failure showed up later and looked like something else.

Key counts from each model’s published tensor index, all of which extracted nothing:

model	layout	tensors in the head
DeepSeek-V3.1, R1, V3-0324	appended	1,564 each
DeepSeek-V3.2-Exp	appended	1,571
GLM-4.5, 4.6, 4.5-Air	appended	502 / 500 / 404
GLM-4.7-Flash	appended	212
MiMo-7B-RL, SFT	namespace	16 each

That is the change in MTPLX#442, which is open. Probe order turned out to matter: a model can declare one prediction layer and still keep its head under a prefix, so looking for the appended layer first would go hunting for a layer that does not exist.

Taking one model all the way through

Extraction alone did not produce a working pack. MiMo needed three more things, and each one stopped forge after a step that had reported success.

The head has its own output projection that only one code path filled in. A forged pack keeps the head in a sidecar, which takes the other path, so that projection stayed at its random initialisation and load_weights(strict=False) bound everything else without complaint. Calibration then scored zero on all 64 candidates and pointed at the sidecar, which was complete and had mapped every tensor.

The family resolver had no branch for MiMo, so it resolved to unknown and tune refused the model, even though MiMo has a native backend that reports it can run verified. And forge asked for a fixed set of depths, while MiMo drafts a single token, so tune failed on a depth it was never going to have.

With those fixed, MiMo-7B-RL forged end to end on my M3 Max: 4-bit body, bf16 sidecar, sampling at 0.6 and 0.95, 2048 tokens of long-code prompt.

1.31× at 69.2% acceptance, and 1.42× at 90.3%. Both are my own runs on one machine with fans pinned and plain decoding measured in the same session. The GLM row needs the mlx-lm attention fix from my previous post to hold up at long context.

What is still in the way

Extraction now handles the appended layout, and it is tested, but no appended-layer model completes a forge build yet. Tune has no family branch for GLM or DeepSeek, so those stop at a gate one step later. That gate is a separate change and I have not written it.

The other half is the one I wrote about last week. These models use Multi-head Latent Attention, and verifying a block of drafted tokens is exactly the multi-token step that MLA implementations were handling badly, which made checking a draft cost more than generating the tokens without one. That fix is merged in mlx-vlm, where it turned out to affect ten models rather than the one I started with, and is still open in mlx-lm.

The models this does not reach

I would like to write that this brings a big speedup to the largest models people run at home, and I cannot, for two reasons.

Kimi K3 and Kimi K2 do not ship a head at all. Both declare zero prediction layers, so there is nothing for forge to extract and no amount of work on the extractor changes that. Getting MTP on those models means training an adapter, which forge can also do, but that is a different claim with different evidence behind it.

For the models that do ship a head, I can tell you the head exists and how many tensors it holds, because I read that out of their published indexes. I cannot tell you what speedup you will get, because acceptance rate depends on the head’s training and the prompt, and the verify cost depends on your Mac. The two numbers above are the two models I have actually taken through forge and measured. When a DeepSeek or GLM pack builds, I will measure it and publish that instead of guessing.

If you want to try it on your own Mac, run the auto-tune before anything else. It takes a few minutes and it will tell you whether your machine and your model benefit, including when the answer is that they do not.