GATED ATTENTION

Softmax attention whose output is modulated by a learned gate before it rejoins the residual stream — the gate is added to the operator, not substituted for it.

Named GATED ATTENTION rather than plain ATTENTION because the ungated case is also attention: a pair reading ATTENTION and GATED ATTENTION would invite the reading that only one of them attends. The gate is an operand the layer either ships or does not, declared once on the component's execution surface, so the label is read from the graph rather than guessed from a tensor's name. Qwen3.8's full-attention layers are of this kind, which is why its precision map has a zgate column beside q, k, v and o.

IN THE GRAPH