CONV-QKV ATTENTION

The hybrid Mamba2Attn stack's attention operator: one fused QKV projection, a causal depthwise convolution over the FULL fused QKV with no activation, partial rotary on the leading dims of each head, then GQA softmax and an output projection — recognisably attention, and not plain softmax.

Reading it as generic attention would drop the convolution — a real mixing step with its own continuation state — and rotate the whole head instead of the declared fraction. So it is its own operator, with five operand roles whose spellings collide with the Mamba2 set at different shapes, resolved per layer by the declared program. Its continuation is TWO regions on one layer: a real per-position KV cache AND a convolution history over the pre-conv fused QKV — the first operator to declare both, and the proof that operation family does not imply state shape. A provider that can hold only KV rows refuses the layer rather than allocating half of it. Witnessed live on the 250M hybrid: 468 of 468 teacher-forced positions argmax-exact at the reference's own numerical floor, and source-hidden LQL generation reproducing the oracle id-for-id.

IN THE GRAPH