A CONCEPT · PROJECTED FROM THE VINDEX KNOWLEDGE GRAPH
O — o_proj
write the answer back
Collects what attention gathered across all heads and writes it into the stream, sized to fit.
The closing move of attention: everything the heads collected, combined and projected back to the hidden width so the residual stream can absorb it.
IN THE GRAPH
o — o_proj— closes →attention