A CONCEPT · PROJECTED FROM THE VINDEX KNOWLEDGE GRAPH
GATE — gate_proj
controls feed-forward feature activation
Builds a second wide vector whose job is judgement: it scores every expanded channel for how much it should matter right now.
The gate's output multiplies the up projection's output, channel by channel — scored channels pass, the rest fade. That single multiplication is the whole trick the literature calls a gated unit (SwiGLU).
IN THE GRAPH
gate — gate_proj— paired with →up — up_proj
gate — gate_proj— belongs to →feed-forward
gate — gate_proj— represented by →gate_up — the paired bank