## Decoder mask: the prediction cannot see later output positions

Finding: The decoder self-attention masks subsequent positions. The shifted output embeddings limit a prediction to earlier known outputs.

Locator: Attention Is All You Need, arXiv:1706.03762v7, section 3.1, Decoder paragraph; corroborating mechanism in section 3.2.3, final bullet.

Source: https://arxiv.org/html/1706.03762v7#S3.SS1

Interpretation: When explaining the architecture, separate the encoder from the decoder. A statement about attending across the encoder input should not be transferred to future decoder output positions.

Boundary: This note describes the architecture in this paper. It does not establish how every later model handles context.

Search terms: decoder-mask future-position

## Six layers in this model specification

Finding: Section 3.1 specifies six identical layers for the encoder and six for the decoder.

Locator: Attention Is All You Need, arXiv:1706.03762v7, section 3.1, Encoder and Decoder paragraphs.

Source: https://arxiv.org/html/1706.03762v7#S3.SS1

Interpretation: Keep the layer count attached to this paper and version when taking notes on implementation details.

Boundary: Six layers is not a definition of all Transformers.

Search terms: layer-count model-specification