flashinfer.gdn_paged.gated_delta_rule_decode_paged

flashinfer.gdn_paged.gated_delta_rule_decode_paged(q: Tensor, k: Tensor, v: Tensor, A_log: Tensor, a: Tensor, dt_bias: Tensor, b: Tensor, scale: float | None = None, output: Tensor | None = None, use_qk_l2norm: bool = True, *, state_pool: Tensor, page_indices: Tensor, page_indptr: Tensor, last_page_len: Tensor, backend: Literal['auto', 'flashinfer', 'cake_gdn', 'rubin'] = 'auto', slots: GDNPagedDecodeSlots | None = None) → Tensor

Single-token GDN decode on a paged recurrent-state pool.

The decoded token is the last token of its sequence: it sits at position last_page_len[b] of the sequence’s last page. When the token opens a new page (last_page_len[b] == 1) the state is read from the previous page and written to the new one, so the previous page, which may be a shared full page, is not modified. Otherwise the last page is read and updated in place.

Parameters:

• q – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• k – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• v – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• A_log – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• a – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• dt_bias – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• b – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• scale – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• output – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• use_qk_l2norm – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• backend – As in flashinfer.gdn_decode.gated_delta_rule_decode_pretranspose() with T = 1.

• state_pool (torch.Tensor) – [num_pages, HV, V, K] state pool, updated in place. Supported dtypes and layouts are those of the initial_state pool of gated_delta_rule_decode_pretranspose().

• page_indices (torch.Tensor) – Contiguous int32/int64 CSR page table, as in chunk_gated_delta_rule_paged().

• page_indptr (torch.Tensor) – Contiguous int32/int64 CSR page table, as in chunk_gated_delta_rule_paged().

• last_page_len (torch.Tensor) – [B] contiguous int32/int64 number of tokens in the last page including the decoded token, in [1, page_size].

• slots (GDNPagedDecodeSlots, optional) – Rows from gdn_paged_decode_slots() for the same page table, typically shared by every layer of a step. Without them the call computes them (one extra launch). They are not checked against the page table.

Returns:

The output [B, 1, HV, V].

Return type:

torch.Tensor

Notes

Every sequence must hold at least one token before the decoded one, so a sequence whose decoded token opens a new page owns at least two pages.