flashinfer.cake_gdn_nondet_train.cake_gdn_nondet

flashinfer.cake_gdn_nondet_train.cake_gdn_nondet(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, scale: float | None = None, initial_state: Tensor | None = None, output_final_state: bool = False, cu_seqlens: Tensor | None = None, cu_seqlens_cpu: Tensor | None = None, use_qk_l2norm_in_kernel: bool = False, *, scan_config: tuple[int, int] | None = None, segment_chunks: int | None = None, finalize: str | None = None, warmup_log2: float | None = None, save_state: bool = True, host_cache: bool = True, dhu_scan: str | None = None, prep: str | None = None, lpt: bool = True, streams: int = 2, tail: str | None = None) → tuple[Tensor, Tensor | None]

Non-deterministic chunked gated delta rule with FLA’s signature (autograd).

Parameters:
  • q (torch.Tensor) – BF16 [B, T, H, 128].

  • k (torch.Tensor) – BF16 [B, T, H, 128].

  • v (torch.Tensor) – BF16 [B, T, HV, 128], HV a positive multiple of H.

  • g (torch.Tensor) – [B, T, HV] natural-log decay gate (computed in FP32).

  • beta (torch.Tensor) – [B, T, HV] FP32 or BF16; its gradient comes back in its dtype.

  • scale (float, optional) – Output scale; 1 / sqrt(128) when None.

  • initial_state (torch.Tensor, optional) – FP32 [N, HV, 128, 128].

  • output_final_state (bool) – Return the final state.

  • cu_seqlens (torch.Tensor, optional) – Packed variable-length batch (B = 1): int32 / int64 offsets on the device.

  • cu_seqlens_cpu (torch.Tensor, optional) – The same offsets on the host: selects the host-table route. Without it the device builds the tables.

  • use_qk_l2norm_in_kernel (bool) – Normalize q / k per token and head inside the kernels.

  • scan_config – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • segment_chunks – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • finalize – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • warmup_log2 – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • save_state – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • host_cache – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • dhu_scan – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • prep – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • lpt – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • streams – Keyword-only plan pins (the package’s ChainOptions; see the package README).

  • tail – Keyword-only plan pins (the package’s ChainOptions; see the package README).

Returns:

(o, final_state): o like v; final_state is None unless output_final_state.

The inputs are checked before the first launch (ValueError). There is no deterministic keyword. Every launch is fixed-order today, so torch.use_deterministic_algorithms(True) (strict or warn_only) changes nothing; the contract does not promise run-to-run bits.

Return type:

tuple[torch.Tensor, Optional[torch.Tensor]]