flashinfer.cake_gdn_nondet_train.cake_gdn_nondet¶
- flashinfer.cake_gdn_nondet_train.cake_gdn_nondet(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, scale: float | None = None, initial_state: Tensor | None = None, output_final_state: bool = False, cu_seqlens: Tensor | None = None, cu_seqlens_cpu: Tensor | None = None, use_qk_l2norm_in_kernel: bool = False, *, scan_config: tuple[int, int] | None = None, segment_chunks: int | None = None, finalize: str | None = None, warmup_log2: float | None = None, save_state: bool = True, host_cache: bool = True, dhu_scan: str | None = None, prep: str | None = None, lpt: bool = True, streams: int = 2, tail: str | None = None) tuple[Tensor, Tensor | None]¶
Non-deterministic chunked gated delta rule with FLA’s signature (autograd).
- Parameters:
q (torch.Tensor) – BF16
[B, T, H, 128].k (torch.Tensor) – BF16
[B, T, H, 128].v (torch.Tensor) – BF16
[B, T, HV, 128],HVa positive multiple ofH.g (torch.Tensor) –
[B, T, HV]natural-log decay gate (computed in FP32).beta (torch.Tensor) –
[B, T, HV]FP32 or BF16; its gradient comes back in its dtype.scale (float, optional) – Output scale;
1 / sqrt(128)whenNone.initial_state (torch.Tensor, optional) – FP32
[N, HV, 128, 128].output_final_state (bool) – Return the final state.
cu_seqlens (torch.Tensor, optional) – Packed variable-length batch (
B = 1): int32 / int64 offsets on the device.cu_seqlens_cpu (torch.Tensor, optional) – The same offsets on the host: selects the host-table route. Without it the device builds the tables.
use_qk_l2norm_in_kernel (bool) – Normalize
q/kper token and head inside the kernels.scan_config – Keyword-only plan pins (the package’s
ChainOptions; see the package README).segment_chunks – Keyword-only plan pins (the package’s
ChainOptions; see the package README).finalize – Keyword-only plan pins (the package’s
ChainOptions; see the package README).warmup_log2 – Keyword-only plan pins (the package’s
ChainOptions; see the package README).save_state – Keyword-only plan pins (the package’s
ChainOptions; see the package README).host_cache – Keyword-only plan pins (the package’s
ChainOptions; see the package README).dhu_scan – Keyword-only plan pins (the package’s
ChainOptions; see the package README).prep – Keyword-only plan pins (the package’s
ChainOptions; see the package README).lpt – Keyword-only plan pins (the package’s
ChainOptions; see the package README).streams – Keyword-only plan pins (the package’s
ChainOptions; see the package README).tail – Keyword-only plan pins (the package’s
ChainOptions; see the package README).
- Returns:
(o, final_state):olikev;final_stateisNoneunlessoutput_final_state.The inputs are checked before the first launch (
ValueError). There is nodeterministickeyword. Every launch is fixed-order today, sotorch.use_deterministic_algorithms(True)(strict orwarn_only) changes nothing; the contract does not promise run-to-run bits.- Return type:
tuple[torch.Tensor, Optional[torch.Tensor]]