flashinfer.cake_gdn_det_train.cake_gdn_det¶
- flashinfer.cake_gdn_det_train.cake_gdn_det(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, scale: float | None = None, initial_state: Tensor | None = None, output_final_state: bool = False, cu_seqlens: Tensor | None = None, cu_seqlens_cpu: Tensor | None = None, use_qk_l2norm_in_kernel: bool = False, *, scan_config: tuple[int, int] | None = None, segment_chunks: int | None = None, finalize: str | None = None, warmup_log2: float | None = None, save_state: bool = True, host_cache: bool = True, dhu_scan: str | None = None, prep: str | None = None, lpt: bool = True, streams: int = 2) tuple[Tensor, Tensor | None]¶
Deterministic packed GDN chunked training as an autograd node (
torch.utils.checkpointcompatible).Returns
(o, final_state):olikev,final_stateFP32[N, HV, 128, 128]orNoneunlessoutput_final_state; gradients toq, k, v, g, beta(andinitial_state). The keyword-only arguments are the plan pins of the package (scan_config,segment_chunks,finalize,warmup_log2,dhu_scan,prep,lpt,streams) and itssave_state/host_cacheswitches; seeflashinfer.experimental.cake.cake_gdn_det_train.cake_gdn_detfor the contract and the input checks (ValueErrorbefore the first launch).- Parameters:
q (torch.Tensor) – BF16
[B, T, H, 128]query and key tensors.k (torch.Tensor) – BF16
[B, T, H, 128]query and key tensors.v (torch.Tensor) – BF16
[B, T, HV, 128]values, withHVa multiple ofH.g (torch.Tensor) –
[B, T, HV]natural-log decay gate, computed in FP32.beta (torch.Tensor) – FP32 or BF16
[B, T, HV]delta-rule gate after the sigmoid.scale (float, optional) – Output scale;
128 ** -0.5whenNone.initial_state (torch.Tensor, optional) – FP32
[N, HV, 128, 128]initial state; zero when omitted.output_final_state (bool, default=False) – Return the final state in addition to the output.
cu_seqlens (torch.Tensor, optional) – Device document offsets of a packed batch (
B == 1).cu_seqlens_cpu (torch.Tensor, optional) – The same offsets on the host; selects the host-table route when given.
use_qk_l2norm_in_kernel (bool, default=False) – Normalize raw query and key rows inside the kernels.
scan_config (tuple of int, optional) – Pin the backend scan configuration.
segment_chunks (int, optional) – Pin the number of chunks per segment.
finalize (str, optional) – Pin the finalize stage of the unfused backward tail.
warmup_log2 (float, optional) – Override the pre-scans’ log2 warm-up threshold.
save_state (bool, default=True) – Keep the forward state tape for the backward.
host_cache (bool, default=True) – Cache the host-side plan and tables.
dhu_scan (str, optional) – Pin the backward adjoint scan implementation.
prep (str, optional) – Pin the backend preparation implementation.
lpt (bool, default=True) – Enable longest-processing-time scheduling in the backend plan.
streams (int, default=2) – Number of streams used by the backend plan:
1or2.