flashinfer.cake_gdn_det_train_kcp.cake_gdn_det_train_kcp_forward¶
- flashinfer.cake_gdn_det_train_kcp.cake_gdn_det_train_kcp_forward(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, ctx: Any, *, scale: float | None = None, save_for_backward: bool = True, workspace: Any | None = None, options: Any | None = None, use_qk_l2norm_in_kernel: bool = False, exchange: str = 'a2a', save_transfer: bool = True, save_mode: str = 'light', boundary_mode: str = 'saved', precision_mode: int = 0, transfer_walk: str = 'tf32_hilo', offset_class: str = 'fla', layout: str = 'tma', group_reduce: str = 'fused', graph: str = 'off', reuse_recompute_tapes: bool | str | None = None, recompute_output: str = 'auto', backend: str = 'kernels', segments: int | None = None, test_knobs: dict[str, Any] | None = None) tuple[Tensor, Any | None]¶
Context-parallel packed GDN training forward of this rank’s fragments.
- Parameters:
q (torch.Tensor) – BF16
[1, T_r, H_qk, 128](or[T_r, H_qk, 128]); strided packed-QKV views accepted.k (torch.Tensor) – BF16
[1, T_r, H_qk, 128](or[T_r, H_qk, 128]); strided packed-QKV views accepted.v (torch.Tensor) – BF16
[1, T_r, HV, 128],HV % H_qk == 0.g (torch.Tensor) – FP32
[1, T_r, HV]natural-log decay gate (preactivation form, no clamp).beta (torch.Tensor) – FP32
[1, T_r, HV]delta-rule gate after the sigmoid.ctx (CakeGDNDetKcpContext) – This rank’s context from
cake_gdn_det_train_kcp_build_context().scale (float, optional) – Output scale;
128 ** -0.5whenNone.save_for_backward (bool) – Keep the saved set (
boundary_mode="saved");Falseorboundary_mode="replay"returnsNone.workspace (CakeGDNDetKcpWorkspace, optional) – A reused caller-held workspace (
cake_gdn_det_train_kcp_workspace_bytes); without one the call allocates its own.options – Keyword-only options (
save_mode"light"|"full",boundary_mode"saved"|"replay",exchange"a2a"|"ring",save_transfer,offset_class"fla"|"fp32",layout"tma"|"copy",group_reduce"fused"|"host",graph"off"|"replay",reuse_recompute_tapes"auto"(default) |True|False(Falsewithgraph="replay"; seecake_gdn_det_kcp()),recompute_output"auto"|"always"(an autograd-node switch; this forward-only entry never skips a launch), …).**knobs – Keyword overrides for the backend options described above.
use_qk_l2norm_in_kernel (bool) – Normalize
q/kper token and head inside the programs (module docstring); the saved set then carries the FP32rof q and k. With anoptionsinstance it overrides only whenTrue.
- Returns:
(out, saved):outBF16 likev(fresh);savedthe saved set orNone.- Return type:
tuple[torch.Tensor, Optional[Any]]