flashinfer.cake_gdn_nondet_train_kcp.cake_gdn_nondet_train_kcp_forward

flashinfer.cake_gdn_nondet_train_kcp.cake_gdn_nondet_train_kcp_forward(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, ctx: Any, *, scale: float | None = None, save_for_backward: bool = True, workspace: Any | None = None, options: Any | None = None, use_qk_l2norm_in_kernel: bool = False, exchange: str = 'a2a', save_transfer: bool = True, save_mode: str | None = None, boundary_mode: str = 'saved', precision_mode: int = 0, transfer_walk: str = 'tf32_hilo', offset_class: str = 'fla', layout: str = 'tma', group_reduce: str = 'fused', graph: str = 'off', reuse_recompute_tapes: bool | str | None = None, recompute_output: str = 'auto', backend: str = 'kernels', segments: int | None = None, test_knobs: dict[str, Any] | None = None) → tuple[Tensor, Any | None]

Context-parallel packed GDN training forward of this rank’s fragments.

Parameters:
  • q (torch.Tensor) – BF16 [1, T_r, H_qk, 128] (or [T_r, H_qk, 128]); strided packed-QKV views accepted.

  • k (torch.Tensor) – BF16 [1, T_r, H_qk, 128] (or [T_r, H_qk, 128]); strided packed-QKV views accepted.

  • v (torch.Tensor) – BF16 [1, T_r, HV, 128], HV % H_qk == 0.

  • g (torch.Tensor) – FP32 [1, T_r, HV] natural-log decay gate (preactivation form, no clamp).

  • beta (torch.Tensor) – FP32 [1, T_r, HV] delta-rule gate after the sigmoid.

  • ctx (CakeGDNNondetKcpContext) – This rank’s context from cake_gdn_nondet_train_kcp_build_context().

  • scale (float, optional) – Output scale; 128 ** -0.5 when None.

  • save_for_backward (bool) – Keep the saved set (boundary_mode="saved"); False or boundary_mode="replay" returns None.

  • workspace (CakeGDNNondetKcpWorkspace, optional) – A reused caller-held workspace (cake_gdn_nondet_train_kcp_workspace_bytes); without one the call allocates its own.

  • options – Keyword-only options (save_mode "light" | "full" | "light_akk" (see the module docstring; unset: "light_akk" with use_qk_l2norm_in_kernel=True, else "light"), boundary_mode "saved" | "replay", exchange "a2a" | "ring", save_transfer, offset_class "fla" | "fp32", layout "tma" | "copy", group_reduce "fused" | "host", graph "off" | "replay", reuse_recompute_tapes "auto" (default) | True | False (see cake_gdn_nondet_kcp()), recompute_output "auto" | "always" (an autograd-node switch; this forward-only entry never skips a launch), …).

  • **knobs – Keyword overrides for the backend options described above.

Returns:

(out, saved): out BF16 like v (fresh); saved the saved set or None (readable by this package’s backward only).

Return type:

tuple[torch.Tensor, Optional[Any]]