flashinfer.cake_gdn_det_train_kcp.cake_gdn_det_train_kcp_backward

flashinfer.cake_gdn_det_train_kcp.cake_gdn_det_train_kcp_backward(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, ctx: Any, dout: Tensor, *, saved: Any | None = None, scale: float | None = None, workspace: Any | None = None, options: Any | None = None, use_qk_l2norm_in_kernel: bool = False, exchange: str = 'a2a', save_transfer: bool = True, save_mode: str = 'light', boundary_mode: str = 'saved', precision_mode: int = 0, transfer_walk: str = 'tf32_hilo', offset_class: str = 'fla', layout: str = 'tma', group_reduce: str = 'fused', graph: str = 'off', reuse_recompute_tapes: bool | str | None = None, recompute_output: str = 'auto', backend: str = 'kernels', segments: int | None = None, test_knobs: dict[str, Any] | None = None) → tuple[Tensor, Tensor, Tensor, Tensor, Tensor]

Context-parallel packed GDN training backward.

Parameters:
  • q – The forward’s operands and context.

  • k – The forward’s operands and context.

  • v – The forward’s operands and context.

  • g – The forward’s operands and context.

  • beta – The forward’s operands and context.

  • ctx – The forward’s operands and context.

  • dout (torch.Tensor) – BF16 gradient of out, laid out like v.

  • saved (optional) – The forward’s saved set; None recomputes it from the inputs (checkpoint replay: the summaries and the exchange are re-run).

  • scale – As in cake_gdn_det_train_kcp_forward() (the same options as the forward’s; a saved set of the other use_qk_l2norm_in_kernel value is refused).

  • workspace – As in cake_gdn_det_train_kcp_forward() (the same options as the forward’s; a saved set of the other use_qk_l2norm_in_kernel value is refused).

  • options – As in cake_gdn_det_train_kcp_forward() (the same options as the forward’s; a saved set of the other use_qk_l2norm_in_kernel value is refused).

  • use_qk_l2norm_in_kernel – As in cake_gdn_det_train_kcp_forward() (the same options as the forward’s; a saved set of the other use_qk_l2norm_in_kernel value is refused).

  • **knobs – Additional backend options, as in cake_gdn_det_train_kcp_forward().

Returns:

(dq, dk, dv, dbeta, dg): BF16 gradients of q, k, v (the HV / H_qk value heads’ contributions to dq / dk reduced in FP32 and rounded once), FP32 gradients of beta and g; fresh tensors, ranked like the inputs. Bitwise reproducible at the same topology; saved and replay modes agree bitwise.

Return type:

tuple[torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor, torch.Tensor]