flashinfer.cake_kda_det_train_kcp.cake_kda_det_kcp

flashinfer.cake_kda_det_train_kcp.cake_kda_det_kcp(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, ctx: Any, scale: float | None = None, save_mode: str = 'light', boundary_mode: str = 'saved', workspace: Any | None = None, *, recompute_handle: Any | None = None, checkpoint_keep: str | None = None, use_qk_l2norm_in_kernel: bool = False, use_gate_in_kernel: bool = False, A_log: torch.Tensor | None = None, dt_bias: torch.Tensor | None = None, lower_bound: float | None = None, save_transfer: bool | None = None, exchange: str = 'auto', head_groups: int = 1, offset_class: str = 'fla', debug_sequence: bool = False, spec3_probe: str = 'off', graph: str = 'off', _kcp_test_knobs: dict[str, Any] | None = None) → Tensor

Autograd-enabled context-parallel packed KDA chunked training step of this rank’s fragments (the forward of cake_kda_det_train_kcp_forward(); the backward of cake_kda_det_train_kcp_backward() from its saved set).

Parameters:
  • q (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_det_train_kcp_forward().

  • k (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_det_train_kcp_forward().

  • v (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_det_train_kcp_forward().

  • g (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_det_train_kcp_forward().

  • beta (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_det_train_kcp_forward().

  • ctx – This rank’s context from cake_kda_det_train_kcp_build_context().

  • scale (float, optional) – Output scale; 128 ** -0.5 when None.

  • save_mode (str, default="light") – Local saved-set mode of cake_kda_det_train_kcp_forward().

  • boundary_mode (str, default="saved") – Keep boundary states with "saved" or recompute them with "replay".

  • workspace (optional) – Caller-owned workspace of this package and the same gate and normalization modes.

  • recompute_handle (CakeKDADetKcpRecomputeHandle, optional) – Keyword-only, default None (unchanged behavior). A handle (flashinfer.cake_kda_det_train_kcp.CakeKDADetKcpRecomputeHandle or any object with an entries dict, such as the convolution’s handle of the same layer) makes the torch.utils.checkpoint recompute of this call run without the forward state exchange: the original forward exchanges as always and stores its incoming states (the saved set’s own tensors, key "kda") in the handle; the recompute inside the backward reads them back, so out and the gradients are bitwise those of the same checkpoint without a handle. The backward still exchanges. Create one handle per layer and micro-step, outside the outermost checkpoint region, and pass it as an argument of the checkpointed function; a forward outside the backward that finds an entry raises. Refused with graph="replay", inside a CUDA-graph capture, with debug_sequence, spec3_probe and boundary_mode="replay". The exchange is a collective: every rank of ctx.group passes a handle for the same call or none, created at the same checkpoint nesting level; a mismatch hangs or corrupts data (no check is made).

  • checkpoint_keep (str, optional) – With a handle only. "states" (default): the handle keeps S_in and the call runs with save_transfer=False (bitwise the default form; the backward re-sends the maps). "transfer": it also keeps the owner’s c1 receive tensor (save_transfer=True; not with exchange="ring"). An explicit save_transfer must agree with the tier ("states" -> False, "transfer" -> True); a contradicting value raises ValueError. The retained bytes are cake_kda_det_train_kcp_saved_bytes(ctx, H=..., checkpoint_keep=..., with_conv=False).

  • use_qk_l2norm_in_kernel (bool) – Keyword-only, default False. As for cake_kda_det_train_kcp_forward(); the gradients go to the raw q / k, and a recompute handle’s fingerprint records the mode (its entry does not change: the recompute writes its rstd afresh).

  • **options – As for cake_kda_det_train_kcp_forward(), the raw gate keywords included: in raw gate mode the gradients go to the raw gate (BF16), A_log and dt_bias (FP32; this rank’s partial sums, see cake_kda_det_train_kcp_backward()), and a recompute handle’s entry records the mode and lower_bound.

Returns:

BF16 out like v (this rank’s tokens).

Return type:

torch.Tensor