flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_kcp

flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_kcp(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, ctx: Any, scale: float | None = None, save_mode: str = 'light', boundary_mode: str = 'saved', workspace: Any | None = None, *, recompute_handle: Any | None = None, checkpoint_keep: str | None = None, checkpoint_handoff: str = 'auto', saved_rows: str = 'auto', use_qk_l2norm_in_kernel: bool = False, use_gate_in_kernel: bool = False, A_log: torch.Tensor | None = None, dt_bias: torch.Tensor | None = None, lower_bound: float | None = None, save_transfer: bool | None = None, exchange: str = 'auto', head_groups: int = 1, offset_class: str = 'fla', debug_sequence: bool = False, spec3_probe: str = 'off', graph: str = 'off', _kcp_test_knobs: dict[str, Any] | None = None) → Tensor

Autograd-enabled context-parallel packed KDA chunked training step of this rank’s fragments (the forward of cake_kda_nondet_train_kcp_forward(); the backward of cake_kda_nondet_train_kcp_backward() from its saved set).

Parameters:
  • q (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_nondet_train_kcp_forward().

  • k (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_nondet_train_kcp_forward().

  • v (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_nondet_train_kcp_forward().

  • g (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_nondet_train_kcp_forward().

  • beta (torch.Tensor) – This rank’s input rows, with the contracts of cake_kda_nondet_train_kcp_forward().

  • ctx – This rank’s context from cake_kda_nondet_train_kcp_build_context().

  • scale (float, optional) – Output scale; 128 ** -0.5 when None.

  • save_mode (str, default="light") – Local saved-set mode of cake_kda_nondet_train_kcp_forward().

  • boundary_mode (str, default="saved") – Keep boundary states with "saved" or recompute them with "replay".

  • workspace (optional) – Caller-owned workspace of this package and the same gate and normalization modes.

  • recompute_handle (CakeKDANondetKcpRecomputeHandle, optional) – Keyword-only, default None (unchanged behavior). A handle (flashinfer.cake_kda_nondet_train_kcp.CakeKDANondetKcpRecomputeHandle or any object with an entries dict, such as the convolution’s handle of the same layer) makes the torch.utils.checkpoint recompute of this call run without the forward state exchange: the original forward exchanges as always and stores its incoming states (the saved set’s own tensors, key "kda") in the handle; the recompute inside the backward reads them back, so out and the gradients are bitwise those of the same checkpoint without a handle. The backward still exchanges. Create one handle per layer and micro-step, outside the outermost checkpoint region, and pass it as an argument of the checkpointed function; a forward outside the backward that finds an entry raises. Refused with graph="replay", inside a CUDA-graph capture, with debug_sequence, spec3_probe and boundary_mode="replay". The exchange is a collective: every rank of ctx.group passes a handle for the same call or none, created at the same checkpoint nesting level. The handle’s presence is part of the rank-agreement check of the call’s first original forward (a mismatch trips a device-side assert); a mismatch of the nesting level is not checked and hangs or corrupts data.

  • checkpoint_keep (str, optional) – With a handle only. "states" (default): the handle keeps S_in and the call runs with save_transfer=False (bitwise the default form; the backward re-sends the maps). "transfer": it also keeps the owner’s c1 receive tensor (save_transfer=True; not with exchange="ring"). An explicit save_transfer must agree with the tier ("states" -> False, "transfer" -> True); a contradicting value raises ValueError. The retained bytes are cake_kda_nondet_train_kcp_saved_bytes(ctx, H=..., checkpoint_keep=..., with_conv=False).

  • checkpoint_handoff (str) – Keyword-only, "auto" (default), "on" or "off" (another value raises ValueError): the checkpoint hand-off. Under torch.utils.checkpoint the forward keeps the form saved_rows names in the saved set and the backward skips that part of the local recompute (saved_rows="auto": the "wy" rows; "tape": the tape set). With the tape set, on the devices of KCP_HANDOFF_VERIFIED the outputs and gradients are bitwise those of "off"; "on" elsewhere has no such evidence. "auto" engages where cake_kcp_plan.kcp_handoff_auto_refusal admits the call (a verified device, H in KCP_HANDOFF_HEADS, H * (total tokens // world size) >= 393216, and on a device with its own bound in KCP_HANDOFF_MIN_HEAD_TOKENS_BY_KEY at least that bound). An engaged call raises the checkpoint peak of this rank by at most cake_kda_nondet_train_kcp_handoff_bytes() (its saved set is cake_kda_nondet_train_kcp_saved_bytes(..., checkpoint_handoff=True): no carry plane); "off" opts out. In raw gate mode (use_gate_in_kernel=True) and with the in-kernel q / k L2 norm (use_qk_l2norm_in_kernel=True) the hand-off engages as with the activated gate (the same tape set). The devices verified in that mode (cake_kcp_plan.KCP_HANDOFF_VERIFIED_MODES) replace those of KCP_HANDOFF_VERIFIED: there the raw gate’s gradient, the parameter partial sums and the raw q / k gradients are bitwise those of "off" too, and "auto" admits only those devices. Counters: cake_kda_nondet_train_kcp_handoff_stats().

  • saved_rows (str) – Keyword-only, "auto" (default), "none", "intra", "wy" or "tape" (another value raises ValueError): which forward rows the saved set keeps (cake_kda_nondet_train_kcp_forward()). "auto" keeps the "wy" rows where the hand-off engages (faster than the tape set and 528 MiB less per layer at 16384 tokens per rank, H 64) and nothing elsewhere; "tape" keeps the tape set where the hand-off engages and nothing elsewhere. "none" keeps no rows in any pass (with checkpoint_handoff="on" it raises ValueError, in a checkpoint pass or not). "intra" / "wy": inside a checkpoint pass, the rows form of the hand-off where the hand-off engages (instead of its tape set; nothing where it is refused, "off" included); outside one, the rows where the call builds an autograd node outside a backward and no saved-tensor hooks of the caller are active (there "off" does not stop them). No rows with boundary_mode="replay" or on a rank without tokens. The rows add 256 * R * H / 768 * R * H bytes per layer (cake_kda_nondet_train_kcp_saved_bytes() saved_rows) until the node’s backward; the gradients stay within the light path’s FP64 error budget.

  • use_qk_l2norm_in_kernel (bool) – Keyword-only, default False. As for cake_kda_nondet_train_kcp_forward(); the gradients go to the raw q / k, and a recompute handle’s fingerprint records the mode (its entry does not change: the recompute writes its rstd afresh).

  • **options – As for cake_kda_nondet_train_kcp_forward(), the raw gate keywords included: in raw gate mode the gradients go to the raw gate (BF16), A_log and dt_bias (FP32; this rank’s partial sums, see cake_kda_nondet_train_kcp_backward()), and a recompute handle’s entry records the mode and lower_bound.

Returns:

BF16 out like v (this rank’s tokens).

Return type:

torch.Tensor