flashinfer.cake_kda_det_train_kcp.cake_kda_det_kcp¶
- flashinfer.cake_kda_det_train_kcp.cake_kda_det_kcp(q: Tensor, k: Tensor, v: Tensor, g: Tensor, beta: Tensor, ctx: Any, scale: float | None = None, save_mode: str = 'light', boundary_mode: str = 'saved', workspace: Any | None = None, *, recompute_handle: Any | None = None, checkpoint_keep: str | None = None, use_qk_l2norm_in_kernel: bool = False, use_gate_in_kernel: bool = False, A_log: torch.Tensor | None = None, dt_bias: torch.Tensor | None = None, lower_bound: float | None = None, save_transfer: bool | None = None, exchange: str = 'auto', head_groups: int = 1, offset_class: str = 'fla', debug_sequence: bool = False, spec3_probe: str = 'off', graph: str = 'off', _kcp_test_knobs: dict[str, Any] | None = None) Tensor¶
Autograd-enabled context-parallel packed KDA chunked training step of this rank’s fragments (the forward of
cake_kda_det_train_kcp_forward(); the backward ofcake_kda_det_train_kcp_backward()from its saved set).- Parameters:
q (torch.Tensor) – This rank’s input rows, with the contracts of
cake_kda_det_train_kcp_forward().k (torch.Tensor) – This rank’s input rows, with the contracts of
cake_kda_det_train_kcp_forward().v (torch.Tensor) – This rank’s input rows, with the contracts of
cake_kda_det_train_kcp_forward().g (torch.Tensor) – This rank’s input rows, with the contracts of
cake_kda_det_train_kcp_forward().beta (torch.Tensor) – This rank’s input rows, with the contracts of
cake_kda_det_train_kcp_forward().ctx – This rank’s context from
cake_kda_det_train_kcp_build_context().scale (float, optional) – Output scale;
128 ** -0.5whenNone.save_mode (str, default="light") – Local saved-set mode of
cake_kda_det_train_kcp_forward().boundary_mode (str, default="saved") – Keep boundary states with
"saved"or recompute them with"replay".workspace (optional) – Caller-owned workspace of this package and the same gate and normalization modes.
recompute_handle (CakeKDADetKcpRecomputeHandle, optional) – Keyword-only, default
None(unchanged behavior). A handle (flashinfer.cake_kda_det_train_kcp.CakeKDADetKcpRecomputeHandleor any object with anentriesdict, such as the convolution’s handle of the same layer) makes thetorch.utils.checkpointrecompute of this call run without the forward state exchange: the original forward exchanges as always and stores its incoming states (the saved set’s own tensors, key"kda") in the handle; the recompute inside the backward reads them back, sooutand the gradients are bitwise those of the same checkpoint without a handle. The backward still exchanges. Create one handle per layer and micro-step, outside the outermost checkpoint region, and pass it as an argument of the checkpointed function; a forward outside the backward that finds an entry raises. Refused withgraph="replay", inside a CUDA-graph capture, withdebug_sequence,spec3_probeandboundary_mode="replay". The exchange is a collective: every rank ofctx.grouppasses a handle for the same call or none, created at the same checkpoint nesting level; a mismatch hangs or corrupts data (no check is made).checkpoint_keep (str, optional) – With a handle only.
"states"(default): the handle keepsS_inand the call runs withsave_transfer=False(bitwise the default form; the backward re-sends the maps)."transfer": it also keeps the owner’s c1 receive tensor (save_transfer=True; not withexchange="ring"). An explicitsave_transfermust agree with the tier ("states"->False,"transfer"->True); a contradicting value raisesValueError. The retained bytes arecake_kda_det_train_kcp_saved_bytes(ctx, H=..., checkpoint_keep=..., with_conv=False).use_qk_l2norm_in_kernel (bool) – Keyword-only, default
False. As forcake_kda_det_train_kcp_forward(); the gradients go to the rawq/k, and a recompute handle’s fingerprint records the mode (its entry does not change: the recompute writes its rstd afresh).**options – As for
cake_kda_det_train_kcp_forward(), the raw gate keywords included: in raw gate mode the gradients go to the raw gate (BF16),A_loganddt_bias(FP32; this rank’s partial sums, seecake_kda_det_train_kcp_backward()), and a recompute handle’s entry records the mode andlower_bound.
- Returns:
BF16
outlikev(this rank’s tokens).- Return type:
torch.Tensor