flashinfer.cake_causal_conv1d_nondet_train_kcp.cake_causal_conv1d_nondet_kcp¶
- flashinfer.cake_causal_conv1d_nondet_train_kcp.cake_causal_conv1d_nondet_kcp(x_local: Tensor, weight: Tensor, ctx: Any, *, recompute_handle: Any | None = None, cooperative: bool | None = None, bwd_reserve_sms: int | None = None) Tensor¶
Autograd-enabled context-parallel packed width-4 depthwise causal convolution + SiLU of this rank’s fragments, non-deterministic backward.
The first tokens of a fragment that continues a document read the history rows of the preceding fragment (exchanged through the context’s halo collective) instead of zeros. FP32 taps, no bias. The backward overlaps the halo exchange and returns an unordered
dweightpartial (y/dxbitwise the deterministic ones). When torch’s deterministic mode is on as the backward runs, the backward raises aRuntimeErrorbefore any exchange (aUserWarning, then it runs, withwarn_only=True).- Parameters:
x_local (torch.Tensor) – BF16
[S_local, C]or[1, S_local, C]contiguous packed activations of this rank’s fragments, in the partition’s order;Cis a multiple of 8.weight (torch.Tensor) – FP32
[C, 4]contiguous taps.ctx (CakeKDADetKcpContext) – From
flashinfer.cake_kda_det_train_kcp.cake_kda_det_train_kcp_build_context(the same context the KDA step of the rank uses).recompute_handle (CakeCausalConv1dNondetKcpRecomputeHandle, optional) – Keyword-only, default
None. A handle (any object with anentriesdict) makes the checkpoint recompute of this call run without the forward halo exchange, under the rules offlashinfer.cake_causal_conv1d_det_train_kcp.cake_causal_conv1d_det_kcp()(one handle per call and micro-step, created outside the outermost checkpoint region; every rank passes one or none).cooperative (bool, optional) – Keyword-only, default
None. The backward launches no cooperative grid:NoneandFalsemean the same andTrueraisesValueError(the cooperative form belongs to the deterministic KCP package:cake_causal_conv1d_det_kcp(..., cooperative=True)).bwd_reserve_sms (int, optional) – Keyword-only, default
None(the package’sRESERVE_DEFAULT, 0). The SMs the one-program backward leaves to kernels of other streams (issue #8365):K >= 1runs that backward after the halo exchange instead of the overlapped one and launches its single-wave gridK x ctas_per_smCTAs short, so a kernel of another stream that holds up toKSMs (for example a communication kernel) does not push the top CTAs into a second wave; the unloaded backward gives up the work of thoseKSMs.y,dxand the exchanges are unchanged anddweightstays within the same budget. WithK = 0(orNone) the backward already launches one SM’s CTAs short wherever the last channel-tile rank of the full grid holds at leastctas_per_smCTAs, soK = 1leaves the grid unchanged at those widths (for exampleC= 10240, 6144 and 5120 on a 152-SM GB200 / GB300,C= 10240, 6144 and 3072 on a 148-SM B200) and launches one SM’s CTAs fewer at every other width. Validated by the package when given: aboolor a non-intraisesTypeError, a negative value or one not below the device’s SM countValueError. On sm_107aK >= 1raisesValueError: there the reserved grid costs up to 1.055x the shipped grid unloaded (C= 1024, 62585 tokens), above this opt-in’s 1.040 limit. The deterministic KCP package’s form beside a kernel of another stream iscooperative=False. Each rank may choose its own value (no collective depends on it).
- Returns:
BF16
yofx_local’s shape. The backward returns BF16dxand the rank-local FP32dweightpartial.- Return type:
torch.Tensor