flashinfer.cake_causal_conv1d_nondet_train_kcp.cake_causal_conv1d_nondet_kcp

flashinfer.cake_causal_conv1d_nondet_train_kcp.cake_causal_conv1d_nondet_kcp(x_local: Tensor, weight: Tensor, ctx: Any, *, recompute_handle: Any | None = None, cooperative: bool | None = None, bwd_reserve_sms: int | None = None) → Tensor

Autograd-enabled context-parallel packed width-4 depthwise causal convolution + SiLU of this rank’s fragments, non-deterministic backward.

The first tokens of a fragment that continues a document read the history rows of the preceding fragment (exchanged through the context’s halo collective) instead of zeros. FP32 taps, no bias. The backward overlaps the halo exchange and returns an unordered dweight partial (y / dx bitwise the deterministic ones). When torch’s deterministic mode is on as the backward runs, the backward raises a RuntimeError before any exchange (a UserWarning, then it runs, with warn_only=True).

Parameters:
  • x_local (torch.Tensor) – BF16 [S_local, C] or [1, S_local, C] contiguous packed activations of this rank’s fragments, in the partition’s order; C is a multiple of 8.

  • weight (torch.Tensor) – FP32 [C, 4] contiguous taps.

  • ctx (CakeKDADetKcpContext) – From flashinfer.cake_kda_det_train_kcp.cake_kda_det_train_kcp_build_context (the same context the KDA step of the rank uses).

  • recompute_handle (CakeCausalConv1dNondetKcpRecomputeHandle, optional) – Keyword-only, default None. A handle (any object with an entries dict) makes the checkpoint recompute of this call run without the forward halo exchange, under the rules of flashinfer.cake_causal_conv1d_det_train_kcp.cake_causal_conv1d_det_kcp() (one handle per call and micro-step, created outside the outermost checkpoint region; every rank passes one or none).

  • cooperative (bool, optional) – Keyword-only, default None. The backward launches no cooperative grid: None and False mean the same and True raises ValueError (the cooperative form belongs to the deterministic KCP package: cake_causal_conv1d_det_kcp(..., cooperative=True)).

  • bwd_reserve_sms (int, optional) – Keyword-only, default None (the package’s RESERVE_DEFAULT, 0). The SMs the one-program backward leaves to kernels of other streams (issue #8365): K >= 1 runs that backward after the halo exchange instead of the overlapped one and launches its single-wave grid K x ctas_per_sm CTAs short, so a kernel of another stream that holds up to K SMs (for example a communication kernel) does not push the top CTAs into a second wave; the unloaded backward gives up the work of those K SMs. y, dx and the exchanges are unchanged and dweight stays within the same budget. With K = 0 (or None) the backward already launches one SM’s CTAs short wherever the last channel-tile rank of the full grid holds at least ctas_per_sm CTAs, so K = 1 leaves the grid unchanged at those widths (for example C = 10240, 6144 and 5120 on a 152-SM GB200 / GB300, C = 10240, 6144 and 3072 on a 148-SM B200) and launches one SM’s CTAs fewer at every other width. Validated by the package when given: a bool or a non-int raises TypeError, a negative value or one not below the device’s SM count ValueError. On sm_107a K >= 1 raises ValueError: there the reserved grid costs up to 1.055x the shipped grid unloaded (C = 1024, 62585 tokens), above this opt-in’s 1.040 limit. The deterministic KCP package’s form beside a kernel of another stream is cooperative=False. Each rank may choose its own value (no collective depends on it).

Returns:

BF16 y of x_local’s shape. The backward returns BF16 dx and the rank-local FP32 dweight partial.

Return type:

torch.Tensor