flashinfer.cake_causal_conv1d_nondet_train_kcp.cake_causal_conv1d_nondet_kcp_multi

flashinfer.cake_causal_conv1d_nondet_train_kcp.cake_causal_conv1d_nondet_kcp_multi(xs: Sequence[Tensor], weights: Sequence[Tensor], ctx: Any, *, recompute_handle: Any | None = None, cooperative: bool | None = None, bwd_reserve_sms: int | None = None) → tuple[Tensor, ...]

Autograd-enabled context-parallel convolutions of several tensors of one layer (for example its q, k and v) with one halo exchange per direction, non-deterministic backward.

Outputs and dx are bitwise tuple(cake_causal_conv1d_nondet_kcp(x, w, ctx) for x, w in zip(xs, weights)); each dweight is that tensor’s own unordered FP32 partial. Under torch’s deterministic mode the backward alerts like the single call’s.

Parameters:
  • xs (sequence of torch.Tensor) – n >= 1 BF16 [S_local, C_i] or [1, S_local, C_i] contiguous packed activations of this rank’s fragments; each C_i is a multiple of 8 (they may differ).

  • weights (sequence of torch.Tensor) – FP32 [C_i, 4] contiguous taps, one per tensor.

  • ctx (CakeKDADetKcpContext) – As in cake_causal_conv1d_nondet_kcp(). Every rank of the group calls this function once per layer, in the same order, with the same tensor count and channel counts.

  • recompute_handle (CakeCausalConv1dNondetKcpRecomputeHandle, optional) – One handle for the whole call (its "conv" entry holds the n history pools), under the rules of cake_causal_conv1d_nondet_kcp().

  • cooperative (bool, optional) – As in cake_causal_conv1d_nondet_kcp().

  • bwd_reserve_sms (int, optional) – As in cake_causal_conv1d_nondet_kcp(), one value for every tensor of the call.

Returns:

The n BF16 outputs, each of its input’s shape.

Return type:

tuple of torch.Tensor