flashinfer.cake_rmsnorm_nondet_train.cake_rmsnorm_nondet_train_backward_workspace_bytes

flashinfer.cake_rmsnorm_nondet_train.cake_rmsnorm_nondet_train_backward_workspace_bytes(rows: int, hidden: int, device: device, *, residual: bool = False) → int

Bytes of the caller-owned backward workspace of width hidden on device for up to rows tokens.

The workspace has one layout for every call (cake_jit.workspace_layout, offsets fixed by the package): the deterministic pairs’ uint32 counters (cake_jit.COUNTERS_BYTES), the single kernel’s counters (cake_jit.SINGLE_COUNTERS_BYTES at cake_jit.SINGLE_COUNTERS_OFFSET), its FP64 column sums (cake_jit.SUMS_BYTES at cake_jit.SUMS_OFFSET, the largest workspace_bytes of the package’s single kernels) and, when a window up to rows of the width runs the deterministic pair, that pair’s FP32 partial rows at cake_jit.PAIR_ROWS_OFFSET, which grow with rows (a pair-row single kernel’s FP32 scratch, one row of FP32 pairs per CTA, takes the same place). The size returned is the largest layout of the records the width’s backward routes (hidden, residual, both weight dtypes) run for up to rows tokens. The buffer must be zero on first use (allocate with torch.zeros or use cake_rmsnorm_nondet_train_backward_workspace()); after every launch the kernels leave the counters at rest and the sums at zero, so with no host write one buffer serves any sequence of this package’s backward calls – hidden sizes, residual flags, weight dtypes, token counts, windows and CUDA Graph replays – whose sizes it covers. Zeroing it again between calls is harmless. Backward calls that may run at the same time (on different streams) need one buffer each.

The deterministic package’s workspace has another layout (size it with flashinfer.cake_rmsnorm_det_train.cake_rmsnorm_det_train_backward_workspace_bytes()). The size does not depend on torch.use_deterministic_algorithms.