flashinfer.cake_rmsnorm_nondet_train.cake_rmsnorm_nondet_train_backward_workspace_bytes¶
- flashinfer.cake_rmsnorm_nondet_train.cake_rmsnorm_nondet_train_backward_workspace_bytes(rows: int, hidden: int, device: device, *, residual: bool = False) int¶
Bytes of the caller-owned backward workspace of width
hiddenondevicefor up torowstokens.The workspace has one layout for every call (
cake_jit.workspace_layout, offsets fixed by the package): the deterministic pairs’uint32counters (cake_jit.COUNTERS_BYTES), the single kernel’s counters (cake_jit.SINGLE_COUNTERS_BYTESatcake_jit.SINGLE_COUNTERS_OFFSET), its FP64 column sums (cake_jit.SUMS_BYTESatcake_jit.SUMS_OFFSET, the largestworkspace_bytesof the package’s single kernels) and, when a window up torowsof the width runs the deterministic pair, that pair’s FP32 partial rows atcake_jit.PAIR_ROWS_OFFSET, which grow withrows(a pair-row single kernel’s FP32 scratch, one row of FP32 pairs per CTA, takes the same place). The size returned is the largest layout of the records the width’s backward routes (hidden,residual, both weight dtypes) run for up torowstokens. The buffer must be zero on first use (allocate withtorch.zerosor usecake_rmsnorm_nondet_train_backward_workspace()); after every launch the kernels leave the counters at rest and the sums at zero, so with no host write one buffer serves any sequence of this package’s backward calls – hidden sizes, residual flags, weight dtypes, token counts, windows and CUDA Graph replays – whose sizes it covers. Zeroing it again between calls is harmless. Backward calls that may run at the same time (on different streams) need one buffer each.The deterministic package’s workspace has another layout (size it with
flashinfer.cake_rmsnorm_det_train.cake_rmsnorm_det_train_backward_workspace_bytes()). The size does not depend ontorch.use_deterministic_algorithms.