flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_workspace_bytes

flashinfer.cake_mhc_nondet_train.cake_mhc_nondet_train_backward_workspace_bytes(T: int, device: device, *, hidden: int = 4096) → int

Bytes of the caller-owned backward workspace for up to T tokens on device.

This package’s backward layout: a 256-byte ticket region and 512 bytes of column-slab arrival counts first (zero on first use; every completed launch leaves them zero), the deterministic package’s regions without its per-chunk FP32 dfn partials, then the column slab’s per-chunk FP32 dfn slots (min(32, ceil(T / 64)) x 24 x 4H x 4 bytes at hidden size H). A deterministic backward workspace (cake_mhc_det_train_backward_workspace()) does not serve this backward.

Parameters:
  • T (int) – Largest token count the buffer serves (the size is the maximum over every token count up to T); T == 0 needs no workspace.

  • device (torch.device) – CUDA device of the calls.

  • hidden (int) – Hidden size of the calls the buffer serves (default 4096), as in cake_mhc_nondet_train_forward_workspace_bytes().

Returns:

The size of the backward workspace in bytes.

Return type:

int