flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_train_kcp_workspace_bytes¶
- flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_train_kcp_workspace_bytes(ctx: Any, *, H: int = 64, save_mode: str = 'light', backward: bool = True, save_transfer: bool | None = None, C: int = 8192, local_arena: str | None = None, lean_passes: int | None = None, use_gate_in_kernel: bool = False, use_qk_l2norm_in_kernel: bool = False) int¶
Bytes of this rank’s KCP workspace for
Hheads: the local arena (that ofcake_kda_nondet_train_workspace_bytes()for a backward workspace; a forward-only one takes the KCP forward layout, 2432 B per internal token-head) plus the boundary pools of the context’s collectives, asCakeKDANondetKcpWorkspaceallocates them (the backward’s send pools share the forward’s storage; the ring exchange’s hop pools, allocated by the firstexchange="ring"call, are not counted). The saved set and the convolution’s halo rows are per-call tensors, soCdoes not change it, andsave_transferdoes not change a backward workspace. A forward-only workspace (backward=False) has two states. After its forwards with a saved set it holds the bytes returned here for the forward’ssave_transfer(None, the default, resolves toTrueas the a2a forward does; passFalsefor the ring or the recompute form); withTrueit holds no c1 receive, transfer staging or carry pool, withFalseit holds the first two. Once a forward without a saved set has allocated the rest, it holdscake_kcp_plan.kcp_arena_layout(ctx, H=H, backward=False, save_transfer=save_transfer)["lazy"]bytes more. Underexchange="ring"it never holds the c1 receive pool.ctxmay be a list of contexts: the bytes of one workspace that serves all of them (CakeKDANondetKcpWorkspace.reserve(ctxs, H)): each pool at its largest need and the largest local arena, not their sum.local_arena/lean_passesare those of the workspace (CakeKDANondetKcpWorkspace(local_arena="lean", lean_passes=...), opt-in: the backward’s lean local arena, its tail inlean_passespasses, two by default, with its small tail-count table, on the head counts it serves);Nonekeeps the arena above.use_gate_in_kernel: the workspace’s gate mode (CakeKDANondetKcpWorkspace(use_gate_in_kernel=True): the raw mode’s local arena, the replay arena; the pools do not change).use_qk_l2norm_in_kernel: the workspace’s in-kernel Q / K L2 norm mode (CakeKDANondetKcpWorkspace(use_qk_l2norm_in_kernel=True): the local arena of that mode, the replay arena; the pools do not change).- Parameters:
ctx – This rank’s context from
cake_kda_nondet_train_kcp_build_context().H (int, default=64) – Number of KDA heads.
save_mode (str) – Local saved-set mode of the call,
"light"by default.backward (bool) – Size a backward-capable workspace when
True; otherwise size a forward-only workspace.save_transfer (bool, optional) – Whether to retain forward transfer buffers; default resolution is described above.
C (int) – Convolution channel count for halo rows; does not change workspace bytes.
local_arena (str, optional) – Local arena mode;
"lean"opts into the lower-memory backward arena.lean_passes (int, optional) – Tail passes of the lean local arena;
Noneuses its default.use_gate_in_kernel (bool) – Raw-gate mode of the call and its workspace.
use_qk_l2norm_in_kernel (bool) – In-kernel query/key normalization mode of the call and its workspace.