flashinfer.cake_kda_det_train_kcp.cake_kda_det_train_kcp_saved_bytes¶
- flashinfer.cake_kda_det_train_kcp.cake_kda_det_train_kcp_saved_bytes(ctx: Any, *, H: int = 64, save_mode: str = 'light', save_transfer: bool | None = None, with_conv: bool = True, C: int = 8192, checkpoint_keep: str | None = None, use_qk_l2norm_in_kernel: bool = False, use_gate_in_kernel: bool = False) int¶
Bytes of this rank’s KCP saved set for
Hheads: the local saved set (cake_kda_det_train_saved_bytes()), the boundary states, and (with_conv) the convolution’s history rows ofCchannels.checkpoint_keep("states"|"transfer"): instead the bytes a recompute handle of that tier retains for the layer (cake_kda_det_kcp(); plus one convolution’s history rows withwith_conv).save_transfer:NoneisFalsewithout a tier; with one, an explicit value that contradicts it raisesValueError.use_qk_l2norm_in_kernel: the saved set of the in-kernel Q / K L2 norm adds its FP32 rstd,8 * S_local * Hbytes (a recompute handle’s bytes do not change).use_gate_in_kernel: the call’s gate mode; the bytes do not depend on it, but withuse_qk_l2norm_in_kernel=Truethe query raisesNotImplementedErrorwhile the in-kernel Q / K L2 norm is not available in that gate mode, as the call does.- Parameters:
ctx – This rank’s context from
cake_kda_det_train_kcp_build_context().H (int, default=64) – Number of KDA heads.
save_mode (str) – Local saved-set mode of the call,
"light"by default.save_transfer (bool, optional) – Whether to retain forward transfer buffers; default resolution is described above.
with_conv (bool) – Include the convolution history rows when
True.C (int) – Convolution channel count used to size halo rows.
checkpoint_keep (str, optional) – Recompute-handle tier,
"states"or"transfer";Nonequeries the ordinary call.use_qk_l2norm_in_kernel (bool) – In-kernel query/key normalization mode of the call and its workspace.
use_gate_in_kernel (bool) – Raw-gate mode of the call and its workspace.