flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_train_kcp_saved_bytes¶
- flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_train_kcp_saved_bytes(ctx: Any, *, H: int = 64, save_mode: str = 'light', save_transfer: bool | None = None, with_conv: bool = True, C: int = 8192, checkpoint_keep: str | None = None, use_qk_l2norm_in_kernel: bool = False, use_gate_in_kernel: bool = False, checkpoint_handoff: bool = False, saved_rows: str = 'auto') int¶
Bytes of this rank’s KCP saved set for
Hheads: the local saved set (cake_kda_nondet_train_saved_bytes()), the boundary states, and (with_conv) the convolution’s history rows ofCchannels.checkpoint_keep("states"|"transfer"): instead the bytes a recompute handle of that tier retains for the layer (cake_kda_nondet_kcp(); plus one convolution’s history rows withwith_conv).save_transfer:NoneisFalsewithout a tier; with one, an explicit value that contradicts it raisesValueError.use_qk_l2norm_in_kernel: the saved set of the in-kernel Q / K L2 norm adds its FP32 rstd,8 * S_local * Hbytes (a recompute handle’s bytes do not change).use_gate_in_kernel: the call’s gate mode; the bytes do not depend on it, but withuse_qk_l2norm_in_kernel=Truethe query raisesNotImplementedErrorwhile the in-kernel Q / K L2 norm is not available in that gate mode, as the call does.checkpoint_handoff(abool, the hand-off decision, notcake_kda_nondet_kcp()’s mode; another value raisesTypeError;Falseby default: the light set):Trueis the saved set, at the samesave_transfer, of a forward that the checkpoint hand-off engages on (pluscake_kda_nondet_train_kcp_handoff_bytes(), without the carry plane), in every gate and q / k L2 norm mode alike (the rstd term stays).saved_rows: the call’ssaved_rows(cake_kda_nondet_kcp();"auto"by default, another value outside"auto"/"none"/"intra"/"wy"/"tape"raisesValueError):"tape"the sets above,"auto"the light set withcheckpoint_handoff=Falseand the"wy"set withTrue,"none"the light set for eithercheckpoint_handoff,"intra"/"wy"the light set plus256 * R * H/768 * R * Hbytes of kept rows (the carry plane stays) for eithercheckpoint_handoff.- Parameters:
ctx – This rank’s context from
cake_kda_nondet_train_kcp_build_context().H (int, default=64) – Number of KDA heads.
save_mode (str) – Local saved-set mode of the call,
"light"by default.save_transfer (bool, optional) – Whether to retain forward transfer buffers; default resolution is described above.
with_conv (bool) – Include the convolution history rows when
True.C (int) – Convolution channel count used to size halo rows.
checkpoint_keep (str, optional) – Recompute-handle tier,
"states"or"transfer";Nonequeries the ordinary call.use_qk_l2norm_in_kernel (bool) – In-kernel query/key normalization mode of the call and its workspace.
use_gate_in_kernel (bool) – Raw-gate mode of the call and its workspace.
checkpoint_handoff (bool) – Whether the queried call engages checkpoint hand-off (a bool, not the autograd mode string).
saved_rows (str) – Kept-row form of the call:
"auto","none","intra","wy"or"tape".