flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_train_kcp_saved_bytes

flashinfer.cake_kda_nondet_train_kcp.cake_kda_nondet_train_kcp_saved_bytes(ctx: Any, *, H: int = 64, save_mode: str = 'light', save_transfer: bool | None = None, with_conv: bool = True, C: int = 8192, checkpoint_keep: str | None = None, use_qk_l2norm_in_kernel: bool = False, use_gate_in_kernel: bool = False, checkpoint_handoff: bool = False, saved_rows: str = 'auto') → int

Bytes of this rank’s KCP saved set for H heads: the local saved set (cake_kda_nondet_train_saved_bytes()), the boundary states, and (with_conv) the convolution’s history rows of C channels. checkpoint_keep ("states" | "transfer"): instead the bytes a recompute handle of that tier retains for the layer (cake_kda_nondet_kcp(); plus one convolution’s history rows with with_conv). save_transfer: None is False without a tier; with one, an explicit value that contradicts it raises ValueError. use_qk_l2norm_in_kernel: the saved set of the in-kernel Q / K L2 norm adds its FP32 rstd, 8 * S_local * H bytes (a recompute handle’s bytes do not change). use_gate_in_kernel: the call’s gate mode; the bytes do not depend on it, but with use_qk_l2norm_in_kernel=True the query raises NotImplementedError while the in-kernel Q / K L2 norm is not available in that gate mode, as the call does. checkpoint_handoff (a bool, the hand-off decision, not cake_kda_nondet_kcp()’s mode; another value raises TypeError; False by default: the light set): True is the saved set, at the same save_transfer, of a forward that the checkpoint hand-off engages on (plus cake_kda_nondet_train_kcp_handoff_bytes(), without the carry plane), in every gate and q / k L2 norm mode alike (the rstd term stays). saved_rows: the call’s saved_rows (cake_kda_nondet_kcp(); "auto" by default, another value outside "auto" / "none" / "intra" / "wy" / "tape" raises ValueError): "tape" the sets above, "auto" the light set with checkpoint_handoff=False and the "wy" set with True, "none" the light set for either checkpoint_handoff, "intra" / "wy" the light set plus 256 * R * H / 768 * R * H bytes of kept rows (the carry plane stays) for either checkpoint_handoff.

Parameters:
  • ctx – This rank’s context from cake_kda_nondet_train_kcp_build_context().

  • H (int, default=64) – Number of KDA heads.

  • save_mode (str) – Local saved-set mode of the call, "light" by default.

  • save_transfer (bool, optional) – Whether to retain forward transfer buffers; default resolution is described above.

  • with_conv (bool) – Include the convolution history rows when True.

  • C (int) – Convolution channel count used to size halo rows.

  • checkpoint_keep (str, optional) – Recompute-handle tier, "states" or "transfer"; None queries the ordinary call.

  • use_qk_l2norm_in_kernel (bool) – In-kernel query/key normalization mode of the call and its workspace.

  • use_gate_in_kernel (bool) – Raw-gate mode of the call and its workspace.

  • checkpoint_handoff (bool) – Whether the queried call engages checkpoint hand-off (a bool, not the autograd mode string).

  • saved_rows (str) – Kept-row form of the call: "auto", "none", "intra", "wy" or "tape".