cake_gdn_nondet_train_kcp_partition(...[, ...])
|
Partition one packed batch of documents over the ranks of a context-parallel group. |
cake_gdn_nondet_train_kcp_build_context(...)
|
This rank's view of a partition: local cu_seqlens (int32 device), the device fragment table, the crossing fragment lists (in / out) with their byte counts, the exchange plan, the convolution halo plan (at most conv_kernel_size - 1 history rows per continuing fragment) and the segment plan. |
cake_gdn_nondet_train_kcp_forward(q, k, v, ...)
|
Context-parallel packed GDN training forward of this rank's fragments. |
cake_gdn_nondet_train_kcp_backward(q, k, v, ...)
|
Context-parallel packed GDN training backward. |
cake_gdn_nondet_kcp(q, k, v, g, beta, ctx, *)
|
Context-parallel packed GDN training as an autograd node (torch.utils.checkpoint compatible). |
cake_gdn_nondet_train_kcp_packed_qkv_views(...)
|
q / k / v as head views [..., T, H_qk | H_qk | HV, 128] of a packed [..., T, (2 H_qk + HV) * 128] row, with no copy.
|
cake_gdn_nondet_train_kcp_workspace_bytes(...)
|
Caller-held workspace bytes of one rank for (ctx, H_qk, HV) (exchange pools, staging, tapes): the most the workspace binds in the forward or the backward of the given options (reuse_recompute_tapes as a keyword; the per-phase terms are cake_bytes.cake_gdn_nondet_train_kcp_workspace_layout of the package). |
cake_gdn_nondet_train_kcp_saved_bytes(ctx, ...)
|
Saved-set bytes of one layer on this rank (save_mode, save_transfer as keywords), counted per crossing fragment. |
cake_gdn_nondet_train_kcp_comm_bytes(ctx, *, ...)
|
Bytes this rank moves per collective (c1, c2, b1, b2, halo_fwd, halo_bwd) and per boundary, counted per crossing fragment; C is the convolution channel count of the halo rows. |
is_cake_gdn_nondet_train_kcp_supported([device])
|
True when device (default: the current CUDA device) is an exported target of the package (sm_100a, sm_103a, sm_107a).
|