flashinfer.cake_gdn_nondet_train_kcp.cake_gdn_nondet_train_kcp_packed_qkv_views¶
- flashinfer.cake_gdn_nondet_train_kcp.cake_gdn_nondet_train_kcp_packed_qkv_views(qkv: Tensor, *, H_qk: int, HV: int) tuple[Tensor, Tensor, Tensor]¶
q/k/vas head views[..., T, H_qk | H_qk | HV, 128]of a packed[..., T, (2 H_qk + HV) * 128]row, with no copy.cake_gdn_nondet_kcp()takes the views as they are and its backward writesdq/dk/dvinto one packed row, which this split’s backward returns as the gradient ofqkvwithout a concatenation. Plainsplitviews ofqkvgive the same gradients through torch’s split backward, which concatenates them (cake_gdn_nondet_kcp()).Needs a dense channel axis (
qkv.stride(-1) == 1). The views are the outputs of one custom autograd node: hooks andretain_gradon them fire as on any autograd output; its backward concatenates instead when the incoming gradients are not the head views of one packed row (a missing gradient counts as zeros, a hook that returns a new gradient); autograd refuses an in-place change to the views whileqkvrequires grad.Example:
q, k, v = cake_gdn_nondet_train_kcp_packed_qkv_views(conv_out, H_qk=16, HV=48) # conv_out: [T, 10240] out = cake_gdn_nondet_kcp(q, k, v, g, beta, ctx)
- Parameters:
qkv (torch.Tensor) – Packed projection rows
[..., T, (2 * H_qk + HV) * 128]with unit channel stride.H_qk (int) – Number of query and key heads, each with 128 channels.
HV (int) – Number of value heads, each with 128 channels.