flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_preload¶
- flashinfer.cake_lm_head_det_train.cake_lm_head_det_train_preload(device: device | str | int | None = None, *, temperature: bool = False)¶
Do part of the one-time work of a process’s first chunked LM-head call ahead of it, then wait for the device once (SM100 / SM103 / SM107 only).
Loads every generated library of the loss and log-probability entries for
device’s architecture (a JIT build when the package ships no prebuilt library; the candidate-support entries’ libraries still load at their first call), creates the per-device side stream (PyTorch’s stream pool), the count cells and constants, launches the PyTorch operators of the valid-row count paths once and synchronizes. It does not launch the package’s kernels: under the driver’s lazy module loading (CUDA_MODULE_LOADING=LAZY, the default) each kernel form still loads at its first launch.device: a CUDA device, its index orNone(the current device).temperature=Truealso loads the libraries of thecake_lm_head_det_train_logprob()temperature path and launches its PyTorch operators once; by default they are not loaded (callers that never pass a temperature build no temperature library).- Parameters:
device (torch.device, int, str, optional) – CUDA device whose libraries to load;
Noneselects the current device.temperature (bool, default=False) – Also load the temperature-path libraries and warm up their PyTorch operators.