flashinfer.logits_processor.MinP¶
- class flashinfer.logits_processor.MinP(**params: Any)¶
Min-p filtering processor.
Keeps tokens with probability at least p times the maximum probability.
TensorType.PROBS
->TensorType.PROBS
- Parameters:
min_p (float or torch.Tensor, Runtime) – Minimum probability threshold as a ratio of max probability. Must be in (0, 1]. Can be a scalar or per-batch tensor.
Examples
>>> import torch >>> from flashinfer.logits_processor import LogitsPipe, Softmax, MinP, Sample >>> torch.manual_seed(42) >>> pipe = LogitsPipe([MinP()]) >>> probs = torch.randn(2, 2, device="cuda") >>> probs_normed = probs / probs.sum(dim=-1, keepdim=True) >>> probs_normed tensor([[ 0.0824, 0.9176], [-0.2541, 1.2541]], device='cuda:0') >>> minp_probs = pipe(probs_normed, min_p=0.05) >>> minp_probs tensor([[0.0824, 0.9176], [0.0000, 1.0000]], device='cuda:0')
- __init__(**params: Any)¶
Constructor for MinP processor. No compile-time parameters are needed.
Methods
__init__
(**params)Constructor for MinP processor.
legalize
(input_type)Legalize the processor into a list of low-level operators.