flashinfer.logits_processor.MinP

class flashinfer.logits_processor.MinP(**params: Any)

Min-p filtering processor.

Keeps tokens with probability at least p times the maximum probability.

TensorType.PROBS -> TensorType.PROBS

Parameters:

min_p (float or torch.Tensor, Runtime) – Minimum probability threshold as a ratio of max probability. Must be in (0, 1]. Can be a scalar or per-batch tensor.

Examples

>>> import torch
>>> from flashinfer.logits_processor import LogitsPipe, Softmax, MinP, Sample
>>> torch.manual_seed(42)
>>> pipe = LogitsPipe([MinP()])
>>> probs = torch.randn(2, 2, device="cuda")
>>> probs_normed = probs / probs.sum(dim=-1, keepdim=True)
>>> probs_normed
tensor([[ 0.0824,  0.9176], [-0.2541,  1.2541]], device='cuda:0')
>>> minp_probs = pipe(probs_normed, min_p=0.05)
>>> minp_probs
tensor([[0.0824, 0.9176], [0.0000, 1.0000]], device='cuda:0')

__init__(**params: Any)

Constructor for MinP processor. No compile-time parameters are needed.

Methods

__init__(**params)	Constructor for MinP processor.
legalize(input_type)	Legalize the processor into a list of low-level operators.