flashinfer.logits_processor.TensorType

class flashinfer.logits_processor.TensorType(value)

TensorType represents the semantic type of tensors in the pipeline.

__init__()

Attributes

LOGITS

Raw or masked float scores, real-valued vectors

PROBS

Non-negative float probabilities, maybe normalized

INDICES

Integer token IDs, terminal type