flashinfer.logits_processor.TensorType¶

class flashinfer.logits_processor.TensorType(value, names=<not given>, *values, module=None, qualname=None, type=None, start=1, boundary=None)¶

TensorType represents the semantic type of tensors in the pipeline.

__init__(*args, **kwds)¶

Attributes

`LOGITS`	Raw or masked float scores, real-valued vectors
`PROBS`	Non-negative float probabilities, maybe normalized
`INDICES`	Integer token IDs, terminal type