Skip to main content

attention_mask_fill_value

Function attention_mask_fill_value 

Source
pub fn attention_mask_fill_value(dtype: DType) -> f64
Expand description

The additive value used for padded key positions, chosen per dtype.

It has to be large enough that exp(score + fill - max) underflows to zero and small enough to stay finite in the tensor’s dtype. F16 tops out at ±65504, so the usual -1e9 (or f32::MIN) would saturate to -inf there; -1e4 is still ~10^4 below any realistic scaled dot product, so softmax returns exactly 0 for those positions in every supported dtype.