pub fn additive_padding_mask(mask: &Tensor, dtype: DType) -> Result<Tensor>Expand description
Turn a (batch, seq_len) padding mask (1 = real token, 0 = pad) into an
additive (batch, seq_len) bias in dtype.
The bias is 0 at real tokens and attention_mask_fill_value at pads,
and is meant to be added to the attention scores before softmax.
Multiplying the probabilities after softmax is not equivalent: it leaves the
surviving probabilities un-renormalised and still lets pad positions steal
mass.
The two ported architectures that take an additive mask want it at
different ranks — ESM-2 as (batch, 1, 1, seq), AMPLIFY as (batch, seq)
which it broadcasts itself — so this returns the 2-D core and each model
reshapes.