Struct LoadOptions 

pub struct LoadOptions {
    pub dtype: DType,
    pub device: Device,
}

Device and dtype to load a model onto.

Defaults to F32, which is what every runner used before this existed.

Fields

dtype: DType

Element type to materialise weights as.

device: Device

Device to place weights on.

Implementations

impl LoadOptions

pub fn new(device: Device) -> Self

F32 on device.

pub fn with_dtype(self, dtype: DType) -> Self

Override the dtype (builder style).

pub fn validate(&self) -> Result<()>

Reject dtype/device combinations the backend cannot run.

candle’s CPU backend has no BF16 matmul, so a BF16 model on CPU fails partway through loading or on the first forward pass with a bare “unsupported dtype BF16 for op matmul”. Catching it here says what to do instead (ferritin-100.9).

The refusal is scoped to the CPU backend rather than to the dtype because Metal does run BF16: ferritin-100.19 loaded and ran both ESM2 and AMPLIFY there at BF16 without changing a single top-1 prediction. See tests/test_plm_dtype_parity.rs for the divergence and throughput tables that establish it. CUDA is assumed to behave like Metal here and remains unmeasured — the tests cover whatever accelerator the build actually has, so running them on a CUDA box is what closes that gap.

Trait Implementations

impl Clone for LoadOptions

fn clone(&self) -> LoadOptions

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LoadOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.