Skip to main content

Module blocks

ferritin_plms::esm3::layers

Module blocks 

Source
Expand description

ESM3 UnifiedTransformerBlock: plain MHA + geometric attention + SwiGLU FFN.

Structsยง

SwiGLU
UnifiedTransformerBlock