Concept

Mixture of experts

also: MoE

A model whose feed-forward layers are many experts, of which only a few run per token. Large on disk, cheaper per token.

Reference ↗
Related
1 article

Glossary →