Concept

AdamW

also: Adam

Adam with decoupled weight decay: per-parameter step sizes from running averages of gradients (β₁) and squared gradients (β₂), both fixed constants.

Reference ↗
Related
2 articles

Glossary →