Loss landscape
The surface the loss forms over all parameters. Optimisers assume it is locally smooth.
Related
2 articles
The fractal optimizer that was plain gradient descent
A blueprint reported a fractal-orbit optimizer reaching loss 0.0176 on two-spirals. We ran its code. The fractal term never switched on: in 36,000 steps the measured dimension never crossed the threshold that activates it.
The Mandelbrot question we have not answered
If the loss landscape is self-similar across scales, a fixed-window optimiser is the wrong instrument by construction. A question, written down before the work, with what it would take to become a claim.