Gradient descent
Moving parameters a small step against the gradient of the loss, repeatedly, towards a minimum: water running downhill.
Reference ↗Related
3 articles
The fractal optimizer that was plain gradient descent
A blueprint reported a fractal-orbit optimizer reaching loss 0.0176 on two-spirals. We ran its code. The fractal term never switched on: in 36,000 steps the measured dimension never crossed the threshold that activates it.
The Mandelbrot question we have not answered
If the loss landscape is self-similar across scales, a fixed-window optimiser is the wrong instrument by construction. A question, written down before the work, with what it would take to become a claim.
Why the errors came from trying to be clever
One day of mistakes, read back. Every one came from reasoning like an intelligence. Every correct step came from reading what was there and following the gradient.