The fractal optimizer that was plain gradient descent

A blueprint reported a fractal-orbit optimizer reaching loss 0.0176 on two-spirals. We ran its code. The fractal term never switched on: in 36,000 steps the measured dimension never crossed the threshold that activates it.

1 Nov 2026 · 3 min read

The Mandelbrot question asks whether a loss landscape could be self-similar across scales, and whether an optimizer that sees across scales would beat AdamW. A blueprint in our corpus went further: it described a PureFractalOrbitOptimizer, shipped its code, and reported a benchmark 1.

reportedfinal loss on two-spirals
SGD + momentum0.57231
AdamW0.00057
PureFractalOrbitOptimizer0.01761

The benchmark was generated inside a notebook session. Nobody had run it on our machine. So we did 2.

What the code actually does

Read past the vocabulary (Buddhabrot orbits, Haar wavelets, Rényi entropy) and the update rule is one line:

θ ← θ − lr · Φ(D_q) · ∇L        Φ(D_q) = 1 / (1 + 0.2 · max(0, D_q − 1))

It keeps the last 32 positions of each parameter tensor, projects them to 2D with PCA, box-counts the result at four scales and fits a slope, D_q. Φ can only shrink the step, and only when D_q is above 1. Otherwise it is exactly 1 and the update is plain gradient descent with weight decay.

The control that matters

Same task, same network (2–64–64–1, tanh), full batch, 2,000 steps, five seeds. The decisive row is the last one: the same optimizer with the fractal scaling switched off.

optimizerfinal loss, median of 5rangeaccuracy
SGD + momentum0.032010.02715–0.0377699.3%
AdamW 3 40.000530.00044–0.00732100%
fractal orbit0.565660.56561–0.5657670.2%
fractal orbit, scaling off0.565660.56561–0.5657670.2%

On and off are identical to five decimals. The fractal part did nothing.

Why: the dimension never crosses 1

We recorded every D_q the optimizer computed:

learning rateD_q medianD_q maxsteps with D_q > 1
0.10.6881.0000 of 12,000
0.50.7281.0000 of 12,000
1.00.7041.0011 of 12,000

Thirty-two points along an optimisation path, flattened to two dimensions, trace something close to a curve, and a curve has dimension about 1. The scaling waits for a value above 1 that this estimator, on this window, almost never produces. The mechanism is not wrong. It is switched off by construction.

The reported 0.01761 did not reproduce. What we measured is what plain SGD without momentum does here: it stalls on the plateau at about 0.566, the same plateau the blueprint attributed to SGD with momentum.

What this does and does not settle

  • It settles this implementation. As written, it is SGD with decoupled weight decay plus five times the compute.
  • It does not settle the question. Whether loss landscapes are self-similar is still open. This estimator cannot see it: 32 samples in 2D are too few to measure a dimension above 1.
  • A fair next attempt needs a dimension estimate that can actually exceed 1 (longer windows, more projected dimensions) and the same switched-off control, run before anyone reports a win.

Why we publish a negative result

Because the blueprint read well. It had a mechanism, a table and a plausible story about chaotic exploration settling into a basin. Running the scaled and unscaled versions side by side took a few minutes and removed the story completely.

A component that is never activated reports its success through everything around it.

The code, the raw results and the probe are in the repository. Re-run them.

References
  1. Fractal Optimization Benchmark: PureFractalOrbitOptimizer vs SGD and AdamW (two-spirals). blueprint, generated in a NotebookLM session (2026)synthesiscorpus: ~/fractal-optimizer
  2. AlteranAI. Fractal-orbit optimizer reproduction: two-spirals, five seeds, D_q probe. github.com/misomalcek/alteranai/research (2026)our recordcorpus: ~/fractal-optimizer
  3. D. P. Kingma, J. Ba. Adam: A Method for Stochastic Optimization. ICLR 2015not in corpus
  4. I. Loshchilov, F. Hutter. Decoupled Weight Decay Regularization. ICLR 2019not in corpus
Our records

The measurements in this article come from these records in the running system.

  • research/bench_fractal.json — five seeds × four optimizers, 2,000 steps, torch 2.9.1 on CPU
  • research/probe_dq.py — D_q recorded at every step, three learning rates
Read next