This interactive sandbox visualizes optimization algorithms for finding minima in 1D and 2D functions: Hill Climbing, Gradient Descent, Simulated Annealing, Nelder-Mead (Simplex), and Coordinate Descent. Each algorithm is implemented as a JavaScript generator, allowing step-by-step execution so you can pause, resume, and observe how they navigate the optimization landscape.

Switch between 2D f(x) (canvas) and 3D f(x,y) (Plotly surface + contour) modes. The path is shown as a magenta trail with a bright current marker. For Simulated Annealing, the marker color transitions from red (hot) to blue (cold). For Nelder-Mead, a green simplex triangle with Best/Good/Worst vertices and an orange centroid shows the "Amoeba" flipping across the landscape.

Hill Climbing greedily moves to the neighboring point with the lowest value—it can get stuck in local minima (candidate dots show evaluated points). Gradient Descent follows the negative gradient with a tangent line at each step. Simulated Annealing uses Metropolis-Hastings: it may accept worse moves with probability exp(−ΔE/T). Nelder-Mead uses a simplex (triangle) that reflects, expands, contracts, or shrinks. Coordinate Descent minimizes along one axis at a time (axis-aligned steps).

Use the function presets to explore: 2D (Bowl, Peaks, Jagged, Double Well, Triple Valley, Gentle Waves) and 3D (Paraboloid, Rosenbrock, Two Basins, Three Basins, Mild Rastrigin, Saddle Valleys). Drag on the graph to set the initial position.

The Algorithms

All five methods search for a minimum of a function, but differ in how they sample the landscape and whether they can escape local minima:

Algorithm	Idea	Escapes local minima?
Hill Climbing	Greedily move to the best neighboring point.	No — gets stuck.
Gradient Descent	Step along the negative gradient: x ← x − α∇f.	No — descends to nearest basin.
Simulated Annealing	Metropolis rule: sometimes accept worse moves, less often as it cools.	Yes — probabilistically.
Nelder–Mead	A simplex (triangle) reflects/expands/contracts/shrinks — derivative-free.	Partially — depends on start.
Coordinate Descent	Minimize along one axis at a time.	No — axis-aligned, local.

The key to Simulated Annealing is the Metropolis acceptance probability for a move that worsens the objective by ΔE at temperature T:

Simulation

The interactive simulator is below. Use the controls to explore the concepts described above.

 

Parameters

• Dimension: 2D f(x) or 3D f(x,y). 3D mode shows Plotly surface (top) and contour (bottom) plots.
• Algorithm: Hill Climbing, Gradient Descent, Simulated Annealing; in 3D also Nelder-Mead (Simplex) and Coordinate Descent.
• Target Function: 2D (Bowl, Peaks, Jagged, Double Well, Triple Valley, Gentle Waves); 3D (Paraboloid, Rosenbrock, Two Basins, Three Basins, Mild Rastrigin, Saddle Valleys, Egg Holder).
• Step Speed: Milliseconds between algorithm steps (10–500 ms).
• Learning Rate (α) (Gradient Descent): Step size [0–5].
• Step Size (Hill Climbing, Nelder-Mead, Coordinate Descent): Neighbor/simplex scale [0–5].
• Cooling Rate, Initial Temp (Simulated Annealing): Temperature control.
• Start X, Start Y (3D): Sliders or drag on the plot to set initial position.

Buttons

• Start/Pause: Runs the algorithm step-by-step. Click again to pause.
• Step: Advances one iteration manually.
• Reset: Resets to the Start X/Y position and clears the path trail.

Visualization Features

• 2D Canvas: Function curve (cyan), path trail (magenta), current point (glowing marker).
• 3D Plotly: Surface plot (top), contour plot (bottom), trajectory, yellow current marker.
• Hill Climbing: Small orange dots show all candidate points evaluated at each step.
• Tangent Line (Gradient Descent): Dashed yellow line shows slope at current point.
• Search Radius (Simulated Annealing): Shrinking circle; marker color red→blue.
• Nelder-Mead Simplex: Green triangle (Best=green, Worst=red), orange centroid, operation label (e.g., Reflection, Inside Contraction).
• Draggable Start: Drag on the 2D canvas or contour plot to set the initial position.
• Stats Panel: X, f(X), iteration, slope/temperature/mode.
• Log Console: SA moves; Nelder-Mead operation per step.

Limitations

• Low-dimensional sandbox. Only 1D and 2D functions are optimized so the path can be drawn. Real optimization lives in hundreds to millions of dimensions, where this geometric intuition only partly transfers.
• Hand-picked test functions. The presets (Bowl, Rosenbrock, Rastrigin, etc.) are classic toy landscapes with known minima; they illustrate behavior but are not surrogates for any real objective.
• No convergence guarantees shown. The demo visualizes trajectories, not stopping criteria, tolerances, or proofs of convergence; runs simply step until paused.
• Sensitive to hyperparameters. Learning rate, step size, and the cooling schedule strongly affect outcomes; poor choices diverge or stall, and the tool does not auto-tune them.
• Single run, no statistics. Stochastic methods (Simulated Annealing) vary run-to-run; there is no averaging over seeds, so a single path is not representative.
• Smooth/cheap evaluations assumed. Gradient Descent needs differentiable functions and the demo evaluates f cheaply; noisy, expensive, or non-differentiable objectives (where Bayesian/evolutionary methods shine) are out of scope.