Finite Difference Playground

Estimate the slope of f at x0 from a few nearby samples. Pick a formula, then shrink h and follow the marker on the right: the error first falls, then rises again. The zoom shows why. Finally add noise to the samples.

 | 

The samples and the slope they give

Gray: f(x). Dashed: true tangent. Colored: line with the estimated slope. Dots: samples used.
Zoom on a window of width 4h around x0. Small dots: the values the computer actually returns.

Error vs. step size

Log-log: |error| against h. Right side: truncation (slope = order). Left side: rounding and noise (slope -1).