Linear Regression from Scratch: Gradient Descent vs OLS

A linear regression fitted two ways with my own NumPy code: iteratively, with a one-layer network (Dense + MSELoss) trained by gradient descent (500 epochs, learning rate 0.01), and in closed form, with an OLS class that solves the normal equation. If both are correct they reach the same coefficients. Use the GD + OLS button on each chart to overlay the closed-form fit. Data are synthetic with known true coefficients.

Code: ml-from-scratch @ 7791264

Simple linear regression

100 points from y = 3x + 2 + noise (σ = 0.5).

Parameter	Gradient descent	OLS (closed form)	\|GD − OLS\|
intercept	1.9966	1.9977	1.1e-03
slope	3.0514	3.0596	8.2e-03

Multiple linear regression (2 features)

100 points from y = 3·x1 + 1.5·x2 + 2 + noise (σ = 0.5). Drag to rotate, scroll to zoom. Gray segments are residuals: the vertical distances that least squares minimizes.

Parameter	Gradient descent	OLS (closed form)	\|GD − OLS\|
intercept	1.9803	1.9804	6.2e-05
x1	2.9508	2.9508	2.1e-05
x2	1.4957	1.4959	1.3e-04