Learning from examples
Normal code follows rules a person wrote. Machine learning works the other way round: give the computer examples with the right answers, and it finds the rule itself. Here the examples are café receipts. Nobody tells it what a cup costs.
Each example has a feature, the input (cups bought, x), and a label, the right answer (the bill, y). The three receipts are 1 → $3, 2 → $6, 3 → $9. Each becomes a dot on the chart: cups across, bill up.
The model is a guess with one knob: bill = w × cups. The number w is a weight, a value that training is allowed to change. It starts at w = 0, so the model's line lies flat along the bottom.
Training
For each example the model predicts w × x. At w = 0 that is $0 every time. The error is prediction minus label: −3, −6, −9. The red bars show how far each guess is from the dot.
The loss turns all the errors into one number. Square each error, then take the average: (9 + 36 + 81) ÷ 3 = 42. Squaring makes every error count as positive and punishes big misses most. Lower is better.
The gradient says how the loss changes as w grows. Here it is −28: a bigger w means a smaller loss. So step the other way, scaled by a learning rate of 0.075: w = 0 − 0.075 × (−28) = 2.1.
Training repeats those five lines. w goes 2.1, 2.73, 2.92, 2.98 while the loss falls 3.78, 0.34, 0.031. Each step is smaller, because the slope of the loss flattens near the bottom. w settles near 3: the model has learned $3 a cup.
Bigger models
A neural network stacks many such units in layers. Each unit multiplies its inputs by weights, adds them up, then applies an activation such as ReLU, which turns negatives into 0. Training is the same loop: loss, gradient, small steps, over millions of weights.
Keep some examples back as test data the model never trains on. A model with too much freedom can bend through every training dot yet miss new ones: overfitting. The test receipt says 4 cups cost $12; the wiggly curve guesses $6.
A large language model is a huge neural network trained the same way, on text. Its one task: predict the next token (a word or a piece of one) from the tokens before it. To write, it adds that token to the text and repeats.
What happens
;; step 1/10 · rules-to-examples normal program rules + input → answer machine learning examples + answers → rules the task: from receipts, find the price of a cup nobody writes bill = 3 × cups into the code
The receipts are toy data from a made-up café that charges exactly $3 a cup, so the rule to discover is known. Real data is noisy, and real models also learn a bias (a starting value, as in bill = w × cups + b). Every number in the figure is computed: run the program on the code card with NumPy and it prints the same weights and losses (w: 2.1, 2.73, 2.92; loss: 42, 3.78, 0.34, 0.031).