الدرس S2 · Special topics

How machine learning works

لم يُترجم بعد؛ يُعرض باللغة الإنجليزية

Instead of writing the rule, let the computer learn it from examples. Follow a model with a single weight as it learns the price of a coffee from three receipts: it predicts, measures its error as a loss, and uses gradient descent to tilt its line step by step until the loss is almost zero. Then see how the same loop scales up to neural networks, why models are judged on test data, and what a large language model is predicting.

12 دقيقة10 خطوات5 أسئلةIntermediate

شكل S2 · تفاعلي

جارٍ تحميل الرسم…

أو ← →

اختبر نفسك

5 أسئلة سريعة

اختر إجابة لتعرف السبب. تُحفَظ نتيجتك مع تقدّمك.

01In the receipts example, what are the feature and the label?

A feature is an input the model sees (cups bought). A label is the right answer it should learn to give (the bill).

02With w = 0 the errors are −3, −6 and −9. What is the mean squared error loss?

Square each error (9, 36, 81), add them (126) and take the average over the three examples: 126 ÷ 3 = 42.

03The gradient of the loss at w = 0 is −28 and the learning rate is 0.075. What is w after one step of gradient descent?

Gradient descent steps against the gradient: w = w − learning rate × gradient = 0 − 0.075 × (−28) = 2.1.

04A model fits every training example perfectly but makes big mistakes on new examples. What is this called?

Overfitting means the model has learned the training data too closely, so it does not generalise. That is why models are judged on test data they never trained on.

05What is a large language model trained to do?

A large language model is a big neural network trained to predict the next token. It writes by adding the token it picks to the text and predicting again.

المصطلحات الأساسية

المسرد

Machine learning: Building programs that improve at a task by learning from examples, instead of following rules a person wrote.
Feature and label: A feature is an input the model sees (cups bought). A label is the right answer for that example (the bill).
Model and weight: A model turns features into a prediction. Its weights are the numbers it can adjust while learning, such as w in bill = w × cups.
Loss: One number for how wrong the model is. Mean squared error squares every error and averages them.
Gradient descent: Repeatedly changing each weight a little in the direction that lowers the loss: w = w − learning rate × gradient.
Learning rate: How big each gradient-descent step is. Too small is slow; too big can overshoot the lowest point.
Overfitting: Fitting the training data so closely that the model does badly on new data. Checked with held-back test data.
Token: A piece of text a language model works with: a word or part of a word.

تفاصيل إضافية

الملاحظات والمصادر

The receipts are toy data from a made-up café that charges exactly $3 a cup, so the rule to discover is known. Real data is noisy, and real models also learn a bias (a starting value, as in bill = w × cups + b).
Every number in the figure is computed: run the program on the code card with NumPy and it prints the same weights and losses (w: 2.1, 2.73, 2.92; loss: 42, 3.78, 0.34, 0.031). The full program: from numpy import array, mean x = array([1, 2, 3]) # feature: cups y = array([3, 6, 9]) # label: the bill in $ w = 0.0 for step in range(6): pred = w * x err = pred - y loss = mean(err ** 2) grad = mean(2 * err * x) w = w - 0.075 * grad print(step + 1, round(float(loss), 4), round(float(w), 4))
For this model the loss is a bowl-shaped curve with its lowest point at w = 3, so every gradient step moves closer. Bigger models have far bumpier loss surfaces, and a learning rate that is too large can overshoot.
The wiggly curve is f(x) = 3x − (x − 1)(x − 2)(x − 3): it goes through all three training dots exactly, yet predicts $6 for 4 cups. Overfitting usually shows up as low training loss but high test loss.
The bars beside the candidate tokens are illustrative, not taken from a real model. Real language models score tens of thousands of possible tokens at every step.

اختبار يتاح بعد الخطوة الأخيرة.