What's Your IQ

What machine learning is, in short

Machine learning is a branch of artificial intelligence in which computers learn patterns from data instead of being given explicit rules. A model is trained on examples, measured on examples it hasn't seen, and then used to make predictions about new cases.

Arthur Samuel, who coined the term in 1959, described it as giving computers the ability to learn without being explicitly programmed. Most of what's called AI today is machine learning.

Term coined: 1959, Arthur Samuel (IBM)
Needs: Data, a model, and a way to measure error
Main kinds: Supervised, unsupervised, reinforcement
Biggest trap: Overfitting: memorising instead of learning

CORE CONCEPT · TRAINING

Train a pass/fail model by hand

Twenty students: hours studied, and whether they passed. Drag the line to set the rule “predict pass above this many hours.” Training means finding the line with the fewest mistakes.

3.0 hPREDICT FAILPREDICT PASS

0 hhours studied10 h

passedfailedpredicted wrong

ACCURACY75%

CORRECT15

FALSE PASSES5

MISSED PASSES0

Training is this, done automatically: try many settings, measure the errors, keep the best. Real models adjust millions of numbers instead of one.

THE BIGGEST TRAP

More complex isn't always smarter

A model is judged on data it hasn't seen. Turn up the complexity: training error keeps falling, but test error falls then rises again as the model starts memorising noise.

Model complexity4 / 10

12345678910

Just right

TRAINING ERROR11%

TEST ERROR14%

The sweet spot: low test error. The model has learned the pattern without memorising noise.

Illustrative errors for a typical model. Purple: error on the training data. Pink: error on held-back test data.

KINDS OF LEARNING

Who gives the answers?

Supervised learningThe model learns from labelled examples, inputs paired with correct answers, then predicts answers for new inputs. The pass/fail model above is supervised.Used for:spam filters, house-price prediction, diagnosis from scans

TYPICAL METHODS

Linear and logistic regression

Decision trees and random forests

Neural networks

IN PRACTICE

From data to deployed model

Most of the work isn't the algorithm. Step through.

Collect dataGather examples that represent the real situation the model will face.Watch out:data that misses whole groups of people or cases.

WHERE IT RUNS

Spam and fraud

Recommendations

Medical imaging

Speech and language

Forecasting

Driver assistance

Still hard

Biased dataIf training data reflects past unfairness, the model learns it. Auditing data and outcomes is essential.

Data hungerMany methods need huge labelled datasets, which are costly and raise privacy questions.

InterpretabilityComplex models rarely explain why they made a decision.

Distribution shiftModels trained on yesterday's data can quietly fail when the world changes.

How we got here

1943Artificial neurons.McCulloch and Pitts described a mathematical model of a neuron.

1958The perceptron.Frank Rosenblatt built a machine that learned to classify simple images.

1959The term.Arthur Samuel coined 'machine learning' for his self-improving draughts program.

1986Backpropagation.Rumelhart, Hinton and Williams popularised the method still used to train neural networks.

2012Deep learning arrives.AlexNet, trained on GPUs, won the ImageNet challenge by a wide margin.

2017Transformers.The architecture behind today's large language models.

READING LIST

Pattern Recognition and Machine Learning· Christopher Bishop2006

The Hundred-Page Machine Learning Book· Andriy Burkov2019

Hands-On Machine Learning· Aurélien Géron2017

Deep Learning· Goodfellow, Bengio & Courville2016

Machine learning