Types of Machine Learning

Types of Machine Learning

Machine learning systems learn from data, but they learn in different ways depending on what kind of feedback the data provides. That difference, rather than the specific algorithm, is the first thing to identify when framing any ML problem. "Is this supervised or unsupervised?" decides what data you need to collect, how you'll measure success, and which tools apply.

1. Supervised learning: learning from labelled examples

Every training example comes with the correct answer (the label). The model learns to map inputs to outputs, then predicts outputs for new inputs.

• Classification predicts a category: spam / not spam, fraud / legitimate, which product category an image shows, which support queue a ticket belongs to.
• Regression predicts a number: a house price, delivery time, next week's demand.

The catch is labels. They're often expensive (doctors labelling scans), slow (you only learn whether a loan defaulted after years), or noisy (inconsistent human judgements). Much of real-world ML effort goes into getting good labels. Details are in Supervised Learning.

2. Unsupervised learning: finding structure without answers

The data has no labels. The algorithm looks for structure on its own.

• Clustering groups similar items: customer segments by purchase behaviour, grouping news articles by topic. K-Means needs you to choose the number of clusters, and DBSCAN finds clusters of arbitrary shape and flags outliers.
• Dimensionality reduction compresses many features into a few while keeping the important variation: PCA for visualization or noise reduction, embeddings that map text or products into dense vectors.
• Anomaly detection learns what "normal" looks like and flags what doesn't fit: unusual transactions, failing machines, intrusions.
• Association rules find items that occur together ("people who buy X often buy Y").

The catch is evaluation. Without correct answers, "is this clustering good?" is partly a judgement call. It's usually validated by whether the result is useful downstream: do the segments respond differently to campaigns?

3. Semi-supervised and weak supervision

In practice you often have a little labelled data and a lot of unlabelled data. Semi-supervised methods use both, for example training on the labelled set, labelling the unlabelled data with confident predictions, and retraining. Weak supervision creates approximate labels from rules or heuristics. Both reduce labelling cost.

4. Self-supervised learning: the data labels itself

The labels are derived automatically from the data itself, typically by hiding part of it and predicting it:

Text:   "The payment service timed out"   →   hide the last word, predict "out"
Images: hide a patch of the image          →   predict the missing pixels

No human labelling is required, so models can train on internet-scale data. This is how large language models are pre-trained (next-token prediction), and how many image and audio models learn general representations. Those "foundation models" are then fine-tuned with smaller amounts of supervised or preference data for specific tasks.

5. Reinforcement learning: learning by trial and reward

An agent takes actions in an environment and receives rewards or penalties. There are no correct answers per step, only feedback on outcomes, sometimes long delayed. The agent learns a policy (what to do in each situation) that maximizes total reward over time.

• Game-playing (Go, chess, video games), robotics, resource allocation, some recommendation and ad-bidding systems.
• RLHF (reinforcement learning from human feedback) is used to tune LLMs toward responses people prefer.

The catches: it needs many interactions (often in simulation), reward functions are hard to design (agents exploit loopholes in badly specified rewards), and exploring by trial and error is risky in the real world.

Choosing the type for a problem

Framing examples:

• "Flag transactions likely to be fraudulent." With historical fraud labels, it's supervised classification. With no labels, it's anomaly detection (unsupervised). Often both are combined.
• "Group our customers for marketing." This is clustering (unsupervised).
• "Estimate how long a delivery will take." This is supervised regression.
• "Answer questions about our documentation." Use a pre-trained (self-supervised) LLM plus retrieval, with no training at all.

Common pitfalls across all types

• Leakage: training data containing information that won't exist at prediction time. It produces great offline scores and poor real performance.
• Unrepresentative data: a model trained on one region or season fails in another.
• Optimizing the wrong metric: accuracy on imbalanced data, or a reward that can be gamed.
• Ignoring the simple baseline: always compare against a rule-based or trivial model. It sometimes wins.

Follow-up questions this topic invites — and their answers

Q: What's the main difference between supervised and unsupervised learning? A: Supervised learning trains on examples paired with correct answers to predict those answers for new data. Unsupervised learning has no answers and discovers structure (clusters, low-dimensional patterns, anomalies) in the data itself.

Q: Is an LLM supervised or unsupervised? A: Its pre-training is self-supervised: the next token in real text acts as the label, so no human labels are needed. It's then typically refined with supervised fine-tuning on instruction examples and with preference-based methods such as RLHF.

Q: How do you evaluate a clustering? A: With internal measures such as silhouette score (how compact and well-separated clusters are), stability across runs, and most importantly usefulness: do the clusters make sense to domain experts, and do they lead to different, better decisions downstream?

Q: When is reinforcement learning the right tool? A: When the problem is a sequence of decisions with delayed outcomes, you can interact with an environment (ideally a simulator), and the goal can be expressed as a reward. For one-off predictions from historical data, supervised learning is simpler and more reliable.

Q: How would you approach a problem with very few labels? A: Start from a pre-trained model (transfer learning or embeddings) and fine-tune it on the few labels. Use semi-supervised or weak-supervision techniques. Apply active learning to label the most informative examples first. And check whether a rule-based baseline is already good enough.