Objectives

Students will be able to:

  • understand a decision tree is a model that makes predictions by repeatedly splitting data into branches based on feature tests, ending in leaf nodes that represent the final decision or class.

Supervised Learning

Supervised learning is a type of machine learning where a model is trained on labelled data — examples that already include the correct answers — so it can learn to predict the right output for new, unseen inputs.

Machine Learning

Theme A4

Parameters

Parameters are the values the model learns from the data during training.

For example, in a simple model y = mx + c where

  • m is the slope
  • b is the intercept

The slope m tells us how much the predicted output changes when the input increases by one unit.

A positive slope means y increases as x increases, while a negative slope means y decreases.

The intercept b represents the predicted value of y when x = 0

It shows where the line crosses the y-axis and provides a baseline prediction.

These parameters are learned automatically during training by fitting the model to the data, rather than being set manually by the programmer.

Hyperparameters

While parameters like the slope and intercept are learned from the data, some values must be chosen before training begins.

These are called hyperparameters.

You have already encountered hyperparameters in other models:

  • choosing the value of k in
  • setting the maximum depth of a decision tree

These choices are made before training the model begins and can strongly influence how the model learns and how complex the final model becomes.

Play with the KNN simulator. For any dataset, is there a situation where the prediction differs for different values of K?

This would be a indication that setting the value of a hyperparameter such as K can impact the accuracy of prediction that the model makes.

And so hyperparameter tuning is conducted to get optimal values for hyperparameters before training begins.

Hyperparameter Tuning

Hyperparameters are model parameters set before training. They influence how accurate the model's predictions will be.

An example of a hyperparameter is the depth of a decision tree.

A tree that is too deep will not generalise well - it will overfit.

A decision tree that is too shallow has insufficient depth to capture the patterns in the data. As a result, it makes overly simple decisions based on only a small number of features or splits.

This is known as underfitting.

By tuning hyperparameters such as tree depth before training, the model's performance can be optimised.

Buffer Your Notes

Now that you have some knowledge of the nature of Decision Trees, is a useful resource you can use to consolidate your knowledge.

Take some time to explore this resource. There may be opportunities to develop your understanding and buffer your notes on Decision Trees!

Glossary

Machine Learning

Supervised Learning

Unsupervised Learning

Labels

Training

Training data

Test data

Features

Exercises

Tags

Machine Learninglabels Supervised learning Unsupervised learning features model training data trainingtest data