Objectives

Students will be able to:

  • understand that KNN is a supervised learning algorithm that attempts to classify/label a new data point in relation to existing data points.

Supervised Learning

Supervised learning is a type of machine learning where a model is trained on labelled data — examples that already include the correct answers — so it can learn to predict the right output for new, unseen inputs.

Machine Learning

Theme A4

K Nearest Neighbour

Prediction:

K Nearest Neighbour is a technique used to make predictions based on known (training) data.

In the simulation above, the training data has already classified/labelled what are cats and what are dogs.

For new data points, the KNN algirithm:

  • calculates the distance between the new data point and every other data point
  • selects the nearest K values
  • performs a majority vote and classifies the new data point as a member of the majority

KNN is a lazy learner. It doesn't have a strict rulebook. It bases its preditions not on rules, but on specific examples of the training data.

Worksheet

Make a copy of the worksheet and move it into your Theme A4 folder.

Complete the Worksheet

Share with your partner/group what you learned about collaborative filtering.

What kind of collaborative filtering does Spotify use to recommend songs to its account holders:

  • User-based CF
  • Item-based CF

Advantages and Disadvantages of KNN

KNNs are simple and effective.

They are versatile and make no assumptions about the data.

When multiple classifications/labels are required, KNNs are easy to implement.

However, KNNs require a lot of computational power, particularly for large data sets. This is because they have to calculate the distance between the new data point (instance) and all the existing instances. It then needs to sort these distances to determine the K neighbours.

They are sensitive to:

  • noisy data
  • missing values
  • outliers

Coding Task - Optional

Let's get coding!

Journal Task

Make a new entry in the Journal section of the booklet.

Did you discover anything new today? Do you have questions in your mind that you want answers to?

Make these notes in your journal.

Glossary

Machine Learning

Supervised Learning

Unsupervised Learning

Labels

Training

Training data

Test data

Features

Exercises

Tags

Machine Learninglabels Supervised learning Unsupervised learning features model training data trainingtest data