Supervised Learning — K Nearest Neighbors Algorithm (KNN)

This article explains one of the simplest machine learning algorithm K Nearest Neighbors(KNN). KNN classifier and KNN regression are explained with examples in this article.

k Nearest Neighbors Classification

K nearest neighbors algorithm basically predicts on the principle that the data is in the same class as the nearest data. According to the name of the algorithm, “nearest neighbors” represents the closest data and “k” represents how many closest data is chosen. K value is a hyper parameter so it is tuned by the user and each trial usually gives different results. For example, let’s show the data in with 2 inputs(features) in the coordinate plane which x and y axis represent the inputs(features) of the dataset. As seen in Figure 1, blue circles and orange triangles represent our dataset, stars represent the data which is wondered the classes and they have put the model which is created with knn algorithm in order to predict classes of them. The algorithm assigns the data to the class of the closest data. In short Figure 1 illustrates that how does the algorithm predict a class of new value with different k(k=1(above), k=3(below)) values. This kind of problem that there are 2 outputs which are circle and triangle are called as binary classification. But knn also can apply for multiple classification. Now let’s take a closer look at the 3-neighbors classification example.