Support Vector Machines Deep Intuition PART-III (Kernels)

This post is the continuation of this and this. Please go through them before you read this post so that you can understand the below things.

For the people who already went through PART-I and PART-II, I updated that article due to the absence of a concept called Margins. please go through that( sorry for that 😅).

In a simple way to explain, SVMs kernels will try to change the dimension of the data from a lower dimension to a higher dimension(like looking a 2-D data in 3-Dimension) so that, Non-linear data in one dimension, looks linear in some other dimension.

Like this

Don’t worry I’ll elaborate with an example.

Let’s consider you have data of 2 columns X1 and X2, with 2 classes in the output Label Y. If the data is non-linear when you plot the data like X1 on the X-axis, and X2 on the Y-axis(or vice-versa), then the data will look like below.

what we have now is 2 Dimension, i.e., we are looking at data in 2-D dimension, so now to increase the dimension, now we’ll add another column in the data X3 which will be equal to X1 * X2 (multiplication of X1 and X2).

Before this, we have only 2 features(2-Dimensional), now X3 is there, so now data dimension is increased, now we plot the data in 3- dimensions with X1 on X-axis, X2 on the y-axis, and X3 on the z-axis. I can’t plot that here, but as we increased the dimension from 2-D to 3-D, the data may be in linear form. looking at different dimensions means literally looking at the plot of the data in different n-axis directions, and in which direction the data looks linear, we’ll draw the hyperplane there, and we fix that hyperplane as the final hyperplane. To get the optimized hyperplane we need to see the data in different dimensions.

The thing that we did above is an example of applying a kernel in SVMs

kernel function K(X1, X2) = X1*X2. ……………………… (1)

Kernels in SVMs are nothing but applying the different mathematical operations (different kernel functions) on the data and transforming that into another form for getting more dimensions, and deciding which hyperplane we need to select.

As like (1), we have different kernel functions in SVMs, which can apply to data for better results.

Some of them are:

1. Polynomial kernels
2. RBF kernels
3. Sigmoid kernels

I don’t want to explain what they are, the main aim of this article is to make understand what are kernels & why we are learning them. start learning one by one on your own as each kernel just represent a math formula.

Now, it’s time for the implementation.

Before that, you need to know something.

Actually, when you surf through the internet about SVMs Implementation, you’ll come across 2 types of Implementation.

1. LinearSVC
2. SVC

The main difference between these 2 Implementation methods is, linear SVC is particularly developed for dealing with only linear data, whereas SVC is developed for dealing with non-linear data by applying different kernels.

Also with SVC, we can use the linear kernel, but as LinearSVC is designed particularly for Linear data, LinearSVC is far better than SVC in the case of linear data. Considering other differences, Here you’ll find calculating loss and other differences. (Have a look at it, as they are explained in simple terminology)

Below is the Implementation of SVM’s with LinearSVC() and SVC():-

import pandas as pd
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.svm import LinearSVC, SVC
from sklearn import metrics
X,y = make_classification(n_samples=200,n_features=10,random_state=20)
X_train,X_test,y_train,y_test = train_test_split(X,y,random_state = 43)
Model = LinearSVC()
Model.fit(X_train, y_train)
print("Model Trained")
predictions = Model.predict(X_test)
print("predictions made")
print(type(predictions[0]))
print(predictions[:5])
print(type(y_test[0]))
Model1 = SVC()
Model1.fit(X_train, y_train)
predictions1 = Model1.predict(X_test)
print("Linear SVC's accuracy is:"+str(round(metrics.accuracy_score(y_test, predictions)*100, 2)))
print("SVC's accuracy is:"+str(round(metrics.accuracy_score(y_test, predictions1)*100, 2)))

yeahhhh…

I think I covered most of the SVM’s theory in 3 parts, still, if something you want to add here or want me to explain a missing concept here, lemme know it from here.

you can follow me on different platforms LinkedIn, Github and, medium.

Happy Learning✌.