Visualizing Support Vector Machines (SVM) with Multiple Explanatory Variables

Question

I was wondering if anyone was aware of any methods for visualizing an SVM model where there are more than three continuous explanatory variables. In my particular situation, my response variable is binomial, with 6 continuous explanatory variables (predictors), one categorical explanatory variable (predictor). I have already reduced the number of predictors and I am primarily using R for my analysis.

(I am unaware if such a task is possible/ worth pursuing.)

Thanks for your time.

Answer 1

Does it matter that the model is created in the form of SVM?

If no, I have seen a clever 6-D visualization. Its varieties are becoming popular in medical presentations. 3 dimensions are shown as usual, in orthographic projection. Dimension 4 is color (0..255) Dimension 5 is thickness of the symbol Dimension 6 requires animation. It is a frequency of vibration of a dot on the screen. In static, printed versions, one can replace frequency of vibration by blur around the point, for a comparable visual perception.

If yes, and you specifically need to draw separating hyperplanes, and make them look like lines\planes, the previous trick will not produce good results. Multiple 3-D images are better.

Answer 2

Have you looked into tourr package in R. This package does hyperplane reduction. In addition it has an optimizer that tries to find the best reduction.

There is a very nice video in https://www.youtube.com/watch?v=iSXNfZESR5I That shows what R is capable even beyound tourr package.

Also I refer you to https://stackoverflow.com/questions/8017427/plotting-data-from-an-svm-fit-hyperplane

Answer 3

Dimension reduction (like PCA) is an excellent way to visualize the results of classification on a high-dimensional feature space.

The simplest approach is to project the features to some low-d (usually 2-d) space and plot them. Then either project the decision boundary onto the space and plot it as well, or simply color/label the points according to their predicted class. You can even use, say, shape to represent ground-truth class, and color to represent predicted class.

This is true for any categorical classifier, but here's an SVM-specific example: http://www.ece.umn.edu/users/cherkass/predictive_learning/Resources/Visualization%20and%20Interpretation%20of%20SVM%20Classifiers.pdf

In particular, see figures 1a and 2a.