KernelSVM

Example Usage

1from mltrain.supervised.KernelSVM import KernelSVM
2
3# Initialize the model
4model = KernelSVM(C=1.0, kernel_type='rbf', epochs=1000, learning_rate=0.01, gamma=0.5, coef0=0, degree=3)
5
6# Train the model
7model.train(X_train, y_train)
8
9# Make predictions
10predictions = model.predict(X_test)
11

Overview

The KernelSVM class implements a Support Vector Machine (SVM) model with various kernel functions. This class supports training with different kernels such as linear, polynomial, radial basis function (RBF), and sigmoid. The model can be used to classify data points based on the provided features and labels.

Hyperparameters

• C (default=1.0): Regularization parameter. The strength of the regularization is inversely proportional to C.
• kernel_type (default='rbf'): The type of kernel function to use ('linear', 'polynomial', 'rbf', 'sigmoid').
• epochs (default=1000): The number of iterations over the training data.
• learning_rate (default=0.01): The learning rate for the gradient descent optimization.
• gamma (default=0.5): Kernel coefficient for 'rbf', 'polynomial', and 'sigmoid'.
• coef0 (default=0): Independent term in kernel function. It is only significant in 'polynomial' and 'sigmoid'.
• degree (default=3): Degree of the polynomial kernel function ('polynomial').

Attributes

• alpha (numpy.ndarray): Dual coefficients of the support vectors.
• b (float): Bias term in the decision function.
• support_vectors (numpy.ndarray): The support vectors identified during training.
• support_labels (numpy.ndarray): Labels of the support vectors.

Methods

__init__(self, C=1.0, kernel_type='rbf', epochs=1000, learning_rate=0.01, gamma=0.5, coef0=0, degree=3)

Initializes the Kernel SVM model with the specified hyperparameters.

• Args:
  • C (float): Regularization parameter.
  • kernel_type (str): Type of kernel function.
  • epochs (int): Number of training iterations.
  • learning_rate (float): Learning rate for gradient descent.
  • gamma (float): Kernel coefficient.
  • coef0 (float): Independent term in kernel function.
  • degree (int): Degree of polynomial kernel function.

kernel(self, x1, x2)

Computes the kernel function between two data points.

• Args:
  • x1 (numpy.ndarray): The first data point.
  • x2 (numpy.ndarray): The second data point.
• Returns:
  • float: The result of the kernel function.

train(self, X, y)

Trains the Kernel SVM model using the provided dataset.

• Args:
  • X (numpy.ndarray): The training data.
  • y (numpy.ndarray): The target labels.
• Notes:
  • The training process involves updating dual coefficients and bias term using gradient descent.
  • The kernel matrix is precomputed to optimize the training process.

predict(self, X)

Predicts the class labels for the provided data points.

• Args:
  • X (numpy.ndarray): The data points to classify.
• Returns:
  • numpy.ndarray: An array of predicted class labels.

accuracy(self, y_true, y_pred)

Computes the accuracy of the model.

• Args:
  • y_true (numpy.ndarray): The true class labels.
  • y_pred (numpy.ndarray): The predicted class labels.
• Returns:
  • float: The accuracy score.

confusion_matrix(self, y_true, y_pred)

Computes the confusion matrix to evaluate the classification accuracy.

• Args:
  • y_true (numpy.ndarray): The true class labels.
  • y_pred (numpy.ndarray): The predicted class labels.
• Returns:
  • numpy.ndarray: The confusion matrix.

Additional Notes

- The model uses gradient descent to optimize the dual coefficients and bias term.
- The kernel matrix is computed during training to facilitate the use of non-linear decision boundaries.
- Choosing an appropriate kernel function and hyperparameters is crucial for model performance.