Linear Regression Model

Example Usage

1
2
3
4
5
6
7
8
9
10
11
# Import the LinearRegression model
from mltrain.models import LinearRegression

# Initialize the model
model = LinearRegression(learning_rate=0.001, epochs=500)

# Train the model
model.train(X_train, y_train)

# Make predictions
predictions = model.predict(X_test)

Overview

The LinearRegression class implements a simple linear regression model using gradient descent optimization. This model can be used to predict continuous target variables based on input features. The class provides methods to train the model, make predictions, and compute the loss.

Hyperparameters

• learning_rate (default=0.01): The step size used for each iteration of gradient descent. A smaller value results in slower but more accurate convergence, while a larger value speeds up training but may lead to instability.
• epochs (default=1000): The number of times the model iterates over the entire dataset during training. More epochs lead to better training but increase computation time.

Attributes

• W (numpy.ndarray): The learned weights of the model, updated during training to minimize the loss.
• b (float): The bias term, also updated during training to minimize the loss.

Methods

__init__(self, learning_rate=0.01, epochs=1000)

Initializes the linear regression model with specified hyperparameters.

• Args:
  • learning_rate (float): The learning rate for gradient descent. Default is 0.01.
  • epochs (int): The number of epochs for training the model. Default is 1000.

predict(self, X)

Makes predictions based on the current weights and bias.

• Args:
  • X (numpy.ndarray): The input features, a 2D array of shape (num_samples, num_features).
• Returns:
  • numpy.ndarray: Predicted values, a 1D array of shape (num_samples,).

compute_loss(self, Y, Y_pred)

Computes the Mean Squared Error (MSE) loss between the actual and predicted values.

• Args:
  • Y (numpy.ndarray): The actual target values, a 1D array of shape (num_samples,).
  • Y_pred (numpy.ndarray): The predicted values, a 1D array of shape (num_samples,).
• Returns:
  • float: The mean squared error loss.

train(self, X, Y, print_loss=False)

Trains the linear regression model using gradient descent. The method updates the weights W and bias b to minimize the loss function.

• Args:
  • X (numpy.ndarray): The input features, a 2D array of shape (num_samples, num_features).
  • Y (numpy.ndarray): The actual target values, a 1D array of shape (num_samples,).
  • print_loss (bool): Whether to print the loss during training for every 100 epochs. Default is False.
• Side effects:
  • Updates the weights (W) and bias (b) of the model.

Additional Notes

- The model uses gradient clipping to avoid gradient explosion, ensuring that weight updates stay stable.
- The train method includes an optional argument print_loss to monitor the training progress every 100 epochs.
- The input features X should be scaled for optimal performance.