Logistic Regression Model

Example Usage

1# Import the LogisticRegression model
2from mltrain.models import LogisticRegression
3
4# Initialize the model
5model = LogisticRegression(learning_rate=0.001, epochs=500)
6
7# Train the model
8model.train(X_train, y_train)
9
10# Make predictions
11predictions = model.predict(X_test)

Overview

The LogisticRegression class implements binary classification using gradient descent optimization. This model predicts the probability of a binary outcome based on input features. It uses sigmoid activation for prediction, log loss for optimization, and thresholding for class assignment.

Hyperparameters

• learning_rate (default=0.001): Step size for gradient updates.
• epochs (default=1000): Number of training iterations.
• threshold_value (default=0.5): Threshold for binary classification.

Attributes

• W (tf.Variable): The weight vector for input features.
• b (tf.Variable): The bias term for the model.

Methods

__init__(self, epochs=1000, learning_rate=0.001, threshold_value=0.5)

Initializes the logistic regression model with hyperparameters.

• Args:
  • learning_rate (float): Learning rate for optimization.
  • epochs (int): Number of training iterations.
  • threshold_value (float): Decision boundary for classification.

predict(self, X_pred)

Makes predictions based on the sigmoid output.

• Args:
  • X_pred (numpy.ndarray): The input data for prediction.
• Returns:
  • tf.Tensor: Predicted probabilities for each sample.

train(self, x_train, y_train, print_loss=False)

Trains the model using gradient descent, optimizing the weight vector and bias to minimize log loss.

• Args:
  • x_train (numpy.ndarray): The input training data.
  • y_train (numpy.ndarray): The true labels.
  • print_loss (bool): Option to print loss every 100 epochs.
• Side effects:
  • Updates weights W and bias b.

log_loss(self, y_true, y_pred)

Calculates binary cross-entropy loss (log loss) between true labels and predictions.

• Args:
  • y_true (numpy.ndarray): The true labels.
  • y_pred (numpy.ndarray): Predicted probabilities.
• Returns:
  • Mean binary cross-entropy loss.

predict_exact(self, X_pred)

Returns the predicted binary class (0 or 1) based on the threshold value.

• Args:
  • X_pred (numpy.ndarray): The input data for prediction.
• Returns:
  • Binary class predictions for each sample (0 or 1).

Additional Notes

- The input features should be scaled for better performance.
- The train method includes an optional argumentprint_loss to monitor progress.
- Binary classification results are based on the threshold_value.