Logistic Regression Machine Learning Algorithm in Python from Scratch

Let’s understand the basics of Logistic Regression

Introduction:

When we are implementing Logistic Regression Machine Learning Algorithm using sklearn, we are calling the sklearn’s methods and not implementing the algorithm from scratch.

In this article, I will be implementing a Logistic Regression model without relying on Python’s easy-to-use sklearn library. This post aims to discuss the fundamental mathematics and statistics behind a Logistic Regression model. I hope this will help us fully understand how Logistic Regression works in the background.

Note that this is one of the posts in the series Machine Learning from Scratch. You may like to read other similar posts like Gradient Descent From Scratch, Linear Regression from Scratch, Decision Tree from Scratch, Neural Network from Scratch

You may like to watch this article as a video, in more detail, as below:

General Terms:

Let us first discuss a few statistical concepts used in this post.

Sigmoid: A sigmoid function is an activation function. The output of the sigmoid function is always between a range of 0 to1.

Optimization: optimization is a process that maximizes or minimizes the variables or parameters of a machine learning model with respect to the selected loss function.

Logistic Regression in Python From Scratch

Import Libraries:

We are going to import NumPy and the pandas library.

import numpy as np
import pandas as pd

Load Data: