Python program for backpropagation
Here’s a simple implementation of feedforward neural network with backpropagation in Python:
- Neural Network Initialization: The
NeuralNetwork
class is initialized with parameters for the input size, hidden layer size, and output size. It also initializes the weights and biases with random values. - Sigmoid Activation Function: The
sigmoid
method implements the sigmoid activation function, which squashes the input to a value between 0 and 1. - Sigmoid Derivative: The
sigmoid_derivative
method calculates the derivative of the sigmoid function. It computes the gradients of the loss function with respect to weights. - Feedforward Pass: The
feedforward
method calculates the activations of the hidden and output layers based on the input data and current weights and biases. It uses matrix multiplication to propagate the inputs through the network. - Backpropagation: The
backward
method performs the backpropagation algorithm. It calculates the error at the output layer and propagates it back through the network to update the weights and biases using gradient descent. - Training the Neural Network: The
train
method trains the neural network using the specified number of epochs and learning rate. It iterates through the training data, performs the feedforward and backward passes, and updates the weights and biases accordingly. - XOR Dataset: The XOR dataset (
X
) is defined, which contains input pairs that represent the XOR operation, where the output is 1 if exactly one of the inputs is 1, and 0 otherwise. - Testing the Trained Model: After training, the neural network is tested on the XOR dataset (
X
) to see how well it has learned the XOR function. The predicted outputs are printed to the console, showing the neural network’s predictions for each input pair.
Python
import numpy as np class NeuralNetwork: def __init__( self , input_size, hidden_size, output_size): self .input_size = input_size self .hidden_size = hidden_size self .output_size = output_size # Initialize weights self .weights_input_hidden = np.random.randn( self .input_size, self .hidden_size) self .weights_hidden_output = np.random.randn( self .hidden_size, self .output_size) # Initialize the biases self .bias_hidden = np.zeros(( 1 , self .hidden_size)) self .bias_output = np.zeros(( 1 , self .output_size)) def sigmoid( self , x): return 1 / ( 1 + np.exp( - x)) def sigmoid_derivative( self , x): return x * ( 1 - x) def feedforward( self , X): # Input to hidden self .hidden_activation = np.dot(X, self .weights_input_hidden) + self .bias_hidden self .hidden_output = self .sigmoid( self .hidden_activation) # Hidden to output self .output_activation = np.dot( self .hidden_output, self .weights_hidden_output) + self .bias_output self .predicted_output = self .sigmoid( self .output_activation) return self .predicted_output def backward( self , X, y, learning_rate): # Compute the output layer error output_error = y - self .predicted_output output_delta = output_error * self .sigmoid_derivative( self .predicted_output) # Compute the hidden layer error hidden_error = np.dot(output_delta, self .weights_hidden_output.T) hidden_delta = hidden_error * self .sigmoid_derivative( self .hidden_output) # Update weights and biases self .weights_hidden_output + = np.dot( self .hidden_output.T, output_delta) * learning_rate self .bias_output + = np. sum (output_delta, axis = 0 , keepdims = True ) * learning_rate self .weights_input_hidden + = np.dot(X.T, hidden_delta) * learning_rate self .bias_hidden + = np. sum (hidden_delta, axis = 0 , keepdims = True ) * learning_rate def train( self , X, y, epochs, learning_rate): for epoch in range (epochs): output = self .feedforward(X) self .backward(X, y, learning_rate) if epoch % 4000 = = 0 : loss = np.mean(np.square(y - output)) print (f "Epoch {epoch}, Loss:{loss}" ) X = np.array([[ 0 , 0 ], [ 0 , 1 ], [ 1 , 0 ], [ 1 , 1 ]]) y = np.array([[ 0 ], [ 1 ], [ 1 ], [ 0 ]]) nn = NeuralNetwork(input_size = 2 , hidden_size = 4 , output_size = 1 ) nn.train(X, y, epochs = 10000 , learning_rate = 0.1 ) # Test the trained model output = nn.feedforward(X) print ( "Predictions after training:" ) print (output) |
Output:
Epoch 0, Loss:0.36270360966344145 Epoch 4000, Loss:0.005546947165311874 Epoch 8000, Loss:0.00202378766386817 Predictions after training: [[0.02477654] [0.95625286] [0.96418129] [0.04729297]]
Backpropagation in Neural Network
Machine learning models learn from data and make predictions. One of the fundamental concepts behind training these models is backpropagation. In this article, we will explore what backpropagation is, why it is crucial in machine learning, and how it works.
Table of Content
- What is backpropagation?
- Advantages of Using the Backpropagation Algorithm in Neural Networks
- Working of Backpropagation Algorithm
- Example of Backpropagation in Machine Learning
- Python program for backpropagation
A neural network is a network structure, by the presence of computing units(neurons) the neural network has gained the ability to compute the function. The neurons are connected with the help of edges, and it is said to have an assigned activation function and also contains the adjustable parameters. These adjustable parameters help the neural network to determine the function that needs to be computed by the network. In terms of activation function in neural networks, the higher the activation value is the greater the activation is.