Basic Three Layer Neural Network in Python

- June 30, 2017

Introduction

As part of understanding neural networks I was reading Make Your Own Neural Network by Tariq Rashid. The book itself can be painful to work through, as it is written for a novice, not just in algorithms and data analysis, but also in programming. Although the code is a verbatim transcription from the text (see Source section), I published it to better understand how neural networks are designed, made easy by the use of a Jupyter Notebook, not to present this as my own work, although I do hope that this helps others develop their talents with data analytics.

Github Source: AzureNotebooks/Basic Three Layer Neural Network in Python.ipynb at master · JamesIgoe/AzureNotebooks (github.com)

Overview

The code itself develops as follows:

Constructor

set number of nodes in each input, hidden, output layer
link weight matrices, wih and who
weights inside the arrays are w_i_j, where link is from node i to node j in the next layer
set learning rate
activation function is the sigmoid function

Define the Training Function

convert inputs list to 2d array
calculate signals into hidden layer
calculate the signals emerging from hidden layer
calculate signals into final output layer
calculate the signals emerging from final output layer
output layer error is the (target actual)
hidden layer error is the output_errors, split by weights, recombined at hidden nodes
update the weights for the links between the hidden and output layers
update the weights for the links between the input and hidden layers

Define the Query Function

convert inputs list to 2d array
calculate signals into hidden layer
calculate the signals emerging from hidden layer
calculate signals into final output layer
calculate the signals emerging from final output layer

Load Libraries

 # for scientific computing with Python  
 import numpy  
   
 # for the sigmoid function expit()  
 import scipy.special

Develop Neural Network Class

 #neural network class definition  
 class neuralNetwork:  
     
   # for scientific computing with Python  
   import numpy  
   
   # for the sigmoid function expit()  
   import scipy.special  
   
   #initialise the neural network   
   def __init__(self, inputnodes, hiddennodes, outputnodes, learningrate):  
     
     #set number of nodes in each input, hidden, output layer  
     self.inodes = inputnodes   
     self.hnodes = hiddennodes   
     self.onodes = outputnodes   
   
     #link weight matrices, wih and who   
     #weights inside the arrays are w_i_j, where link is from node i to node j in the next layer   
     #w11 w21   
     #w12 w22 etc   
     self.wih = numpy.random.normal(0.0, pow(self.hnodes, -0.5), (self.hnodes, self.inodes))   
     self.who = numpy.random.normal(0.0, pow(self.onodes, -0.5), (self.onodes, self.hnodes))   
   
     #learning rate   
     self.lr = learningrate   
   
     #activation function is the sigmoid function   
     self.activation_function = lambda x: scipy.special.expit(x)   
   
     pass   
   
   #train the neural network   
   def train(self, inputs_list, targets_list):   
       
     #convert inputs list to 2d array   
     inputs = numpy.array(inputs_list, ndmin=2).T   
     targets = numpy.array(targets_list, ndmin=2).T   
   
     #calculate signals into hidden layer   
     hidden_inputs = numpy.dot(self.wih, inputs)   
   
     #calculate the signals emerging from hidden layer   
     hidden_outputs = self.activation_function(hidden_inputs)   
   
     #calculate signals into final output layer   
     final_inputs = numpy.dot(self.who, hidden_outputs)   
   
     #calculate the signals emerging from final output layer   
     final_outputs = self.activation_function(final_inputs)  
   
     #output layer error is the (target  actual)   
     output_errors = targets - final_outputs   
   
     #hidden layer error is the output_errors, split by weights, recombined at hidden nodes   
     hidden_errors = numpy.dot(self.who.T, output_errors)   
   
     #update the weights for the links between the hidden and output layers   
     self.who += self.lr * numpy.dot((output_errors * final_outputs * (1.0 - final_outputs)), numpy.transpose(hidden_outputs))   
   
     #update the weights for the links between the input and hidden layers   
     self.wih += self.lr * numpy.dot((hidden_errors * hidden_outputs * (1.0 - hidden_outputs)), numpy.transpose(inputs))   
       
     pass   
   
   #query the neural network   
   def query(self, inputs_list):   
       
     #convert inputs list to 2d array   
     inputs = numpy.array(inputs_list, ndmin=2).T   
   
     #calculate signals into hidden layer   
     hidden_inputs = numpy.dot(self.wih, inputs)   
   
     #calculate the signals emerging from hidden layer   
     hidden_outputs = self.activation_function(hidden_inputs)   
   
     #calculate signals into final output layer   
     final_inputs = numpy.dot(self.who, hidden_outputs)   
   
     #calculate the signals emerging from final output layer   
     final_outputs = self.activation_function(final_inputs)   
       
     return final_outputs

Run and Output the Result

 nn = neuralNetwork(3, 3, 3, .5)  
 nn.train([1.0, 0.5, -1.5], [1.0, 0.5, -1.5])  
 nn.query([1.0, 0.5, -1.5])  

 array([[ 0.4353349 ],  
     [ 0.64146723],  
     [ 0.3650745 ]])

Source

Tariq's GitHub: Code for the Make Your Own Neural Network Book

Book: Make Your Own Neural Network by Tariq Rashid: Make Your Own Neural Network (GoodReads)

Search This Blog

Data Analytics Workouts