Recent Forum Topics

beeen a while

since i been oon here :( - summer break = me away from computer.... - how was everyones summer..... - ....any1 have ideas for a csharp project

How to setup php so I can use mail() function?

You can set it up in php.ini. http://www.w3schools.com/PHP/php_ref_mail.asp has information on the configuration options required.

Creating Identical Tables: HTML and CSS Article

http://programming-designs.com/page/editorials/item/6 - It was under the articles/editorials section of the main site, not the most logical place but PD only had one html tut...

LISP

Lisp is cool in a few ways, but the one that strikes most people is that everything is a list. Hence, it is a pretty good list processor. When the first element of a list is not escaped (quoted - described below), th...

Review Script: New Job Board Software

We are happy to announce the release of [url=http://www.smartjobboard.com]SmartJobBoard[/url] job board software. [url=http://www.smartjobboard.com]SmartJobBoard[/url] i...

Neural Networks: Understanding Using Visual Basic

Neural Networks as a subject was the most difficult one to learn when I started taking interest in AI. Although, Internet was full of NN tutorials but they all seemed cryptic and too much mathematical. Anything which was available was hard to digest for a beginner in this field. I swept through tons of code and tutorials just to understand what the hell ,much hyped, NNs were. Then, one fine day, Eureka! I finally understood them and their practical applications. I immediately decided to write code in Visual Basic(my fav lang). After an hour of writing code and few more for tweaking it, I finally produced something which I had only dreamt of. It was one of the the best moments of my life.

The real thing: There isn't a single NN tutorial available for Visual Basic Programmers. That is why I decided to write this tutorial.

Introduction to Neural Networks

Neural Network or more appropriately Artificial Neural Network is basically a mathematical model of what goes in our mind(or brain). The brain of all the advanced living creatures consists of neurons, a basic cell, which when interconnected produces what we call Neural Network. The sole purpose of a Neuron is to receive electrical signals, accumulate them and see further if they are strong enough to pass forward.

So simple in its basic functionality but the interconnections of these produces beings(me, u and others) capable of writing about them. phew! The real thing lies not in neurons but the complex pattern in which they are interconnected. NNs are just like a game of chess, easy to learn but hard to master. As the moves of chess are simple, yet the succession of moves is what makes the game complex and fun to play. Imagine, a chess game in which you are allowed only one single move. Would that game be fun to play?

In the same way, a single neuron is useless. Well, practically useless. It is the complex connection between them and values attached with them(explained later) which makes brains capable of thinking and having a sense of consciousness(much debated).

Basic Workings

As explained earlier, a neuron is basically a cell which accumulates electrical signals with different strengths. What it does more is that it compares the accumulated signal with one predefined value unique to every neuron. This value is called bias. Well, now I think is time to explain with an image. So here is it:



The circles in the image represents neurons. This network or more appropriately this network topology is called feed-forward multi layered neural network. It is the most basic and most widely used network. I will be explaining this network only.

The network is called multi layered because it consists of more than two layers. The neurons are arranged in a number of layers, generally three. They are input, hidden/middle and output layers. The names signify the function of the layer.

This network is feed-forward, means the values are propagated in one direction only. There are many other topologies in which values can be looped or move in both forward and backward direction. But, this network allows the movement of values only from input layer to output layer. The functions of various layers are explained below:

• Input layer: As it says, this layer takes the inputs(the values you pass) and forwards it to hidden layer. You can just imagine input layer as a group of neurons whose sole task is to pass the numeric inputs to the next level. Well it depends on the implementation but, in my implementation the numeric input has to be in range 0 and 1 (both inclusive). The larger the number greater its strength. E.g. 0.51 is stronger than 0.39 but 0.93412 is stronger still. But, the interpretation of this strength depends upon the implementation and the problem NN is required to solve. E.g. for an OCR you connect every pixel with its respective input neuron and darker the pixel, higher the signal/input strength. In short shades of gray would correspond to input strength. But at last again, remember: Input layer never processes data, it just hands over it.
• Middle layer: This layer is the real thing behind the network. Without this layer, network would not be capable of solving complex problems. There can be any number or middle or hidden layers. But, for most of the tasks, one is sufficient. The number of neurons in this layer is crucial. There is no formula for calculating the number, just hit and trial works. This layer takes the input from input layer, does some calculations and forwards to the next layer, in most cases it is the output layer.
• Output layer: This layer consists of neurons which output the result to you. This layer takes the value from the previous layer, does calculations and gives the final result. Basically, this layer is just like hidden layer but instead of passing values to the next layer, the values are treated as output.
• Dendrites: No! it is not some creature from X-Files, but a name given to straight lines joining two neurons of consecutive layers, which you can see in the image. They are just a passage(or method) through which values are passed from one layer to the next. There is a value attached with dendrite called weight. The weight associated with a dendrites basically determines the importance of incoming value. A weight with larger value determines that the value from that particular neuron is of higher significance. To achieve this what we do is multiply the incoming value with weight. So no matter how high the value is, if the weight is low the multiplication yields the final low value. I know it is hard to explain and understand, but take a look at code and you'll understand this concept more clearly.

So now lets get to the code and define the basic elements we'll need in NNs.



The code above is pretty self explanatory. And for the things you haven't understood, don't worry that belongs to belong to the training section, which I would be explaining a little later on. The only thing that you need to understand now is bias. Bias is just another value or parameter associated with a neuron. In my and most of the implementations this value is added to the accumulated incoming value. You will better understand this when we see code for running the NN. For now see the code for creating the net:



Basically, what we are doing above is:

• Creating an array of Neurons and dendrites.
• Looping through all the neurons and assigning their respective baises a random value.
• Looping through all the dendrites and assigning their respective weights a random value.

Now lets see, how do we run a neural network:



What we have done above may be summed up as:

• Assigning all the input neurons the values provided in the 'ArrayOfInputs' array. For every other type of neuron, the value assigned is 0 because we have to calculate that value.
• Now for every hidden layer (also for output layer), the value is sum of values of previous layers multiplied with the weight of dendrites.
• To the resulting value, add up the bias value.
• Pass the resulting value through Activation function.
• Give back the values of output neurons as final results.

I would like to explain that activation function. What this function does is that, no matter what number you pass, it always returns a number between 0 and 1. This is because we want the final value of every neuron between 0 and 1, but what if it comes to be something like 2.5 or 1.25, etc. Thus, in order to avoid this incompatibility with standards we use the activation function. The function is something like:



I will suppose you wouldn't like to get into the mathematical details(hurray!). In case you like, you can easily google it and find out.

Training

Training is the most important part of a neural network and the one consisting of the most mathematics. We'll be using Backpropagation method for training the NN. I would be explaining only the basic idea how it is done and not all the details. And you need not worry, just apply the formulae written in the code and you can get the training up and ready. Always remember, it is not necessary to know the details of any system in order to get something practical and working out of it. The best example illustrating this principle is Charles Darwin(what?). Yes, at the time when he wrote 'On the Origin of Species', DNA was not known. So, he propounded the evolution without even knowing the method of how it is done i.e. how traits are passed on from parents to offspring.
Back to NN training, first lets see the code:

So lets see what we have here in arguments, inputdata a sample input array and outputdata, its corresponding output which we require the NN to produce.

• Run the NN using inputdata as inputs. Now, we have all the values of individual neurons.
• Loop through last layer to first layer.
• Calculate the difference between output values desired and output values produced.
• Using that difference, adjust the values of bias and weights accordingly.

This training procedure must be repeated for larger number of samples so that our NN can produce accurate results for untrained input samples.
Yes, I know I haven't explained the training part in details(well practically zero). You see, I had written this code 1-2 years back and was not actively involved with NNs. During that period I have forgotten all the mathematics behind Backpropagation. But anyway, If you want to delve into the details, Google is to your rescue.

Practical Applications

There are numerous applications of NNs limited only by your imagination. Innovation is key to success, so dude use NNs to create something which will revolutionize the world!
For the sake of writing :) a few applications are given below:

• Optical Character Recognization
• Stock market prediction
• Creating new art forms
• Modeling human behavior
• Loan risk analysis
• Classification of patterns, Image analysis, music generation, etc, etc.

Further Readings

There is more theory in this world about NNs than you can read in your lifetime. There are Hopfield networks, Recurrent networks, NeuroGenetics, etc. They are for people who really want to do serious research in this field. But in case, you want to read further, you can. Internet is a great free information highway. You can get anything you want on the internet if you know how to search effectively. Even if you don't, train a NN to fetch NN tutorials for you :-)
Thanks for reading this basic tutorial, Hope you enjoyed the ride. This tutorial was provided by our partner, ParasChopra.com

Click here to download the accompanying code.