j.a.bullinaria@cs.bham.ac.uk

This page is the main source of information for these modules. From it you will eventually be able to obtain all the slides, handouts, and exercise sheets used in the lectures, details about the continuous assessment and examination, and so on.

Module Outline

This module introduces the basic concepts and techniques of neural computation, and its relation to automated learning in computing machines more generally. It covers the main types of formal neuron and their relation to neurobiology, showing how to construct large neural networks and study their learning and generalization abilities in the context of practical applications. The Level 4/M version also provides practical experience of designing and implementing a neural network for a real world application.

Lecture Timetable and Handouts

Here's an outline of the module structure and lecture timetable. All the module handouts will be made available here as pdf files shortly after the paper versions have been distributed. Spare paper copies will be deposited in the School library.

Week	Session 1 Thursdays 13:00-14:00	Session 2 Fridays 12:00-13:00
1	Introduction to Neural Networks and their History [pdf]	Biological Neurons and Neural Networks, Artificial Neurons [pdf]
2	Networks of Artificial Neurons, Single Layer Perceptrons [pdf]	Learning and Generalization in Single Layer Perceptrons [pdf]
3	Hebbian Learning, Gradient Descent Learning [pdf]	The Generalized Delta Rule, Practical Considerations [pdf]
4	Learning in Multi-Layer Perceptrons - Back-Propagation [pdf]	Learning with Momentum, Conjugate Gradient Learning [pdf]
5	Bias and Variance - Under-Fitting and Over-Fitting [pdf]	Improving Generalization [pdf]
6	Applications of Multi-Layer Perceptrons [pdf]	Exercise Session [html]
7	Recurrent Neural Networks [pdf]	Radial Basis Function Networks: Introduction [pdf]
8	Radial Basis Function Networks: Algorithms [pdf]	Radial Basis Function Networks: Applications [pdf]
9	Self Organizing Maps: Fundamentals [pdf]	Self Organizing Maps: Algorithms and Applications [pdf]
10	Learning Vector Quantization [pdf]	Committee Machines [pdf]
11	Mixture Models and Model Selection [pdf]	Exercise Session

Aims, Learning Outcomes and Assessment

For formal details about the aims, learning outcomes and assessment you should look at the official Module Description (Level 4/M or Level 3/H) and Syllabus (Level 4/M or Level 3/H).

The Level 3 module Neural Computation is assessed by 100% Examination.

The Level 4 module Introduction Neural Computation is assessed by 80% Examination and 20% Continuous Assessment.

In both cases the examination will be closed book, and you will be expected to answer all four questions which will each be worth 25% of the total.

A series of exercise sheets, largely based on recent past examination questions, will give an idea of the standard and type of questions you can expect in this year's examination. These will be distributed when the associated material has been covered in the lectures. They do not contribute to the assessment for the module. The Exercise Sessions will be used to talk through appropriate answers to the questions on the exercise sheets. All six Exercise Sheets have now been distributed: Exercise Sheet 1, Exercise Sheet 2, Exercise Sheet 3, Exercise Sheet 4, Exercise Sheet 5 and Exercise Sheet 6.

I have also produced an Equation Sheet which contains the main equations that you should be familiar with from the module.

Recommended Books and Links

The Recommended Books for this module are:

Title	Author(s)	Publisher, Date	Comments
Neural Networks: A Comprehensive Foundation	Simon Haykin	Prentice Hall, 1999	Very comprehensive, but heavy in mathematics.
Neural Networks for Pattern Recognition	Christopher Bishop	Clarendon Press, Oxford, 1995	This is the book I always use, but it doesn't cover the whole module.
The Essence of Neural Networks	Robert Callan	Prentice Hall Europe, 1999	Concise introductory text.
An Introduction to Neural Networks	Kevin Gurney	Routledge, 1997	Non-mathematical introduction.
Fundamentals of Neural Networks	Laurene Fausett	Prentice Hall, 1994	Good intermediate text.
Introduction to Neural Networks	R. Beale & T. Jackson	IOP Publishing, 1990	Introductory text.
An Introduction to the Theory of Neural Computation	J. Hertz, A. Krogh & R.G. Palmer	Addison Wesley, 1991	Good all round book. Slightly mathematical.
Principles of Neurocomputing for Science and Engineering	F. M. Ham & I. Kostanic	McGraw Hill, 2001	Good advanced book, but rather mathematical.

If you can only afford to buy one book for this module, I would recommend getting the one by Haykin.

If you want to find online information about Neural Networks, probably the best places to start are: The Neural Networks FAQ web-site, and the Neural Network Resources web-site, both of which contain a large range of information and links about all aspects of neural networks.

Potentially useful web-sites for related past modules are still available:

Peter Tino's web-site for this module in 2007

John Bullinaria's web-site for a Level 2 version of this module in 2004

When programming your own MLP neural networks, it may be useful to start with my Step by Step Guide to Implementing a Simple Neural Network in C. It should be fairly straightforward to see how to use it with related programming languages such as C++ and Java.