This module introduces the basic concepts and techniques of neural computation, and its relation to automated learning in computing machines more generally. It covers the main types of formal neuron and their relation to neurobiology, showing how to construct large neural networks and study their learning and generalization abilities in the context of practical applications. The Level 4/M version also provides practical experience of designing and implementing a neural network for a real world application.
Here's an outline of the module structure and lecture timetable. All the module handouts will be made available here as pdf files shortly before the paper versions are distributed in the lectures. Any spare paper copies will be deposited in the School library.
I will leave last year's notes in place, marked "[2014pdf]", until they are replaced by this year's notes, marked "[pdf]".
Lecture 1 |
Lecture 2 |
|1||Introduction to Neural Networks and their History [pdf]||Biological Neurons and Neural Networks, Artificial Neurons [pdf]|
|2||Networks of Artificial Neurons, Single Layer Perceptrons [pdf]||Learning and Generalization in Single Layer Perceptrons [pdf]|
|3||Hebbian Learning, Gradient Descent Learning [2014pdf]||The Generalized Delta Rule, Practical Considerations [2014pdf]|
|4||Learning in Multi-Layer Perceptrons - Back-Propagation [2014pdf]||Learning with Momentum, Conjugate Gradient Learning [2014pdf]|
|5||Bias and Variance - Under-Fitting and Over-Fitting [2014pdf]||Improving Generalization [2014pdf]|
|6||Applications of Multi-Layer Perceptrons [2014pdf]||Exercise Session|
|7||Recurrent Neural Networks [2014pdf]||Radial Basis Function Networks: Introduction [2014pdf]|
|8||Radial Basis Function Networks: Algorithms [2014pdf]||Radial Basis Function Networks: Applications [2014pdf]|
|9||Self Organizing Maps: Fundamentals [2014pdf]||Self Organizing Maps: Properties and Applications [2014pdf]|
|10||Learning Vector Quantization [2014pdf]||Committee Machines [2014pdf]|
|11||Model Selection and Evolutionary Optimization [2014pdf]||Exercise Session|
|12||Revision Lecture Covering the Whole Module [2014pdf]|
For formal details about the aims, learning outcomes and assessment you should look at the official Module Description (Level 4/M or Level 3/H) and Syllabus (Level 4/M or Level 3/H).
The Level 3 module Neural Computation is assessed by 100% Examination.
The Level 4 module Introduction Neural Computation is assessed by 80% Examination and 20% Continuous Assessment.
In both cases the examination will be closed book, and you will be expected to answer all four questions which will each be worth 25% of the total.
The Continuous Assessment for the Level 4 module Introduction Neural Computation will be distributed and discussed in the Thursday lecture of week 6, and the deadline will be 12noon on the Wednesday of week 1 in the Spring Term. Full details will also be published here and on Canvas in week 6.
A series of Exercise Sheets, largely based on recent examination questions, will give an idea of the standard and type of questions you can expect in this year's examination. These will be distributed when the associated material has been covered in the lectures. They do not contribute to the assessment for the module. They are designed to help you monitor your progress on the module - try to answer the questions without your notes, and then use your notes to see whether your answers are correct. The Exercise Sessions will be used to talk through answers to any questions you have difficulty with. So far, none have been distributed.
The Recommended Books for this module are:
|Neural Networks and Learning Machines||Simon Haykin||Pearson, 2009||Very comprehensive, but heavy in mathematics.|
|Neural Networks: A Comprehensive Foundation||Simon Haykin||Prentice Hall, 1999||Older edition of the above book, but still covers the whole module.|
|Neural Networks for Pattern Recognition||Christopher Bishop||Clarendon Press, Oxford, 1995||This is the book I always use, but it doesn't cover the whole module.|
|The Essence of Neural Networks||Robert Callan||Prentice Hall Europe, 1999||Concise introductory text.|
|An Introduction to Neural Networks||Kevin Gurney||Routledge, 1997||Non-mathematical introduction.|
|Fundamentals of Neural Networks||Laurene Fausett||Prentice Hall, 1994||Good intermediate text.|
|Introduction to Neural Networks||R. Beale & T. Jackson||IOP Publishing, 1990||Introductory text.|
|An Introduction to the Theory of Neural Computation||J. Hertz, A. Krogh & R.G. Palmer||Addison Wesley, 1991||Good all round book. Slightly mathematical.|
|Principles of Neurocomputing for Science and Engineering||F. M. Ham & I. Kostanic||McGraw Hill, 2001||Good advanced book, but rather mathematical.|
If you can only afford to buy one book for this module, I would recommend getting either of the Haykin books.
If you want to find online information about Neural Networks, probably the best places to start are: The Neural Networks FAQ web-site, and the Neural Network Resources web-site, both of which are rather old now, but still contain a large range of information and links about all aspects of neural networks. This module was previously taught by Peter Tino, and you may find his 2007 module web-site useful.
When programming your own MLP neural networks, it may be useful to start with my
Step by Step Guide to Implementing a Simple Neural Network in C.
It should be fairly straightforward to see how to use it with related programming languages such as C++ and Java.