Aims

• To introduce the basic concepts and terminology of machine learning.
• To give an overview on the main approaches to machine learning (except neural nets, dealt with in a separate course)
• To show similarities and differences between different approaches
• To present basic principles for the classification of approaches to machine learning

Objectives

• know the main approaches of machine learning
• be able to apply the main approaches to unseen examples
• be able to apply their programming knowledge to write algorithms that can learn
• be able to discuss the differences, advantages and problems of the main approaches in machine learning.
• be able to see the relationship of approaches to machine learning and of human learning.
• be able to explain limitations of current approaches to machine learning, and be able to discuss possible extensions to overcome these limitations.

Prerequisites

Basic knowledge of AI Techniques and Logic (as presented in SEM1A8 and SEM1A6). Some programming knowledge in Pop11 (as presented in SEM1A9) is necessary in order to benefit from the implementation examples.

Teaching Methods

The delivery will be a mixture of conventional lectures and exercise classes with two unassessed class tests. (together 2 hours per week throughout the semester).

Students are expected to spend a sixth of their weekly labour time on this 10 credit course (that makes inclusively presence time at least six and a half hours). In particular students are expected to prepare each lecture by reading the advised material or corresponding related material and to work on the exercises. The exercises are crucial for a sufficient understanding of the material presented in the lecture.

Assessment

The course will be assessed by a 2-hour unseen examination in May.

Recommended Books

Title	Author(s)	Publisher	Comments
Artificial Intelligence - A Modern Approach, Chapters 18, 20, and 21	Stuart Russell & Peter Norvig	Prentice Hall 1995	Essential
Artificial Intelligence, 2nd Edition, Chapter 17	Elaine Rich & Kevin Knight	McGraw-Hill 1994	Further Reading
Artificial Intelligence, Chapter 9	Ian Pratt	Macmillan 1994	Further Reading
Artificial Intelligence, Third edition	Patrick Henry Winston	Addison-Wesley 1992	Further Reading
Learning and Memory - An Integrated Approach	John R. Anderson	John Wiley & Sons 1995	Further Reading
Induction - Processes of Inference, Learning, and Discovery, 2nd Edition	John H. Holland, Keith J. Holyoak, Richard E. Nisbett & Paul R. Thagard	MIT Press 1986	Further Reading
Machine Learning	Ryszard S. Michalski, Jaime G. Carbonell & Tom M. Mitchell	Springer 1984	Further Reading
Inductive Logic Programming	Stephen Muggleton (ed.)	Academic Press 1992	Further Reading
An Introduction to Kolmogorov Complexity and Its Applications, 2nd Edition	Ming Li & Paul Vitányi	Springer 1997	Further Reading

The following courses are currently planned, modifications are possible. Slides and Exercises are available in gzipped PostScript files, during the course, spare copies can be found in the School's library, these should be taken first in order not to waste printer resources. The files will be made readable after the corresponding lectures took place. You can directly preview/print the files (locally) by something like:
zcat /bham/ftp/pub/authors/M.Kerber/Teaching/SEM2A4/ex2.ps.gz | ghostview -
zcat /bham/ftp/pub/authors/M.Kerber/Teaching/SEM2A4/ex2.ps.gz | lpr -
zcat /bham/ftp/pub/authors/M.Kerber/Teaching/SEM2A4/l2.ps.gz | ghostview -seascape -
zcat /bham/ftp/pub/authors/M.Kerber/Teaching/SEM2A4/l2.ps.gz | psnup -r -4 | lpr -

Week	Content	Slides	Exercises
1	Basic notions of learning (architecture of a learning agent, performance element, feedback, representation issues), inductive learning, reflexive learning, literature	Slides	Exercises
2	Decision Tree Learning	Slides	Exercises
3	Decision Tree Learning (Finding best attributes), Current-best-learning	Slides	Exercises
4	Least-Commitment Search, Computational Learning Theory	Slides	Exercises
5	Reinforcement Learning (LMS, ADP, TD learning, passive & active learning)	Slides	Exercises
6	Q-learning	Slides	Exercises
7	Knowledge in learning, explanation-based learning	Slides	Exercises
8	Analogy	Slides	Exercises
9	Inductive Logic Programming	Slides	Exercises
10	FOIL, Genetic Algorithms	Slides	Exercises
11	Program Size Complexity, Outlook	Slides	-
12	Revision lecture	-	-

A file with all the slides can be found here

Pop11 Program fragments: Note, most of the program fragments can easily be extended to running programs. This is partly a task in the exercises. Here are some slides on the terminology used in the programs.

Topic	Program Fragment	Full Program
Reflexive Learning	p1.p	p1-solution.p
Decision Tree Learning	p3.p	p3-solution.p
Decision Tree Learning (Restaurant Example)	restaurant-example.p
Decision Tree Learning (Bank Example)	bank-example.p
LMS learning, 3x4-world	p5.p	p5-solution.p
TD learning, 3x4-world	p6.p	p6-solution.p
Q-learning, 3x4-world	p7.p	p7-solution.p