For more information including details of the test see the web site."We (Saeed Dehnadi, Richard Bornat) have discovered a test which divides programming sheep from non-programming goats. This test predicts ability to program with very high accuracy before the subjects have ever seen a program or a programming language."
A member of the CAS group wrote: > For the last three years I have used the 'Mental Models' programming > aptitude test, giving the test in the first or second lessons of a new > AS level course. > > Rather to my surprise I have found it extremely accurate in > identifying students who "won't make it" - students who, despite the > right motivation and masses of personality, can't handle the demands > of programming, even at the novice level. When I first learnt about the test from one of the authors, Richard Bornat, several years ago, it rang bells because I had previously found that there was a subset of students whom I seemed to be unable to teach programming, even though they were highly motivated, worked very hard, and even sat with me in one to one sessions trying to get over their hurdles. Examining their problems closely led me to the hypothesis that a subset of students lack a particular class of short term memory mechanisms required for advanced programming, even if they are very good at reading books and papers in other disciplines, writing essays and getting very good marks and in some cases doing very simple programming tasks. There are some types of programming (and I suspect also certain branches of mathematics, and also musical composition, and poetry composition) that require the ability to construct quite complex interlocking structures in short term memory, save them, create new structures and then perform operations on the old and new structures involving changing relationships. An example might be building two parse trees then noticing the kind of transformation that will convert the first to the second. I have deliberately chosen a fairly extreme case, to illustrate the type of thing. There are probably more complex cases and also simpler cases. I have no idea whether the students who seemed to be unable to do that sort of thing (even with the help of diagrams and prompts from me) had an innate cognitive deficiency or whether then had not been given the opportunity to develop because of some aspect of their environment, or whether they had the potential but required highly specialised teaching or training of a type that I could not offer them. Anyhow, it seems clear that that's something like what the test is probing, by requiring students to invent a syntax and a semantics for some inherently ambiguous symbolic structures then use what they have invented to solve a problem. It's intrinsic to the test that there is no single correct solution. And it is totally different from the task of interpreting some correct code in a conventional programming language. Anyone who looks at it as if it were a simple programming task or program description task will probably completely misunderstand the point of the test, so I suspect that it cannot be used for students who have already learnt some programming -- e.g. because whatever instructions are given, they will misunderstand the test and try to map the structures onto some programming language they already know that it doesn't fit and so fail the test, even if in fact they have the abilities being tested for. Sometimes misleading prior knowledge can interfere with finding a solution to a problem. > As with all such tests, there's a danger that the result becomes self- > fulfilling - leading a teacher to concentrate effort on those who do > well and neglect those who do poorly, but with that health warning, it > is certainly worth a look. If what I've understood about it is correct, then a prior test may be required for detecting students who already have knowledge that will prevent them doing the test well even if they have the ability being tested for. Or in some cases they may be able blindly to apply some prior knowledge that happens to work, but not because they have worked out a solution themselves. In either case the test will give a misleading result -- false negative or false positive. > The test with marking guide is here: > > http://www.eis.mdx.ac.uk/research/PhDArea/saeed/ > > It needs to be given early, before you have formally covered assignment. But even if students have not been taught assignment they may have come across similar ideas that interfere with the functioning of the test. I recommend anyone interested to read the paper before forming any opinions about the test. It's a long time since I read it, so what I've written may be misleading in some details. Long term memory can also be faulty! Aaron http://www.cs.bham.ac.uk/~axs/
School of Computer Science
The University of Birmingham