Date:Mon, 8 Dec 1997 10:33:33 +0000 
Original message: here.
Reply-To: "PSYCHE Discussion Forum (Biological/Psychological emphasis)"
Sender:    "PSYCHE Discussion Forum (Biological/Psychological emphasis)"
From:         Aaron Sloman <>
Subject:      More on consistency and consciousness: reasons for view B

To recapitulate: View A is that the contents of consciousness must be internally consistent because the brain uses some mechanism for enforcing this. View B is that sometimes the contents of consciousness are consistent and sometimes they are not: i.e. internal consistency is not an *absolute* requirement for the contents of consciousness, but may often be a consequence of other requirements. I said there were three types of reasons for preferring B: 1. No known type of mechanism could implement the general consistency constraint. 2. Empirically there are counter examples. 3. There are more biologically plausible and biologically useful, mechanisms whose side-effects would include achieving consistency most of the time, without totally ruling out inconsistency. I'll now elaborate on each of these in turn. 1. No known mechanisms could do it. People who have worked on the general problem of inconsistency detection and eradication know that it is inherently intractable, for the reasons indicated in an earlier message: i.e. in the most general case it is undecidable (Go"del's incompleteness theorem, etc), and even decidable cases (e.g. propositional logic) are combinatorially explosive, and therefore intractable in general. Even if the brain does not use propositional or predicate logic it seems (in the case of humans) to have combinatorial capabilities both as regards the variety of 2-D images that can be "parsed" and the variety of 3-D percepts that can be formed. Motion merely adds to the combinatorial complexity. Anyone who doesn't believe that inconsistency detection is intractable in general, should try to design in information processor which can tell whether an arbitrary collection of propositions is consistent or not, or an arbitrary representation of a visual scene, including all variants of Penrose/Escher figures that can be taken in as a whole. Finding a non-exponentially explosive (in space or time) way to check consistency of an arbitrary propositional expression could bring fame and fortune and answer one of the open questions in the theory of computation. Some people may think that parallelism in neural nets can defeat the combinatorics, but although I have not thought it through, I suspect there is probably a theorem about limitations of perceptron-like neural nets on this sort of task, analogous to the Minsky/Papert results regarding parity and connectedness: such essentially global properties require serial mechanisms (or enough perceptron layers to simulate serial processing). Marvin, is that right? (Perhaps I am out of my depth here.) Vision has to work fast, at least if you are a bird, squirrel, chimp, tennis player, car driver, etc. So an engine that sits chugging away looking for ever longer sequences of inferences till all possibilities are exhausted is unlikely to work for the purposes of vision. If there are at most N layers of consistency checking in the visual networks preceding the stage of visual processing which feeds into the contents of conscious experience (whatever that means, but forget that for now), then perhaps a class of inconsistencies detectable in N steps could be eliminated, but not others. We seem to be able to detect 2-step and 3-step visual inconsistencies fairly easily, at least as adults. (What about other animals and young children?) (Maybe quantum-gravity computers can do better than human brains?? Is Penrose listening?) 2. There is empirical evidence that View A is false. I have presented a number of cases of different sorts in previous messages, saved in the file One class of counter-examples discussed previously involves examples of perception of "long range" Penrose/Escher figures where detecting the inconsistency requires more than two or three steps of transitive inference, e.g. a Penrose decagon. It's easy to see such a complex figure as locally consistent everywhere without noticing that if you perform an inference chain involving as many steps as the sides of the polygon that it leads to an inconsistency: X is nearer than Y and X is further than Y. As Stan Klein and Irvin Rock (among others) admit, pictures of impossible objects can be perceived as impossible objects. (I've seen an advertisement in which the typical line drawing is replaced by a superb coloured, textured picture of a Penrose/Escher object, which is easily experienced as a realistic photograph of a globally impossible object.) Even in the case of the simple Penrose triangle I think I have no problem experiencing it as a complete but incoherent 3-D object, likewise the devil's pitchfork etc. That's why those pictures are fun: they are not like the necker figure which (normally) flips between two consistent interpretations. My binocular view of two pencils passing through the same bit of space is inconsistent in a different way. Likewise the motion after-effect described previously, in which motion is observed but nothing changes its location. Bernie previously reacted to some of this by asking whether there is evidence that two (differently shaped?) objects can be seen in exactly the same place. I previously tried to indicate that on one interpretation of the question this is possible (a cube and a sphere of the same diameter embedded in each other), and on another interpretation it is impossible because the question is then incoherent: A cube cannot occupy the same volume of space as a sphere because if it did it would be a sphere, not a cube. Likewise: one reason it is impossible to see the necker cube simultaneously in both of the "standard" ways is not because the brain has a mechanism imposing a consistency constraint but because it is logically impossible and therefore no mechanism is needed to rule it out. That's because seeing both interpretations in the same place at the same time would require seeing a particular edge as both sloping further away to the right and sloping nearer to the right, and that would require the existence of neighbouring bits of the edge, e1 and e2, such that e1 is nearer than e2 and also further than e2. I.e. the very description is incoherent, if the two views are supposed to share edges in 3-D. It's really just the same point as the earlier point that if a cube were in exactly the same space as a sphere it would not be a cube. Likewise if an edge sloping away to the right were seen in exactly the same location as an edge sloping nearer to the right, then it could not be an edge sloping nearer. If they are in the same place they have the same slope. Well then, is it possible to see the necker drawing as depicting not two differently oriented cubes in exactly the same place, but as two wire-frame cubes with corrresponding edges sloping differently but *completely aligned*, i.e. either with edges passing through each other or one nearby and one further away, one cube flipped one way and the other cube flipped the other way, while the edges of BOTH project onto the same lines on a 2-D image plane? On that intepretation the cubes are not sharing all their edges, but the edges of the two cubes are paired so that they cannot be visibly distinguished. That experience would depend on the ability simultaneously to experience a 2-D line, e.g. this: _____________________________, as two coexisting 3-D lines, one sloping away to the right and one sloping away to the left and both projecting to the same 2-D line. Well, I've just tried, and I *think* I did it but frankly I am not even sure how to tell the difference between trying and failing and trying and succeeding. It's like the example I gave in an earlier message of seeing a square circle from the edge. Can we denounce as a liar an individual who sincerely claims to see a line that way? Maybe, like Einstein or Mozart she can do things we can't? So: empirical evidence shows that incoherent visual experiences are possible, as indicated above and in my previous messages (and as stated by Stan Klein and by Irvin Rock). Sometimes they are detected as incoherent, e.g. the penrose triangle, sometimes not, e.g. penrose dodecagon, or a long devil's pitchfork, or cluttered Escher drawing, etc. I am not sure that a very young child would detect the inconsistency in the simple cases. (I once asked a five year old child where two cars would meet if they start off in this configuration, and drive towards each other, where A is an ordinary slow car and B is a very fast racing car: A --> ^ <=== B C He pointed at a location nearer to B, e.g. C. I asked why, and got the answer: "It goes faster so it will get there sooner." He was totally unaware of the inconsistency in what he was saying (and believing, presumably. Some of Piaget's observations are also relevant.) If View A is expressed in a particularly strong form requiring not just visual percepts to be internally consistent but the total contents of consciousness then there are of course even more counter examples, e.g. perceptual experiences which are known to be wrong. Example of the flipping folded card: Get a card about 20cm by about 10cm. Fold it down the middle to form a V with an angle of about 30 degrees (or more, or less: sizes and angles are not critical). Then either put it on the table with the V edge on the table, forming two walls meeting at a corner or balance it with the ends of the V on the table and the folded edge horizontal above the table, like a ridge tent. Stare at it from a few feet away with one eye for a while, and you should be able to make it appear to flip from one of the two configurations to the other. When it has flipped, move your head from side to side a little and watch it twist and slide on the table. I KNOW the card is rigid and stationary and I simultaneously SEE it twist and move. Anyone who claims that I really must be alternating between the two states of consciousness must be too much in the grip of a theory to face facts. Or maybe we have different sorts of brains. Summary of second argument against the consistency hypthesis: there are empirical counter examples if you look for them. A qualification is in order, unfortunately: Some of the issues which appear to be empirical may not be because the questions asked can be shown to be incoherent. 3. Argument 3: there's a better explanation of consistency The third argument was hinted at in previous messages by Minsky and one of my earlier postings and possibly others. A version of it can perhaps be read into George Mckee's message of 25th Nov about high-order processes though I am not sure. The argument is that some of the cases where consistency is found may actually be cases of something else: lets call it "invocation of perceptually learnt models or schemata". The basic idea is very old: it pervades the writings of Richard Gregory, is developed in some detail in Minsky's paper on Frames (circa 1973?) goes back much further to Kant and Bartlett, is implemented in a number of AI vision systems and will probably often be re-invented, though possibly with newer more powerful implementation mechanisms. Visual mechanisms in part have the role of identifying perceived objects or processes either as belonging to some class (e.g. flower, chair, furniture, spinning, sliding, fighting, breaking, supporting, etc.) or being a known individual (Fido, Freda or Freddy, the moon, or the Eiffel Tower, etc.) For now lets ignore the difference between classification and (re)identification, and just use the neutral notion of "applying a schema" (rather than "recognizing a pattern", to allow for the variability in the data and the flexibility in the mechanisms, as noted by Kant and others centuries ago). Some of the categories used may be innate (e.g. edge, surface, edge-feature, curvature, orientation, closure, linear radial and circular optical flow, etc. in humans, and probably other things in other animals, e.g. the tongue-aiming bug-detectors in frogs) others learnt (e.g. learning to read words or music, learning to identify plants or birds, learning to see a pair of identical twins as looking very different (which took me a couple of months) and learning to recognize most of the "affordances" in our environment, in Gibson's sense). For now it doesn't matter which are innate or which learnt: both are derived from experience of conditions for success and failure of actions, or merely by induction from salient frequently encountered examples, whether the derivation is done by the individal or by the gene pool working through many individuals in an evolutionary process. The only important point is that as long as the process of derivation from ACTUAL examples is fairly direct, the derived schemata will be internally consistent. Where the derived schemata involve combinatorial generalisation (e.g. induced grammars, induced composition rules for geometrical structures or forms of motion) then it's possible to create a schema or schema instance which is not derived from any actual object and which turns out to be internally inconsistent. (Like the village barber who shaves all and only those who don't shave themselves.) The rest of the story should be fairly obvious. We need a principle which can be roughly formulated thus, ignoring for the moment what percepts or thoughts might be. The visual system tries to find a way of accounting for as much of the available information as possible by creating percepts instantiating the smallest possible number of high level schemata (in the sense of Kant and Barlett). The cognitive system attempting to understand some complex collection of information attempts to create a thought using as few schemata as possible which together subsume all the information. or in other words: If we can recognize some intelligible high level structure in a perceived scene or thought-about situation we will normally do so. where intelligible means something like "accommodated in known schemata". (The schemata need not be previously known: maybe some are created on the fly. A lot of art is like that.) (The biological advantages to an animal of being able to do this, and the engineering advantages to a robot, should be obvious, but we could discuss them.) Corollary: wherever schemata directly abstracted from previously encountered concrete cases are deployed the thought or percept will be internally coherent. However, sometimes the data driving the process will invoke novel combinations of schemata and then there's no guarantee that the result will be coherent. Penrose/Escher figures and the devil's pitchfork are examples. Likewise puns? CONJECTURE: some of the innately determined perceptual schemata will be implemented at a very low level in brain mechanisms using mutual excitation and inhibition in "winner takes all" networks, so as to ensure that wherever possible local ambiguities can be resolved by combinations of top down and bottom up and sideways processing in something like a "relaxation" process. (An old idea. E.g. I learnt it over 20 years ago from Geoff Hinton, but perhaps others had thought of it earlier.) Sometimes these networks will be driven by data supporting a small number of stable states equally well (e.g. necker figure) and then various mechanisms can make the network flip from one stable state to another. In some cases extra bottom-up clues (shading, texture, etc.) or top-down thought processes ("this is coming towards me") can drive sub-schemata to form smaller stable, but no longer mutually consistent alliances. That would account for some of the strange incoherent views of the necker cube. Maybe similar winner-takes-all multi-stable networks are involved in the management of the contents of thought processes, though here we don't get a rich dense collection of inputs directly clamped by the environment, and there seems to be more scope for structural variation. (Consider the differences and similarities between proving a deep mathematical theorem in your head, reminiscing about your holiday, planning a meal for your friends, composing a sonnet or limerick, wondering whether someone reciprocates your feelings, thinking about consciousness, etc.) Because of the combinatorial richness of linguistic processes, and their disconnection from current sensory inputs, linguistically mediated thoughts will be even less constrained to be consistent than perceptual contents, although when you are thinking about actual situations they will normally be consistent because anything actual must be. (I am speaking more loosely than some philosophers and logicians would like). This is why it is much easier to produce ambiguous or inconsistent sentences, puns, linguistic jokes of various kinds, and nonsense sentences, than pictures with those qualities. But of course expert cartoonists have been doing it for a long time: e.g. drawing a face which is clearly recognizable as a certain politician and as a certain animal. To summarise point 3: Although perceptual contents are normally internally consistent and thought contents often are, this consistency is not something the brain can enforce. Rather it's a SIDE-EFFECT of powerful biologically useful processing mechanisms which by and large have developed to fit the actual world, not a world full of things that are impossible in our world. That's just as well, since if the brain needed to detect and enforce general semantic consistency in any particular sub-structure it would require mechanisms which, as far as we know, cannot exist. By allowing (as Stan Klein and Irvin Rock do) that the search for a high level consistent integrating schema subsuming the current visual information can fail, we allow the possibility of a globally inconsistent percept involving smaller scale locally consistent percepts. And that seems to fit empirical data about inconsistent percepts very nicely, while also explaining the cases where a coherent whole does occur as a side effect of a *soft* requirement of a biologically effective system, not a hard constraint imposed by some impossible mechanism. If all this is just a long winded way of saying the following (quoted in a previous message): No matter what we do, the visual system tries to find a single coherent conscious interpretation at any given moment. (Page 87 of "In the Theater of consciousness") I.e. *tries* but doesn't necessarily *succeed* then Bernie Baars is saying the same as I am, and I apologise for misreading it previously. (Especially as I find a lot in common with other aspects of his theories.) CODA: At the Elsinore conference last August, Doug Watt repeatedly claimed that any good theory of consciousness must accommodate what he referred to as "disorders of consciousness". I agree: disorders and variations both across and within species and individuals need to be accounted for. From the little I know about autism, it seems that one feature of some autistic people involves visual processing remaining fragmented, without the global coherence that comes from applying a high level (abstract) concept. This could be part of the explanation of the ability of some of them to produce surprisingly accurate paintings and drawings: because their processing of visual input stops at the kind of local 2-D structure which an artist needs to replicate, instead of moving on to the integrating 3-D structure which is then much harder to project back onto the page. (Compare: Lorna Selfe, Nadia: a case of extraordinary drawing ability in an autistic child, London, Academic Press, 1977.) Aaron

Back to: Top of message | Previous page | Main PSYCHE-B page

LISTSERV.UH.EDU CataList email list search Powered by LISTSERV email list manager