http://www.cs.bham.ac.uk/research/projects/cogaff/misc/binocular-rivalry.txt
('.txt' suffix added 6 Feb 2019)

Aaron Sloman
22 Dec 1997
Some contributions to the psyche-B email list, discussing binocular
rivalry, and the claim by Baars that contents of consciousness must be
consistent or coherent.

Note added 6 Feb 2019
Until recently the archives of the old psyche-B were still available on the
internet (I forget where). I can no longer find them. But this is one of the
threads that I saved in 1997, as I thought it worth preserving.
If anyone knows of a more complete archive please send me information, which
I'll add here.
Aaron Sloman
http://www.cs.bham.ac.uk/~axs
a.sloman AT cs.bham.ac.uk

Some addional saved archives are listed at the end.

CONTENTS

 From Aaron Sloman Wed Nov 19 22:28:37 GMT 1997
 Subject: Re: Illusions
 From Aaron Sloman Fri Nov 21 22:01:36 GMT 1997
 Subject: Baars on Binocular Rivalry (was Re: Illusions)
 From Aaron Sloman Mon Nov 24 01:19:46 GMT 1997
 Subject: Re: Baars on Binocular Rivalry (was Re: Illusions)
 From Aaron Sloman Thu Nov 27 13:58:42 GMT 1997
 Subject: Re: Baars on Binocular Rivalry (was Re: Illusions)
 From Aaron Sloman Mon Dec  1 01:33:20 GMT 1997
 Subject: inconsistent Necker?
 From Aaron Sloman Sun Dec  7 02:00:13 GMT 1997
 Subject: More on consistency: two rival viewpoints
 From Aaron Sloman Sun Dec  7 18:45:49 GMT 1997
 Subject: More on consistency and consciousness: reasons for view B
 From Aaron Sloman Mon Dec  8 10:33:33 GMT 1997
 Subject: More on consistency and consciousness: reasons for view B
 From Aaron Sloman Mon Dec 15 00:54:59 GMT 1997
 Subject: Re: More on consistency and consciousness: reasons for view B
    Response to Cariani
 From Aaron Sloman Thu Dec 18 12:03:41 GMT 1997
 Subject: Motion after effect (was: Baars on Binocular Rivalry)
    Response to Trehub

=======================================================================

From Aaron Sloman Wed Nov 19 22:28:37 GMT 1997
To: PSYCHE-B@LISTSERV.UH.EDU
Subject: Re: Illusions

Date: Tue, 18 Nov 1997 09:11:10 -0800
From: Stanley Klein <klein@ADAGE.BERKELEY.EDU>

> The disappearing face illusion is one of many examples of binocular rivalry.
> One easy demonstration of it is to roll a sheet of paper into a tube, hold it
> with your left hand and look through it with your left eye. Simultaneously
> hold your right hand in front of your right eye. You will see a dramatic hole
> in your hand.

Depends on what you are looking at. From my simple experiments if the
tube is aimed at something very bright (e.g. a sunny view outside) then
the "hole in the hand" effect is very strong. If the tube is aimed at a
very much duller background there may be no hole: the hand looks normal.

In between various kinds of fusion or superimposition can occur: the
hand is visible but partly transparent: e.g. as you move it up and down
you can see BOTH the moving texture on the skin and the stationary
background. A kind of fusion. But different from the integrative fusion
(coherence) of normal stereo vision which creates a new unified
information structure containing 3-D information. (How many theories of
vision can explain how we see both motion and non-motion simultaneously
in the same part of the visual field? Compare looking through a dirty
window at trees swaying in the breeze, or through a moving transparent
but dirty or textured sheet of glass at static objects. You can also
have both moving in different directions at once.)

> Binocular rivalry is especially effective if one image is
> blurred (like the close hand in the tube example) or is moving (like the face
> disappearing because of the dominant moving hand in the earlier example).

In the hand and tube case the motion can sometimes UNDO the binocular
rivalry and cause superimposition.

> The neural mechanisms of binocular rivalry are especially interesting for
> the study of the neural correlates of consciousness (NCC) because binocular
> rivalry also occurs when the two eyes view two different static, in-focus
> scenes. In that case the eye correlated with the perception switches back
> and forth.

Not always. A common counter example is seeing two fingers and the
background at the same time when you hold up a single finger some
distance from your eyes and look at the background (especially if you
move the finger against a bright background??)

Quite a lot of people seem to think that the ONLY possibilities are
complete coherent fusion or binocular rivalry. E.g. Baars claims this
unequivocally on page 89 of "In the Theater of Consciousness" (OUP 1996)

This reminds me of the frequently made claim in text books that if you
look at the famous face/vases ambiguous figure (Rubins?) you can see it
either as two faces with the middle bit as background or as one vase
with background on either side, but never as faces and vase
simultaneously, as if this were a fact to be explained by brain science.

But it's NOT a fact: many people can easily come to see both face and
vase at once simply by interpreting the picture as representing a vase
of the right shape wedged between two faces. It takes nothing more than
the thought of the possibility to make it happen. (Except for people who
have learnt that it is impossible...)

Moral: beware of alleged facts about how the brain works -- for many can
be refuted by considering different cultures,  a new experimental
situation, or simply doing a bit of creative thinking. E.g. some alleged
facts about memory limits are refuted by learning simple tricks for
remembering things.

(Compare the alleged fact that mirrors reflect things with left and
right sides swapped but never top and bottom swapped: often quoted by
eminent people, who then propose elaborate explanatory theories. But
it's not a fact. Put a mirror on the floor and look down into it. )

(The tendency of scientists to make claims that are not false, but
lacking in any clear meaning, is another thing to be avoided: like the
claim that a neural event precedes consciousness of something by a few
hundred milliseconds. Compare: it's noon on the moon exactly 3 hours
after it's noon in New York. Or Clinton's popularity first exceeded that
of Bush exactly 0.005 microseconds after a certain sub-atomic event
occurred in a cloud chamber in this lab.)


> Many neurons in visual cortex don't switch and many do. The ones
> that do switch in synch with perception are candidates for belonging to the
> NCC. The vast number of neurons that don't switch are not part of the visual
> NCC.

Er... does that mean that when I see inconsistent left and right views
superimposed something has gone wrong with my brain?

Maybe I am just a mutant.

Cheers

Aaron
=====
Aaron Sloman, ( http://www.cs.bham.ac.uk/~axs )
School of Computer Science, The University of Birmingham, B15 2TT, UK
EMAIL   A.Sloman@cs.bham.ac.uk
Phone: +44-121-414-4775 (Sec 3711)       Fax:   +44-121-414-4281

From Aaron Sloman Fri Nov 21 22:01:36 GMT 1997
To: psyche-b@listserv.uh.edu
Subject: Baars on Binocular Rivalry (was Re: Illusions)

"<Bernard Baars>" <BBaars8788@AOL.COM>

Dear Bernie,
Thanks for your rapid response to my comment. Apologies taking so long
to reply, owing to umpteen other parallel tasks.

> Aaron Sloman nails me on something I did not  say. He writes, " Quite a lot
> of people seem to think that the ONLY possibilities are complete coherent
> fusion or binocular rivalry. E.g. Baars claims this unequivocally on page 89
> of "In the Theater of Consciousness" (OUP 1996)"

Well, perhaps I mis-interpreted  what you wrote.

    ...But binocular fusion breaks down if there are significant
    differences between the input to the two eyes. .... if one image is
    slightly offset from the other, binocular rivalry results, and to
    obtain coherence the visual system will suppress one image in favour
    of another.

I thought that was pretty unequivocal. And to me it implied that
if you are getting two different views one will be suppressed, which is
not what happens if you look past a finger pointing upwards and see two
fingers both of which are transparent, which is what happens to the
people I've asked. (It's also a common textbook example, I think.)

Later on the same page (Baars p.89):

    ... Any significant disparity between the two eyes or ears causes
    one of the two flows to be suppressed. Conscious perception is
    always coherent, even if the nervous system needs to cancel (sic!!)
    some input in favor of another.

    ... In general, it seems to be impossible for human beings to hold
    two different interpretations of the same thing in consciousness at
    the same time. In many cases we can prove that two representations
    exist fleetingly in the brain but only one can be conscious at a
    time.

And so it goes on. Now to me that reads exactly like the claim I
criticised, and which I've also heard from others in the field, and
which I think is clearly false for the reasons I gave.

But maybe you intended it to say something I've not understood.

In your response to me you write:

> My claim is that consciousness reflects an internal consistency constraint.
> The observations Aaron cites are not internally inconsistent, and do not
> contradict that hypothesis.

Consider the sorts of superimposed images I described, e.g. resulting in
your simultaneously seeing
BOTH
    the moving texture of your skin as you move your finger up and down
AND
    the static texture of the wall beyond the skin
as a result of two DIFFERENT retinal images both contributing
simultaneously to your current experience.

If you claim that that is not a case of internal inconsistency then I
have no idea what you mean by "internal inconsistency". It seems to me
to be exactly a counter example to what you describe by saying

    ...if one image is slightly offset from the other, binocular rivalry
    results...

Perhaps you've changed your views slightly since writing those words, as
suggested by your comment on my message:

> The visual system seems to interpret those
> binocular effects as looking through a diaphanous object in one eye and
> seeing a solid one in the other.

Not in my case: it actually looks as if I am seeing the SAME
(inconsistent) thing in BOTH eyes! Doesn't it to you?

> ...It looks actually like a really clever and
> creative solution to the problem of seeing two different objects, one in each
> eye, and giving a consistent interpretation.

My suspicion is that with your current view as expressed here, nothing
is ruled out: the visual system can in some contexts find a way to
accommodate ANY two retinal images.

The new theory seems to me to be far more interesting, as well as being
true (with qualifications to allow for the cases where rivalry really
does occur).

It's more interesting because it helps to draw attention to the real
creativity of interpretative processes involved in perception,
undermining the common view (among scientists) of perception as a (much
more boring) process of directly extracting information about the
environment through a succession of stages of image analysis.

Whether it fits in with the global workspace and searchlight and theatre
metaphors is another question. In my experience, searchlights and
theatres have the wrong properties to be capable of explaining the
phenomena. "Global workspace" seems to be closer to an idea which is
frequently reinvented (as you acknowledge) but as yet far too murky.

We shall almost certainly have to find an entirely new non-metaphorical
way to think about these things, involving the concept of an information
state in which various kinds of information about spatial and temporal
and functional and causal relationships are integrated in some kind of
mechanism, not yet understood at all, which makes that information
readily available for a wide variety of tasks. How?

What these examples of superposition show is that various sorts of
simple-minded ideas about constraints on the kinds of information that
can be simultaneously held in this mechanism (e.g. simple notions of
"coherence", "consistency", etc.) are wrong.

What the facts about rivalry show is that sometimes such constraints do
operate.

Nothing I've heard about in neuroscience, AI, computer aided design,
seems to be adequate to the task of implementing these information
states. But that's another story. (I've spelled out, rather crudely,
some of the requirements in a paper in the Journal of Experimental and
Theoretical AI ,1989, vol 1, no 4. But it's very crude. I'm still
trying to expand the ideas in terms of what is to perceive
possibilities inherent in a structure.)

> I don't know of a persuasive counterexample to the consistency hypothesis.
> It's just an empirical question --- does anybody know of one?

Maybe it's not an empirical question if the phenomena described above
are not counterexamples.

Aaron


From Aaron Sloman Mon Nov 24 01:19:46 GMT 1997
To: PSYCHE-B@LISTSERV.UH.EDU
Subject: Re: Baars on Binocular Rivalry (was Re: Illusions)

Thanks Bernie,

I don't know how you find all the time, and I am not sure I can keep up,
but here goes. (I still owe Stan Klein a reply, explaining the
difference between being meaningless and having no clear meaning, to
come later...)

I'll separate out several different claims about the requirement for
coherence in perception, and argue against them, while supporting a
loosely analogous argument about the strong likelihood of resource
limits in higher level parts of the architecture for an intelligent
agent. The problem of resource limits could be confused with a
requirement for a consistency constraint.

> Date: Sat, 22 Nov 1997 00:25:53 -0500
> From: "<Bernard Baars>" <BBaars8788@AOL.COM>
>
> Aaron thinks we can have internally inconsistent perceptual experiences. Here
> is the test.
>
> CAN YOU SHOW THAT A PERSON CAN PERCEIVE TWO SOLID VISUAL OBJECTS IN THE SAME
> PERCEIVED SPATIAL LOCATION AT EXACTLY THE SAME PERCEIVED TIME?

OK, we now have three distinct theses of diminishing strength:

1. When retinal inputs differ one or other will be suppressed unless
    they fuse into a single percept as in normal stereo vision.

That strong thesis seems to believed by many people and that's what I
read into page 89 of B.Baars "In the Theater of Consciousness" (OUP
1996). It now seems that we are agreed it is false, so you didn't mean
to say what your words seemed (to me) to be saying.

Then came this less strong thesis:

2. Consciousness reflects an internal consistency constraint.

I assumed this weaker thesis was an attempt to rule out combinations in
which different, inconsistent, things are perceived in the same portion
of the visual field.

There are various counter-examples discussed in my previous messages,
and many more: e.g. a motion after-effect in which perception of upward
motion occurring at a particular location coincides with perception of
only static objects there.

But you didn't want to accept the things I listed as counter-examples,
so I suggested that perhaps nothing could be a counter example and the
claim is therefore non-empirical. You've now responded with a third,
still weaker thesis. (NB: "weak" does not mean "bad". It's just a
neutral description of the number and diversity of consequences of the
thesis.)

3. A person cannot "PERCEIVE TWO SOLID VISUAL OBJECTS IN THE SAME
    PERCEIVED SPATIAL LOCATION AT EXACTLY THE SAME PERCEIVED TIME"

This weakest thesis is still interesting, but I am not sure whether it's
true or false or perhaps just a non-empirical piece of mathematics or
logic.

My initial reaction is that it is false, like the simplistic
interpretations of the first two theses, since, for instance, Picasso in
one of his sculptures showed us how to see a sports car and a baboon's
head in the same place at the same time. Michaelangelo encouraged a
percept of a human body struggling to emerge from a block of marble.
I don't think we are invited to see the marble as *enclosing* a separate
body. That would be a far less powerful work of art.

Or consider the well known fruit-face: assemble a pumpkin, a banana, a
couple of cherries and a carrot in the appropriate configuration, with
the help of some adhesive, and then you can see in the same place at the
same time a smiling face with two eyes a nose and a mouth, and a
collection of different items of fruit: two solid visual objects in the
same perceived spatial location at exactly the same perceived time.

Perhaps you'll say those are not counter examples to thesis 3, since in
each case we see the same 3-D bounding surface but can apply different
inconsistent classifications to it simultaneously.

Can you actually describe in any detail a type of perception which you
would accept as a counter example if it were to be produced in some
carefully contrived experimental situation?

If not, perhaps that's because what you are ruling out is actually
something incoherent. I.e. it's not a fact about the brain that it can't
do it, just as it's not a fact about the brain that it can't make the
number 27 a prime number.

For example, suppose that by "solid" you really meant "opaque" (i.e
impervious to vision). and by "two objects" you really meant
"differently shaped objects". E.g. you are asking (for example) can a
person see an opaque cube and an opaque sphere in the same perceived
spatial location at exactly the same perceived time".

Unfortunately, the problem now is not an empirical question about what
the brain can or cannot do but a conceptual question about what it could
mean for two differently shaped objects to occupy exactly the same
location. If their shapes are different they certainly cannot: if a
solid object occupies all and only the same spatial locations as a cube
then it must be a cube, not a sphere. But that's a trivial (well, nearly
trivial) matter of definition, not a fact about what brains can and
cannot do. Or, to be more precise: it's a fact of mathematics (geometry)
not a fact of biology (brain science).

But perhaps you meant something like this: can a person see two opaque
solid objects in roughly the same place, i.e. with overlapping volumes,
e.g. Michaelangelo's human bodies struggling to emerge from blocks of
marble, with which they somehow co-occur.

You seem to say No, whereas I expect the answer is yes: provided the
person, the context, and the stimuli are right. But in any case, it is
very hard to reach a negative conclusion from any number of experiments.

Imagine trying to prove by experiment that no human brain can find a
proof of Fermat's last theorem. Millions of negative experimental
results would still not give an answer to the question, which we now
know is positive. Or imagine experiments prior to Einstein&Minkowski
seeking to find out whether a person can imagine curved 3-D space, or
instantaneous events which are not unambiguously ordered in time, or...

Human brains are the most creative machines known to science, and any
generalisation about their limitations needs to be treated with
*extreme* caution. (Even that one...)

But I now have to ask, does this consistency/coherence constraint
really add anything significant to theories of perception or
consciousness?

Why is it felt to be important? Is it just a relic of a rationalist view
of mind, taken by granted by many philosophers (even Dennett, in weak
moments)?

Maybe one of the most important things about (human) brains is that they
can transcend what's consistent, coherent, rational, etc.

Why?

Because at some stage evolution produced a powerful mechanism (I don't
necessarily mean a computational mechanism) for assembling new
information structures out of old ones, and then found all sorts of deep
ways to make use of it, or variants of it. E.g. it's crucial to planning
and much problem solving, and to communication in natural language. It
makes mathematical learning and problem-solving possible.

But, for deep mathematical reasons, it is hard or impossible to combine
such combinatorial creativity with a guarantee of consistency, and sure
enough humans are frequently seduced by incoherent thoughts, objectives,
ethical systems, religious propaganda, etc. So why not incoherent
percepts too, including visual percepts?

Some people used to think the reason was that visual percepts employed
spatial forms of representation, and spatial representations cannot
represent what is spatially impossible. We now know that is false:
examples include the Penrose triangle, Richard Gregory's 3-D
implementation of it, Escher figures, and one of my own favourites:

    _______________________________


i.e.  a round square seen edge on.

NB: the Penrose/Escher figures (e.g. impossible triangle) are not
illusions or simply ambiguous figures. Rather they are locally
consistent projections of globally inconsistent 3-D structures. When the
picture is sufficiently complex, e.g. not a triangle but a decagon, many
people will see only the local consistency, and not detect the global
inconsistency: i.e. they will see a geometrically impossible/incoherent
object. (Does that refute the consistency/coherence theory?)

So why struggle to defend such an embattled theory, as the theory that
the contents of consciousness must be coherent? I think it's a confusion
of a processing issue with a content issue, to be explained later.

Of course, the creativity of each brain is limited, but the limits
change over time. History is littered with the corpses of ideas about
what is incoherent, inconceivable, impossible: the earth moving through
space, space-filling one-dimensional curves, everywhere continuous but
nowhere differentiable functions, sets which are in one to one
correspondence with proper subsets of themselves, the square root of a
negative number, adding light to light in order to produce darkness,
humans evolving from micro-organisms, distant events being both before
and after each other, force-fields in totally empty space, curvature of
3-D space, transmission of music or pictures across empty space,
something behaving as both wave and particle, ... to mention a few of
the better known cases.

Instead of stressing the requirement for contents of consciousness to be
coherent, consistent, etc. we should perhaps be trying to understand
what sorts of mechanisms can fruitfully push their own boundaries beyond
what till then are the limits of consistency. That is what many great
scientists, mathematicians and artists have done.

Maybe every child does it as part of its normal process of development,
until we use bad schools and bad scientific text books and religious
dogma to stifle the process.

Anyhow, I am perfectly aware that this is hand-waving. The real work
remains to be done. Let's just not shut out theories because so far no
established evidence seems to support them, if there's a strong
theoretical reason for accepting them (in this case the theory that
perception, like thought, involves combinatorial creativity, at least
in some animals. Maybe not house flies?).

> My reading of
> all the research going back to Helmholtz is that the answer is no.

Until recently all the evidence was that nobody could prove Fermat's
last theorem. There were probably even more failed attempts at that task
than psychological experiments on perception of inconsistent scenes.

> All the
> claimed exceptions to the rule involve a clever visual-brain solution to the
> problem,

We seem do be agreed on the essential creativity (cleverness) of
perceptual mechanisms?

> as in the case where we perceive a hand over one eye as a
> diaphanous, filmy object, THROUGH WHICH  we see another object. The visual
> brain is just brilliant in solving such conflicting inputs.

Given all that, what are the mechanisms underlying such brilliance? My
conjecture is that they depend on combinatorial creativity. And given
that, why should such creativity be constrained to fit any particular
kind of coherence criterion?

And how could it be? (It's not always easy to build semantic constraints
into the syntax of a powerful form of representation.)

> Most of the time,
> in normal vision, the dominant eye's interpretation "wins," though the
> nondominant eye provides a sense of depth.

(I must be a mutant: my eyes are symmetric on this test.)

I think that if you abandon this (arbitrary?) consistency constraint it
will enrich your theory of consciousness, make it closer to the facts,
and I suspect make it easier to produce explanatory mechanisms.

> I think Aaron is entirely right that local or peripheral hypotheses about
>  such conflicting input situations don't get us very far. It's a
> multi-layered, if-one-thing-doesn't-work- try-another system, not at all
> "boring."

I apologise for the rhetorical flourish. Boringness is less important
than truth. Or, as J.L.Austin said, truth is more important than
importance.

> So the theoretical claim is that the consistency constraint applies not just
> to vision, not just to audition and the other senses, but to "conscious" or
> "explicit" ideas, meanings, and probably dominant intentions as well. I give
> a whole bunch of additional evidence for this general claim in my 1988 book.

I think it is important, as hinted above, to distinguish two distinct
claims:

(A) the contents of vision, consciousness, audition (motivation, ?) etc.
are constrained to be consistent. (I think this is false.)

(B) The mechanisms involved in self-awareness and in deliberation and
reasoning (or other things using a "global workspace"?) are inherently
resource limited. I.e. they can't simultaneously perform arbitrarily
many distinct, unrelated, tasks.

As far as (B) is concerned, I think it is true, for deep engineering
design reasons to do with the impossibility of implementing physical
mechanisms which violate it in a well integrated system.


Reasons for the resource limits restricting parallelism (and thereby
incidentally increasing coherence) include the following: (1) processes
involving combinatorial exploration require repeated re-use of the same
temporary storage space. (2) the high level processes use a long term
content addressable memory, which needs to use all available parallelism
for optimising retrieval speed, at the cost of ruling out answering
different questions concurrently (since cross-talk could result).
(3) An argument I first heard from Dana Ballard: various learning tasks
explode exponentially if too many things are done in parallel (e.g.
finding which subset of N concurrent actions produced some effect,
potentially requires considering and testing 2 to the power N possible
subsets, and it gets MUCH worse if delayed consequences are allowed).

I.e. (B) is not just an empirical fact about human consciousness, but
something deeper. It could apply to robots, martians, etc.

However the status of (A) is different: checks and constraints
preserving consistency have great heuristic power, and therefore we can
expect to find them in many places. However, since they are hard to
implement in general, and since an apparent contradiction may be an
important first clue towards an expanded conceptual framework giving
access to a deeper more general ontology, as often happened in the
history of science and mathematics, we should not expect the consistency
requirement to have any sort of ABSOLUTE status.

I.e. exceptions can occur, and may be an important part of learning or
development.

For me, perhaps the most compelling evidence that human vision does not
rigidly exclude inconsistent or incoherent percepts are the motion
after-effects in which one sees simultaneously motion, and nothing
moving, in the same location.

> On theory, Aaron claims all ideas we have today are computationally
> inadequate.

I didn't say "computationally inadequate". Computation is but one form
of information processing though its boundaries are somewhat ill
defined. (Does it include continuously varying systems, which are
non-computational in the sense in which computation requires a
succession of discrete states.)

When I talk about information processing mechanisms I leave open the
possibility of including mechanisms that are as different from today's
computational mechanisms as the latter were different from the cogs,
levers and strings of yesteryear.

> That may be true. People like Stan Franklin are working to push
> that envelope.

Stan's work is certainly an interesting example of a raft of
explorations of architectures for intelligent agents. (I have my own pet
models too.) But I suspect all this work will turn out to be missing
something very powerful, which is still waiting to be discovered.
(NB It has nothing to do with Godel's theorem, which I think is a
complete red herring.)

> As a person concerned primarily with evidence, I'm initially
> interested in EMPIRICAL adequacy. Then we have to try to fix problems of
> ALGORITHMIC adequacy. There are lots of examples in the other sciences that
> have worked just like this, where the mathematical algorithm became available
> after the evidence was reasonably well understood conceptually.

I personally prefer the approach of great physicists, like Newton and
Einstein: First (by inspired guesswork) produce a great new theory of
the architecture of the world, accounting for a wide range of already
known phenomena, and absorbing the best previous theories as special
cases, possibly inaccurate special cases. Then work out its new
consequences with as much mathematical precision as possible. Then do
experiments, etc. to see how the consequences fit reality, and where
they don't fit, be prepared to modify the theory, or consider
alternatives. If all the consequences are too vague or ambiguous to
allow any refutation then don't necessarily throw the theory away as
unscientific: try to sharpen it.

Algorithms provide just one sort of detail within the framework of an
architecture: don't be too impressed by them.

Nowadays any theory of adequate richness is likely to be too complex for
consequences to be derived without simulation. If simulation is
impossible, start worrying about whether that's because the theory is
too ill defined.

Enough. I've gone on too long again.

Cheers.

Aaron


From Aaron Sloman Thu Nov 27 13:58:42 GMT 1997
To: PSYCHE-B@LISTSERV.UH.EDU
Subject: Re: Baars on Binocular Rivalry (was Re: Illusions)

I am (belatedly) responding to these three

> Date: Mon, 24 Nov 1997 02:18:00 -0500
> From: "<Bernard Baars>" <BBaars8788@AOL.COM>

> Date: Tue, 25 Nov 1997 00:28:48 -0600
> From: George McKee <mckee@STARBASE.NEOSOFT.COM>

> Date: Tue, 25 Nov 1997 16:49:21 -0500
> From: Arnold Trehub <trehub@PSYCH.UMASS.EDU>

All are reactions to my message posted Mon Nov 24 01:19:46 GMT 1997

I guess I was both unclear in what I wrote and also intemperate in how I
wrote it. Apologies. Comes in part from always being in too much of a
hurry.

[Bernie Baars wrote]
> I'm afraid Aaron is not reading the plain meaning of my words. Let me quote
> from In the Theater of Consciousness, p. 89 ......
> .... But it's meant to be read with precision.
> .....
>  Internal consistency is a basic property of conscious experience, but the
> raw input to the brain may not start off as consistent.

I have a number of different problems which I had not thought through
clearly, and which caused me to focus not on the words in this bit
of text, but the ones I earlier quoted (see below).

But first let's look at the general problem I have about the notion that
conscious experience has to be internally consistent.

If someone told me he had designed some kind of general information
processing system which could contain information about the environment
and was constrained to be internally consistent I would not believe him,
unless the mechanism was very limited in the information it could hold.

There are two related reasons.

(a) in general telling whether something is consistent or not is a
problem which cannot be decided by any sort of mechanism. (Is it or is
it not inconsistent to say that there's a largest pair of twin primes,
i.e. prime numbers which differ by exactly 2, like 11 and 13? As far as
I know, that's an unsolved problem, though very simply stated. Maybe
some people believe there is a largest pair and others believe there
isn't. One belief is inconsistent, though nobody knows which.) Any
precisely defined, sufficiently rich, system cannot have a decision
procedure (Goedel, circa 1931. Compare the halting problem in
computing.).

(b) Even the cases that are decidable, e.g. expressions in propositional
logic, are not tractable in general. E.g. a logical expression involving
N variables requires a truth table of size 2 to the N to be explored to
check consistency, and exponential functions grow very fast. (E.g. an
expression involving a mere 10 propositons can require over a thousand
combinations to be checked.)

So I'd be amazed if the brain had any *general* method for detecting
(and eliminating or avoiding) inconsistency, and as I remarked there are
cases where it fails, e.g. perception of a picture of an impossible
polygon, with many sides. The devil's pitchfork (shown in Baars' book)
and the Penrose triangle are easily seen to be impossible, but a more
complex picture of the same general sort looks globally consistent when
it isn't. The inconsistency is not detected by a human visual system.

So I could not believe Bernie was making a *general* claim about
consistency in the global workspace.

I focused on his more specific comments, e.g. this:

    ...But binocular fusion breaks down if there are significant
    differences between the input to the two eyes. .... if one image is
    slightly offset from the other, binocular rivalry results, and to
    obtain coherence the visual system will suppress one image in favour
    of another. [Baars, p89, quoted previously]

This looks unambiguous and precise to me. And false, since rivalry
requires additional conditions, as I indicated a couple of messages ago,
and Stan Klein seemed to agree. I.e. the claim is too strong. But I
think Bernie also agrees with that. He has offered a different claim
which I'll come back to in a minute.

[George McKee writes]

> I have to agree with Aaron Sloman in his debate with Bernard Baars about
> consistency constraints in awareness.  To put it facetiously, Bernie needs
> to spend more time in looking-glass land with the Red Queen, believing
> six impossible things before breakfast.
>
> Baars appears to have a very precise notion of where "consciousness" resides
> in the system of stages between sensory transduction and muscular action --
> a notion that Sloman and I don't share.

I think there's more to it than that. It has to do not just with a
factual disagreement (as there was over rivalry), but with the problem
how to make sense of some of the claims which at first look as if they
are saying something clear, but (to me) lose their clarity when examined
closely.

As I've indicated in the past, I don't have a global disagreement with
Bernie: we are, I think, putting forward closely related ideas. I think
human intelligence involves a "central" architecture with (at least)
three importantly different kinds of mechanisms (reactive, deliberative,
meta-management) which evolved at different times, which often interact
strongly, which are shared (to varying extents) with different animals
(insects, rats, ....), which develop within an individual over time
between conception and adulthood, which are not necessarily directly
manifested in physical brain structure (because they may involve virtual
machines), which provided evolutionary pressure for a similar layering
of perceptual and motor mechanisms, which manifest themselves in three
quite different classes of emotions, which play different roles in
relation to skilled and unskilled performances, which use information in
different forms, which involve very different learning mechanisms, etc.
etc. (This is a crude and partly inaccurate summary.)

I think the global workspace idea has a lot in common with some features
of these ideas about architecture (especially the mechanisms involved in
meta-management), and goes much further in attempting to map details
onto brain structure.

But the consistency constraint looks like the wrong kind of thing to
include (for reasons indicated above), and it results in some very
obscure claims.

For example Bernie wrote
> > So I just don't see the problem. I don't know of a single, solitary example
> > showing  that "YOU CAN PERCEIVE TWO SOLID VISUAL OBJECTS IN THE SAME
> > PERCEIVED SPATIAL LOCATION AT EXACTLY THE SAME PERCEIVED TIME?"

At first I thought this was an empirical claim, but as indicated in my
previous message it began to evaporate as I looked more closely. E.g.
it's simply logically impossible for a solid cube and a solid sphere to
be in the very same spatial location at exactly the same time. If they
occupied the same 3-D volume they would both be cubes or both spheres,
or both something else.

So if being in the same perceived location means occupying the very same
spatial volume then I have no idea what sort of visual experience is
being excluded. If someone claimed to have such a visual experience,
rather than treating that as an empirical refutation of Bernie's theory
I would treat it as evidence of his misusing the words describing the
experience.

I can easily imagine visual experiences in which a cube and a sphere
will be seen to have the same *centre*, but each will protrude from the
other. I.e they are partly in the same place at the same time.

Probably such objects already exist. The curved and flat surfaces could
be given different colours. You might then see a red cube and a blue
sphere in the same place at the same time, each partly obscuring the
other.

But I expect Bernie would say that's not a refutation because it's a new
coherent experience of a geometrically possible object which is wrongly
described as two objects.

So what about binocular presentation of cube in one eye and sphere in
the other? I haven't tried, but I have tried something similar which is
a variant of the experience of seeing your finger and the distant
background at the same time, when the finger splits into two.

Get two pens or pencils with different markings on their surfaces, and
perhaps different colours. Hold them in front of you, maybe 18 inches
(45 cm) away, each sloping at about 45 degrees, one to the left and one
to the right, so that they touch in the middle and form a cross, with a
distant background beyond them (e.g. the far wall of the room).

Fix your vergence by looking at a distant background, while keeping the
pencils in focus (as if viewing a stereogram without a stereo viewer).
Ensure that there's a good strong light on the pencils so that you see
surface details e.g. lettering, texture, wood grain, very clearly.

Each pencil should split into two parallel pencils with the four of them
taking this sort of shape, where I've replaced each pencil with a single
thin dashed line, instead of two lines, to save tedious drawing in a
text file:

          A1  A2  B1  B2
           \   \  /   /
            \   \/   /          Pencil A is seen as A1 and A2
             \  /\  /
              \/  \/            Pencil B is seen as B1 and B2
              /\  /\
             /  \/  \
            /   /\   \
           /   /  \   \

Now tilt the further pencil slightly towards you and the nearer pencil
slightly away from you so that at the upper point of intersection (the
place where A2 and B1 intersect) they appear to be at the same distance.

You can wiggle them slightly or move the pencils on their long axis to
keep relative distances unchanged, to see what's going on. I actually
see two pencils passing through each other. I can see, quite clearly,
the marks on both surfaces (I've learnt to focus on close up surfaces
while my eyes converge in the distance in order to view various kinds
of stereograms.)

It's actually quite hard to describe the experience precisely, because
it's not something for which our normal vocabulary for describing
physical configuration is apt: we don't often meet this sort of thing
except perhaps in ghost films. But saying that the pencils pass through
each other comes as close as anything. Now because of the unclarity in
Bernie's claim that he doesn't know of a single, solitary example

> > showing  that "YOU CAN PERCEIVE TWO SOLID VISUAL OBJECTS IN THE SAME
> > PERCEIVED SPATIAL LOCATION AT EXACTLY THE SAME PERCEIVED TIME?"

I don't know whether he does or doesn't regard this as a refutation.

Certainly I do *not* get the binocular rivalry which he seems to think I
should get, i.e. one of the pencils being suppressed at the point of
overlap.

Of course I may have entirely misunderstood what he (like many others
who talk about rivalry) is saying. I am not a brain scientist.

Or maybe I am a mutant after all.

(Bernie is that a logical possibility according to your theory? I.e.
could there be a sub-species whose global workspace mechanism does not
use the consistency constraint? Or have you ruled it out on
non-empirical grounds?)

I wonder if deep down he is ruling it out apriori:

[Baars]
>  As a mathematician, Aaron may be acutely aware of contradictions that we can
> be conscious ABOUT --- but that is vastly different from consciously
> experiencing two sources of  contradictory information at the same time. Just
> try thinking of electrons as waves and particles AT THE SAME INSTANT IN TIME.
> If you think you can do it we've got to get you into a cognitive laboratory.

Er, I think many physicists now do that all the time. It's not
particularly hard for people used to thinking about wave packets, etc.

In any case, "consciously experiencing two sources of contradictory
information at the same time" (my pencil example) is very different from
thinking of electrons as having wave-properties and particle properties
at the same time. The latter need not involve any sensory modality.

Space-filling curves are harder (was that Sierpinski's work?). That's
because they exist only as the *limit* of a process of growing them, and
the notion of the limit of an infinite sequence of steps as something
actually achieved has its own problems.

Thinking of an infinite set as the same size as a part of itself was a
bit difficult at first, but soon became easy, even though there's a sort
of contradiction (It's the same size as something smaller!)

Arnold Trehub writes about one of my examples:

[AS]
> > For me, perhaps the most compelling evidence that human vision does not
> > rigidly exclude inconsistent or incoherent percepts are the motion
> > after-effects in which one sees simultaneously motion, and nothing
> > moving, in the same location.

[Trehub]
> My experience of motion after-effects (and that of others who have
> reported their experience to me) is quite different. I see all
> (objectively) stationary stimuli in the field of induced motion as streaming
> in a direction opposite to the motion of the inducing stimuli. I *believe*
> the test stimuli are not moving but, at the same time, I *see* them in a
> peculiar kind of motion.

Let's make sure we are talking about the same thing. I view a text
window with the text smoothly scrolling up. Then when it stops I can see
downward motion even though I see no text or other visual object
actually changing its location either relative to the window frame or
relative to the direction in which I am looking or relative to anything
else. I.e. there's both visible motion and nothing moving visibly, i.e.
no change of absolute or relative location.

If you and your informants see the text moving relative to the frame, or
relative to the direction of gaze, i.e. letters actually changing their
location, then it's a different sort of motion after effect from the
kind I am referring to. (However, I suspect it may be the same one, not
inspected with sufficient care, because the relevant questions were not
asked -- a common problem with psychological experiments.)

(I sometimes get a similar effect while flying if a plane banks while I
am sitting still gazing at the bulkhead. I presume proprioceptive
sensors and pressure sensitive sensors, etc. send signals to my brain
recording a tilt and I *see* a tilt happening, yet nothing actually
moves in my visual field. It's quite strange. Some other people confirm
that they have had a similar experience.)


[McKee]
> Sloman offered a long list of visual situations that might be violations
> of Baars' strong consistency-of-perception claim.  I could add more, such
> as low-frequency periodicity pitch tones where you can perceive (well,
> *I* can...) the pulses as individuals and/or as a continuous tone.  I was
> particularly struck by the example of paintings such as the vegetable face
> or trompe-l'oeil paintings in general.  I don't feel any perceptual strain
> when seeing these as multiple classes of objects simultaneously, or even
> as seeing them in two and three dimensions simultaneously.

This is very important. Although I don't yet know much about the types
of information structures and information mechanisms involved in what I
call the meta-management system (which can attend to internal states),
or in the global workspace, one thing seems to be clear: the structure
is not anything like a spatial model or replica of some external reality
since if it were there would have to be a homunculus perceiving it and
we'd get the familiar infinite regress. Rather it's got to be something
more like a richly and immediately accessible collection of "pointers"
to sensory contents (perhaps analysed at different levels of
abstraction) combined with perhaps something like "pointers" to
predicates/predicates/concepts/schemata providing interpretations and
classifications (including identified possibilities for motion,
functional and causal relationships etc., i.e. Gibson's
"affordances"???).

This perspective (admittedly still far too vague to be a theory or
specification for a mechanism) allows the possibility of combining
inconsistent predicates/concepts/schemata, though it's not surprising
that in SOME cases the inconsistency is detected and rejected, and one
interpretation dominates.

E.g. if there are already mutually inhibitory linkages between two of
them they can't both occur in the same structure at the same time. If
there are not (e.g. because the inconsistency has not yet been noticed,
or because frequent co-occurrence shows that the inconsistency needs to
be endured) then the consistency constraint will not apply.

I suspect the mechanism allows enough plasticity to permit cultural and
individual variation, and even variation within one individual over
time. e.g. do very young children see the penrose triangle as
impossible, or does that come later? Some people may have learnt to see
it as a Gregory model (Richard Gregory built a 3-D structure which, when
viewed from a particular direction, looks like a Penrose triangle. I.e.
it looks impossible because two things which are actually at different
distances look as if they form a junction, from that viewpoint. It takes
time to learn to think about curved 3-D space, or wave-like particles.

[mckee]
> The background
> image I've installed on my PC (a rocky seashore scene) was chosen expressly
> because it generates a percept of great depth -- but that doesn't mean that
> I lose the semi-flat percept of the front of the CRT when I pay attention
> to the represented far horizon.

I.e. two classes of inconsistent schemata/concepts or whatever are
simultaneously appled to the same sensory contents. No problem for us.
How many other animals can do it?

[Mckee]
> Does this mean that my nervous system is abnormal?  Should I consult a
> neurologist about this?

I'd say it's a manifestation of a type of combinatorial mechanism
outlined in my previous message and above, which contributes to the
power of human brains, and which makes consistency detection possible in
special cases and intractable in general. (I suspect mathematicians,
composers, novelists, programmers, .... have it to a higher degree than
most ordinary people.)

Maybe there are different kinds of brains, and George and I (and others)
share some gene not everyone does?

Is that possible? I've certainly met people who appear to be totally
incapable of viewing stereograms which I (and others) see easily. Is it
possible that in some people the Baars consistency constraint operates
in a more powerful fashion, so that rivalry occurs? I wonder what
happens to them in normal binocular vision?

There certainly are differences in visual systems. Many people can see
colours which I can't.

[mckee]
> The standard way science has of breaking out of conceptual boxes like
> the one Baars has described is to look at the system in more detail.
> But a naive conception of detail as microscopic space or time doesn't
> work with a holistic phenomenon like consciousness.

I agree and have been trying to make similar points in a different
thread, regarding different temporal frameworks for different classes of
events.

He goes on with some interesting comments on how consistency constraints
might break down in certain neural mechanisms.

> ....
> ....What happens in my mind (and probably in Aaron's)
> is that a higher-order process ....
> .... has learned the shape of both attractors
> and can hold the percept exactly on the separatrix between them.
> This usually takes more effort than it's worth, but it can be done.

I don't know if this is right. But if it is right, it may be that this
kind of "effort" is exactly what is required for the kinds of human
creativity referred to in my previous message.

He continues
[Mckee]
> I'm not sure where higher-order processes fit into GW theory, but
> it seems to me that they are essential to a fully-developed theory
> of consciousness.  I also think that they are incompatible with any
> strong claim about the unity of consciousness at our current level
> of sophistication. .....

Agreed. The unity of consciousness is not something "directly" observed
as often supposed, but, I suspect, something imposed in culturally
influenced introspective states, just as perception of external
phenomena (e.g. words on a sheet of paper, forms of dance, etc.) can be
strongly influenced by cultural learning.

Cheers

Aaron
PS: in case anyone wants to look back at my earlier comments referred to
above I've put my messages on this topic in a plain text file at
    http://www.cs.bham.ac.uk/~axs/misc/binocular-rivalry
including this message.
====
Aaron Sloman, ( http://www.cs.bham.ac.uk/~axs/ )
School of Computer Science, The University of Birmingham, B15 2TT, UK
EMAIL   A.Sloman@cs.bham.ac.uk
Phone: +44-121-414-4775 (Sec 3711)       Fax:   +44-121-414-4281

From Aaron Sloman Mon Dec  1 01:33:20 GMT 1997
To: PSYCHE-B@LISTSERV.UH.EDU
Subject: inconsistent Necker?

Until today, if anyone had asked me how many ways I can see the Necker
cube, I'd have answered "two". But following on recent psyche-b
discussions I though I'd try harder. So I sat staring at this
                                  D
                 *---------------*
                /|              /|
               / |             / |
              /  |            /  |
             /   |           /   |
            *----+----------*B   |
            |    |          |    |
            |    |          |    |
            |    |          |    |
            |  A *----------+----*
            |   /           |   /
            |  /            |  /
            | /             | /
            |/              |/
           C*---------------*

Sure enough, after not very much effort I was able to create a new
slightly bizarre experience, in which the two vertices labelled A and B
above came out of the screen at me, with edges AC and BD both sloping
away. However, the rest of the scene became a rather messy somewhat
incoherent configuration which is hard to describe and the whole thing
was not very stable. (A curious experience of fighting my own brain, and
sometimes winning, sometimes losing.)

Am I the only person in the world for whom this is possible?

Aaron
===

From Aaron Sloman Sun Dec  7 02:00:13 GMT 1997


From Aaron Sloman Sun Dec  7 18:45:49 GMT 1997
To: PSYCHE-B@LISTSERV.UH.EDU
Subject: More on consistency: two rival viewpoints

I've been fascinated by the number and variety of responses to my
semi-tongue-in-cheek note about my incoherent experience of the necker
diagram with vertices A and B coming out of the picture.

I had actually expected far more reaction to the earlier message
reporting the experiment in which two pencils with different
orientations are seen passing through each other, so that two
differently oriented cylinders are seen in the same location. But maybe
that's too unfamiliar, or too hard for most people to do.

Anyhow, here are some observations arising out of the portions of the
discussion which I've managed to take in, and a confession in response
to

    Date: Mon, 01 Dec 1997 11:53:36 +0000
    From: Chris Malcolm <cam@DAI.ED.AC.UK>

> ....do you really mean to say you can't see it flat?

Yes, I was being sloppy. I agree: I've often pointed out to my students
that the necker cube can be seen as a flat 2-D configuration, and that
when drawn suitably it makes a nice symmetrical hexagonal flat pattern.
It was silly of me to say I had found only two interpretations
previously. (I was too influenced by the context: discussion of 3-D
views of the cube.)

And yes, various other people are quite right in saying that besides the
third non-2D interpretation which I described I should have pointed out
that there are others.

However, in the context of discussion of a claim that visual experiences
must be internally coherent it sufficed to produce just ONE counter
example -- and in my experience of vertices A and B coming out towards
me the whole experience really did become incoherent, as well as being
unstable in that form. I.e. it was not blurred, and not oscillating,
just incoherent, and very hard to describe, except for the locally
consistent central bit.

A few other people who responded seem to have experienced something more
coherent. (I could not view Minsky's picture as I am using a unix
system, via a slow text-only link from home, and I couldn't decode his
binhex file.).

Perhaps if I had tried harder, I could have seen a consistent non-cubic
version with curved 3-D lines projecting onto straight 2-D lines. But
that did not emerge spontaneously for me.

In any case that was only one of  several counter examples to the
consistency claim. We've also since then had non-visual counter
examples, e.g. John Barnden's sunset puns.

(In case anyone noticed the email address and wondered: John and I do
occupy adjacent offices, but were not in collusion: until I saw his
message, I was not aware that he was a lurker on this list.)


HOW FAR HAVE WE GOT?

It seems to me that in the discussion (or what I've taken in so far)
there are two views.

View A is that the contents of consciousness must be internally
consistent because the brain uses some mechanism for enforcing this.

View B is that sometimes the contents of consciousness are consistent
and sometimes they are not: i.e. internal consistency is not an
*absolute* requirement for the contents of consciousness, but may often
be a consequence of other requirements.

I'll elaborate on both views for a while.

View A: Consistency rules OK
----------------------------

On this view an incoherent conscious visual percept is impossible: the
brain can unconsciously handle inconsistency between interpretations or
views which never reach consciousness, but the contents of consciousness
are *always* coherent.

One version of this states that if two inconsistent streams of sensory
information come in via the two eyes, both are processed unconsciously
but if they cannot be fused into a consistent percept then binocular
rivalry ensues and only one can be consciously experienced at a time.

Likewise, on this view, a sentence cannot be heard with two meanings at
the same time. Similarly, I assume, it's not possible simultaneously to
experience motion while experiencing all perceived objects as
stationary.

I.e. on View A, the "global workspace" or whatever it is that holds the
contents of consciousness is constrained so that it cannot admit
inconsistent contents.

I have the impression that's the majority view.

There are some residual ambiguities as to what exactly this view does or
does not include. E.g. I am not clear whether Bernie Baars allows the
view to be qualified by saying that *some* percepts are actually
inconsistent, even though the brain *tries* to avoid that. He explicitly
rules out two inconsistent contents coexisting in the global workspace
(i.e. seeing two views of the necker cube at once) and for a while I
thought this claim

    No matter what we do, the visual system tries to find a single
    coherent conscious interpretation at any given moment.
        (Page 87 of "In the Theater of consciousness")

implied that incoherent conscious interpretations could never exist.

But now I am not sure. Maybe he does allow (like Stan Klein) that we can
experience the devil's pitchfork and other pictures of impossible
objects as globally incoherent, like my unstable view of the necker
cube.

Stan Klein allows the possibility of incoherent experiences:

    Date: Sun, 30 Nov 1997 20:25:24 -0800
    From: Stanley Klein <klein@ADAGE.BERKELEY.EDU>
    Aaron is correct that there are many, many local interpretations of any
    stick figure. Very few of those interpretations are globally consistent as 3-
    dimensional objects. Sometimes our perception corresponds to the
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    inconsistent interpretations, but usually it picks out the consistent ones.
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    There are of course a number of illusions (Escher) without consistent
    interpretations. The challenge to the neuroscientists is to figure out the
    circuitry of how the brain does its global calculation unconsciously,
    usually resulting in the conscious perception of a consistent
    ^^^^^^^
    interpretation..

I would not call the Escher pictures illusions, but apart from that we
are in agreement.

The underlined bit implies that Stan thinks there are *sometimes*
internally inconsistent visual experiences when we look at pictures of
impossible objects, though *usually* a consistent one dominates.

My impression of the global workspace theory is that it rules out Stan's
view, but perhaps I have simply been misreading it all along.

My (mis)reading is based on comments like this: "conscious contents
are always (sic) internally consistent" (from his recently circulated
paper "A thoroughly empirical approach to consciousness:  Contrastive
analysis.").

But his remarks on the devil's pitchfork leave me uncertain:

    "...an impossible figure that looks from a distance as if it is
    coherent --  but trace its lines with your eyes, and your
    visual brain gets into trouble." (In the Theater, p.87)

He doesn't say exactly what that "trouble" involves, e.g. whether, as
Stan suggests, it is actually a case of having an incoherent visual
experience because the brain cannot find a way to resolve the
inconsistency.

He links this to "... our inability to keep two inconsistent perceptual
experiences in mind at the same time."

However an impossible object picture does not produce *two* inconsistent
perceptual experiences. Rather, as Stan implicitly says, it produces
*one* internally inconsistent experience, just as this is one internally
inconsistent conjunctive proposition:

    A is nearer than B and B is nearer than C and C is nearer than A.

As in the penrose triangle, no two parts are inconsistent, but the whole
proposition has implications which are inconsistent. You have to do some
work to extract them. E.g. using transitivity of "nearer", you can get
"A is nearer than A" which is internally inconsistent (on its normal
interpretation.)

The Penrose/Escher impossible figures likewise do not include two
inconsistent objects or experiences. Rather they include fragments which
are individually and pairwise consistent, but have globally inconsistent
implications, which it takes some work to detect. In an n-sided Penrose
polygon any n-1 corners will be mutually consistent, so detecting the
inconsistency gets harder as n increases, except for those who have
moved up a level of abstraction and can detect the high level pattern.

[I wonder whether there is a developmental stage at which the
inconsistency even in the triangle is not detected because the visual
mechanisms have not yet developed the ability to chain the required
inferences.]

Some of Escher's more complex pictures have incoherence which even some
adults don't notice at first, e.g. the ascending and descending hooded
figures, and perhaps the waterfall.

The devil's pitchfork and similar figures are different (E.g. see "in
the theater of consciousness page 88 for an illustration). Here you have
an experience which includes the same sub-object being interpreted
differently in different parts of the picture.

E.g. at one end an edge is seen as convex and at the other end it is
seen as concave. In a world of curved objects that would be
OK, but when the picture is seen as having only plane surfaces and
straight edges it isn't OK.

Another inconsistency comes from surfaces: one of the long surfaces is
seen at one end as horizontal and at another end as vertical.

If the brain really required consistency in conscious experience, you'd
expect it to invent a curved surface linking the two ends, but it
doesn't normally do that spontaneously, although in other contexts we
have no difficulty seeing curved surfaces. This is a clue to the
mechanisms that are at work, on which more later. (Ask yourself: Why
should the presence of straight lines rule out curved surfaces, when it
is perfectly possible for curved surfaces to project as straight edges?)

Again, I wonder whether up to a certain age a child's visual system
would simply accept that incoherent interpretation because it cannot
detect the inconsistency.

Even with simpler figures an adult may fail to detect the inconsistency
if the picture is stretched so that the locally consistent but mutually
inconsistent components are not close together. Also I suspect that if
the devil's pitchfork is included in a picture showing a cluttered
collection of objects of varying shapes on a table it might turn out
that the whole collection is experienced without the incoherent object
being seen to be incoherent.

I.e. there's a mostly but not totally consistent visual experience, and
people who have the incoherent experience can be unaware of the fact
that it is incoherent. (As indicated above, some of the Escher drawings
have that quality.)

In the last chapter of The Logic of Perception (MIT Press 1983) Irvin
Rock discusses some of these issues, and seems to accept reluctantly
that in some cases the brain produces an incoherent percept because the
sensory data drive an incoherent interpretation that cannot be altered
as a result of detecting the inconsistency, even if one knows that a
different percept would remove the inconsistency (e.g. curved surfaces
in the devil's pitchfork, or Gregory's wooden Penrose triangle).

Tentative conclusion:
    1. The visual system sometimes produces inconsistent conscious
       experiences
    2. In some cases the incoherence is detected, in others not
    3. The incoherence is most easily detected by the visual system
       when
            (a) this does not involve multiple chained inferences
            (b) the sources of incoherence are close together
    and maybe:
            (c) there isn't distracting clutter
    4. When incoherence is detected it sometimes forces a revised
       experience (e.g. suppressing one eye's input in binocular
       rivalry, or re-interpreting minor binocular disparities as
       as due to depth differences).
    5. However, when the perceptual data *strongly* imply an incoherent
       interpretation that is sometimes not overridden.

This leads to:

View B (first draft, subject to revision):

B.1. Consistency of contents of consciousness can be violated
     in a range of different ways.
B.2. When experiences are internally consistent that may be a
     consequence of something other than a general consistency
     constraint.

This minority(?) view seems to have been expressed in different ways in
recent messages by, among others, Marvin Minsky, George McKee, John
Barnden and myself. It's also implicit in Stan Klein's remarks, and in
the admission in Rock's book (which Rock appears to wish he could
avoid).

The general theme I see emerging is this:

    Although there may be *local* consistency constraints the apparent
    evidence that the brain somehow constrains the contents of
    consciousness to be totally consistent is merely evidence for a
    biologically useful mechanism (explained later) which *generally*
    produces the result that the contents are consistent, but does not
    necessarily *always* do so.

There are three different lines of reasoning leading to the minority
view.
1. No known type of mechanism could implement the general consistency
   constraint.

2. Empirically there are counter examples.

3. There are more biologically plausible and biologically useful,
   mechanisms whose side-effects would include achieving consistency
   most of the time, without totally ruling out inconsistency.

I'll elaborate on these three later in a separate message, as this one
is already too long.

Aaron


From Aaron Sloman Mon Dec  8 10:33:33 GMT 1997
To: PSYCHE-B@LISTSERV.UH.EDU
Subject: More on consistency and consciousness: reasons for view B

To recapitulate:

View A is that the contents of consciousness must be internally
consistent because the brain uses some mechanism for enforcing this.

View B is that sometimes the contents of consciousness are consistent
and sometimes they are not: i.e. internal consistency is not an
*absolute* requirement for the contents of consciousness, but may often
be a consequence of other requirements.

I said there were three types of reasons for preferring B:

1. No known type of mechanism could implement the general consistency
   constraint.

2. Empirically there are counter examples.

3. There are more biologically plausible and biologically useful,
   mechanisms whose side-effects would include achieving consistency
   most of the time, without totally ruling out inconsistency.

I'll now elaborate on each of these in turn.

1. No known mechanisms could do it.

People who have worked on the general problem of inconsistency
detection and eradication know that it is inherently intractable, for
the reasons indicated in an earlier message: i.e. in the most general
case it is undecidable (Go"del's incompleteness theorem, etc), and even
decidable cases (e.g. propositional logic) are combinatorially
explosive, and therefore intractable in general.

Even if the brain does not use propositional or predicate logic it seems
(in the case of humans) to have combinatorial capabilities both as
regards the variety of 2-D images that can be "parsed" and the variety
of 3-D percepts that can be formed. Motion merely adds to the
combinatorial complexity.

Anyone who doesn't believe that inconsistency detection is intractable
in general, should try to design in information processor which can tell
whether an arbitrary collection of propositions is consistent or not, or
an arbitrary representation of a visual scene, including all variants of
Penrose/Escher figures that can be taken in as a whole. Finding a
non-exponentially explosive (in space or time) way to check consistency
of an arbitrary propositional expression could bring fame and fortune
and answer one of the open questions in the theory of computation.

Some people may think that parallelism in neural nets can defeat the
combinatorics, but although I have not thought it through, I suspect
there is probably a theorem about limitations of perceptron-like neural
nets on this sort of task, analogous to the Minsky/Papert results
regarding parity and connectedness: such essentially global properties
require serial mechanisms (or enough perceptron layers to simulate
serial processing).

Marvin, is that right? (Perhaps I am out of my depth here.)

Vision has to work fast, at least if you are a bird, squirrel, chimp,
tennis player, car driver, etc. So an engine that sits chugging away
looking for ever longer sequences of inferences till all possibilities
are exhausted is unlikely to work for the purposes of vision.

If there are at most N layers of consistency checking in the visual
networks preceding the stage of visual processing which feeds into the
contents of conscious experience (whatever that means, but forget that
for now), then perhaps a class of inconsistencies detectable in N steps
could be eliminated, but not others. We seem to be able to detect 2-step
and 3-step visual inconsistencies fairly easily, at least as adults.

(What about other animals and young children?)

(Maybe quantum-gravity computers can do better than human brains??
Is Penrose listening?)

2. There is empirical evidence that View A is false.

I have presented a number of cases of different sorts in previous
messages, saved in the file

    http://www.cs.bham.ac.uk/~axs/misc/binocular-rivalry

One class of counter-examples discussed previously involves examples of
perception of "long range" Penrose/Escher figures where detecting the
inconsistency requires more than two or three steps of transitive
inference, e.g. a Penrose decagon. It's easy to see such a complex
figure as locally consistent everywhere without noticing that if you
perform an inference chain involving as many steps as the sides of the
polygon that it leads to an inconsistency: X is nearer than Y and X is
further than Y.

As Stan Klein and Irvin Rock (among others) admit, pictures of
impossible objects can be perceived as impossible objects. (I've seen an
advertisement in which the typical line drawing is replaced by a superb
coloured, textured picture of a Penrose/Escher object, which is easily
experienced as a realistic photograph of a globally impossible object.)

Even in the case of the simple Penrose triangle I think I have no
problem experiencing it as a complete but incoherent 3-D object,
likewise the devil's pitchfork etc. That's why those pictures are fun:
they are not like the necker figure which (normally) flips between two
consistent interpretations.

My binocular view of two pencils passing through the same bit of space
is inconsistent in a different way. Likewise the motion after-effect
described previously, in which motion is observed but nothing changes
its location.

Bernie previously reacted to some of this by asking whether there is
evidence that two (differently shaped?) objects can be seen in exactly
the same place. I previously tried to indicate that on one
interpretation of the question this is possible (a cube and a sphere of
the same diameter embedded in each other), and on another interpretation
it is impossible because the question is then incoherent: A cube cannot
occupy the same volume of space as a sphere because if it did it would
be a sphere, not a cube.

Likewise: one reason it is impossible to see the necker cube
simultaneously in both of the "standard" ways is not because the brain
has a mechanism imposing a consistency constraint but because it is
logically impossible and therefore no mechanism is needed to rule it
out.

That's because seeing both interpretations in the same place at the same
time would require seeing a particular edge as both sloping further away
to the right and sloping nearer to the right, and that would require the
existence of neighbouring bits of the edge, e1 and e2, such that e1 is
nearer than e2 and also further than e2. I.e. the very description is
incoherent, if the two views are supposed to share edges in 3-D. It's
really just the same point as the earlier point that if a cube were in
exactly the same space as a sphere it would not be a cube. Likewise if
an edge sloping away to the right were seen in exactly the same location
as an edge sloping nearer to the right, then it could not be an edge
sloping nearer. If they are in the same place they have the same slope.

Well then, is it possible to see the necker drawing as depicting not two
differently oriented cubes in exactly the same place, but as two
wire-frame cubes with corresponding edges sloping differently but
*completely aligned*, i.e. either with edges passing through each other
or one nearby and one further away, one cube flipped one way and the
other cube flipped the other way, while the edges of BOTH project onto
the same lines on a 2-D image plane?

On that interpretation the cubes are not sharing all their edges, but
the edges of the two cubes are paired so that they cannot be visibly
distinguished.

That experience would depend on the ability simultaneously to experience
a 2-D line, e.g. this: _____________________________,  as two coexisting
3-D lines, one sloping away to the right and one sloping away to the
left and both projecting to the same 2-D line. Well, I've just tried,
and I *think* I did it but frankly I am not even sure how to tell the
difference between trying and failing and trying and succeeding.

It's like the example I gave in an earlier message of seeing a square
circle from the edge. Can we denounce as a liar an individual who
sincerely claims to see a line that way? Maybe, like Einstein or Mozart
she can do things we can't?

So: empirical evidence shows that incoherent visual experiences are
possible, as indicated above and in my previous messages (and as stated
by Stan Klein and by Irvin Rock). Sometimes they are detected as
incoherent, e.g. the penrose triangle, sometimes not, e.g. penrose
dodecagon, or a long devil's pitchfork, or cluttered Escher drawing,
etc.

I am not sure that a very young child would detect the inconsistency
in the simple cases.

(I once asked a five year old child where two cars would meet if they
start off in this configuration, and drive towards each other, where A
is an ordinary slow car and B is a very fast racing car:

    A -->                               ^      <=== B
                                        C

He pointed at a location nearer to B, e.g. C. I asked why, and got the
answer:

    "It goes faster so it will get there sooner."

He was totally unaware of the inconsistency in what he was saying (and
believing, presumably. Some of Piaget's observations are also relevant.)

If View A is expressed in a particularly strong form requiring not just
visual percepts to be internally consistent but the total contents of
consciousness then there are of course even more counter examples, e.g.
perceptual experiences which are known to be wrong.

Example of the flipping folded card: Get a card about 20cm by about
10cm. Fold it down the middle to form a V with an angle of about 30
degrees (or more, or less: sizes and angles are not critical).
Then either put it on the table with the V edge on the table, forming
two walls meeting at a corner or balance it with the ends of the V on
the table and the folded edge horizontal above the table, like a ridge
tent.

Stare at it from a few feet away with one eye for a while, and you
should be able to make it appear to flip from one of the two
configurations to the other.

When it has flipped, move your head from side to side a little and watch
it twist and slide on the table. I KNOW the card is rigid and stationary
and I simultaneously SEE it twist and move. Anyone who claims that I
really must be alternating between the two states of consciousness must
be too much in the grip of a theory to face facts.

Or maybe we have different sorts of brains.

Summary of second argument against the consistency hypothesis: there are
empirical counter examples if you look for them.

A qualification is in order, unfortunately:
Some of the issues which appear to be empirical may not be because the
questions asked can be shown to be incoherent.


3. Argument 3: there's a better explanation of consistency

The third argument was hinted at in previous messages by Minsky and one
of my earlier postings and possibly others. A version of it can perhaps
be read into George Mckee's message of 25th Nov about high-order
processes though I am not sure.

The argument is that some of the cases where consistency is found may
actually be cases of something else: lets call it "invocation of
perceptually learnt models or schemata".

The basic idea is very old: it pervades the writings of Richard Gregory,
is developed in some detail in Minsky's paper on Frames (circa 1973?)
goes back much further to Kant and Bartlett, is implemented in a number
of AI vision systems and will probably often be re-invented, though
possibly with newer more powerful implementation mechanisms.

Visual mechanisms in part have the role of identifying perceived objects
or processes either as belonging to some class (e.g. flower, chair,
furniture, spinning, sliding, fighting, breaking, supporting, etc.) or
being a known individual (Fido, Freda or Freddy, the moon, or the Eiffel
Tower, etc.) For now lets ignore the difference between classification
and (re)identification, and just use the neutral notion of "applying a
schema" (rather than "recognizing a pattern", to allow for the
variability in the data and the flexibility in the mechanisms, as
noted by Kant and others centuries ago).

Some of the categories used may be innate (e.g. edge, surface,
edge-feature, curvature, orientation, closure, linear radial and
circular optical flow, etc. in humans, and probably other things in
other animals, e.g. the tongue-aiming bug-detectors in frogs) others
learnt (e.g. learning to read words or music, learning to identify
plants or birds, learning to see a pair of identical twins as looking
very different (which took me a couple of months) and learning to
recognize most of the "affordances" in our environment, in Gibson's
sense).

For now it doesn't matter which are innate or which learnt: both are
derived from experience of conditions for success and failure of
actions, or merely by induction from salient frequently encountered
examples, whether the derivation is done by the individual or by the
gene pool working through many individuals in an evolutionary process.

The only important point is that as long as the process of derivation
from ACTUAL examples is fairly direct, the derived schemata will be
internally consistent. Where the derived schemata involve combinatorial
generalisation (e.g. induced grammars, induced composition rules for
geometrical structures or forms of motion) then it's possible to create
a schema or schema instance which is not derived from any actual object
and which turns out to be internally inconsistent. (Like the village
barber who shaves all and only those who don't shave themselves.)

The rest of the story should be fairly obvious. We need a principle
which can be roughly formulated thus, ignoring for the moment what
percepts or thoughts might be.

    The visual system tries to find a way of accounting for as much of
    the available information as possible by creating percepts
    instantiating the smallest possible number of high level schemata
    (in the sense of Kant and Barlett).
[OOPS I meant Hebb]

    The cognitive system attempting to understand some complex
    collection of information attempts to create a thought using
    as few schemata as possible which together subsume all the
    information.

or in other words:

    If we can recognize some intelligible high level structure in a
    perceived scene or thought-about situation we will normally do so.

where intelligible means something like "accommodated in known
schemata". (The schemata need not be previously known: maybe some are
created on the fly. A lot of art is like that.)

(The biological advantages to an animal of being able to do this, and
the engineering advantages to a robot, should be obvious, but we could
discuss them.)

Corollary: wherever schemata directly abstracted from previously
encountered concrete cases are deployed the thought or percept will be
internally coherent.

However, sometimes the data driving the process will invoke novel
combinations of schemata and then there's no guarantee that the result
will be coherent. Penrose/Escher figures and the devil's pitchfork are
examples. Likewise puns?

CONJECTURE: some of the innately determined perceptual schemata will be
implemented at a very low level in brain mechanisms using mutual
excitation and inhibition in "winner takes all" networks, so as to
ensure that wherever possible local ambiguities can be resolved by
combinations of top down and bottom up and sideways processing in
something like a "relaxation" process. (An old idea. E.g. I learnt it
over 20 years ago from Geoff Hinton, but perhaps others had thought of
it earlier.)

Sometimes these networks will be driven by data supporting a small
number of stable states equally well (e.g. necker figure) and then
various mechanisms can make the network flip from one stable state to
another.

In some cases extra bottom-up clues (shading, texture, etc.) or top-down
thought processes ("this is coming towards me") can drive sub-schemata
to form smaller stable, but no longer mutually consistent alliances.

That would account for some of the strange incoherent views of the
necker cube.

Maybe similar winner-takes-all multi-stable networks are involved in the
management of the contents of thought processes, though here we don't
get a rich dense collection of inputs directly clamped by the
environment, and there seems to be more scope for structural variation.

(Consider the differences and similarities between proving a deep
mathematical theorem in your head, reminiscing about your holiday,
planning a meal for your friends, composing a sonnet or limerick,
wondering whether someone reciprocates your feelings, thinking
about consciousness, etc.)

Because of the combinatorial richness of linguistic processes, and their
disconnection from current sensory inputs, linguistically mediated
thoughts will be even less constrained to be consistent than perceptual
contents, although when you are thinking about actual situations they
will normally be consistent because anything actual must be. (I
am speaking more loosely than some philosophers and logicians
would like).

This is why it is much easier to produce ambiguous or inconsistent
sentences, puns, linguistic jokes of various kinds, and nonsense
sentences, than pictures with those qualities. But of course expert
cartoonists have been doing it for a long time: e.g. drawing a face
which is clearly recognizable as a certain politician and as a certain
animal.

To summarise point 3:
Although perceptual contents are normally internally consistent
and thought contents often are, this consistency is not something the
brain can enforce. Rather it's a SIDE-EFFECT of powerful biologically
useful processing mechanisms which by and large have developed to fit
the actual world, not a world full of things that are impossible in our
world.

That's just as well, since if the brain needed to detect and enforce
general semantic consistency in any particular sub-structure it would
require mechanisms which, as far as we know, cannot exist.

By allowing (as Stan Klein and Irvin Rock do) that the search for a high
level consistent integrating schema subsuming the current visual
information can fail, we allow the possibility of a globally
inconsistent percept involving smaller scale locally consistent
percepts.

And that seems to fit empirical data about inconsistent percepts very
nicely, while also explaining the cases where a coherent whole does
occur as a side effect of a *soft* requirement of a biologically
effective system, not a hard constraint imposed by some impossible
mechanism.

If all this is just a long winded way of saying the following (quoted in
a previous message):

    No matter what we do, the visual system tries to find a single
    coherent conscious interpretation at any given moment.
        (Page 87 of "In the Theater of consciousness")

I.e. *tries* but doesn't necessarily *succeed* then Bernie Baars is
saying the same as I am, and I apologise for misreading it previously.
(Especially as I find a lot in common with other aspects of his
theories.)

CODA:

At the Elsinore conference last August, Doug Watt repeatedly claimed
that any good theory of consciousness must accommodate what he referred
to as "disorders of consciousness". I agree: disorders and variations
both across and within species and individuals need to be accounted for.


From the little I know about autism, it seems that one feature of some
autistic people involves visual processing remaining fragmented, without
the global coherence that comes from applying a high level (abstract)
concept.

This could be part of the explanation of the ability of some of them to
produce surprisingly accurate paintings and drawings: because their
processing of visual input stops at the kind of local 2-D structure
which an artist needs to replicate, instead of moving on to the
integrating 3-D structure which is then much harder to project back onto
the page.
    (Compare: Lorna Selfe,
    Nadia: a case of extraordinary drawing ability in an autistic child,
    London, Academic Press, 1977.)


Aaron


From Aaron Sloman Mon Dec 15 00:54:59 GMT 1997
To: PSYCHE-B@LISTSERV.UH.EDU
Subject: Re: More on consistency and consciousness: reasons for view B

Date: Wed, 10 Dec 1997 15:57:55 +0000
From: Peter Cariani <peter@EPL.MEEI.HARVARD.EDU>

Peter made some useful critical comments on my list of arguments for
view B --- the view that sometimes the contents of visual and other
types of consciousness are consistent and sometimes they are not. I.e.
arguments for the view that internal consistency is not an *absolute*
requirement for the contents of consciousness, but may often be a
consequence of mechanisms serving other requirements.

[AS]
> > I said there were three types of reasons for preferring B:
> >
> > 1. No known type of mechanism could implement the general consistency
> >    constraint.
> >
> > 2. Empirically there are counter examples.
> >
> > 3. There are more biologically plausible and biologically useful,
> >    mechanisms whose side-effects would include achieving consistency
> >    most of the time, without totally ruling out inconsistency.
>
[Peter wrote]
> While I generally agree with View B (as it seems most of us here do),
> that an internally consistent experiential world is usually,
> but not always constructed, ....

Interesting. I had formed the impression that it was a minority view.
Otherwise I'd not have gone to such lengths to defend it!

[PC]
> ...I think some
> of the proposed arguments against View A have problems.

I think you interpret View A more narrowly than the view I challenged,
namely that contents of consciousness (including at least conscious
beliefs and percepts) CANNOT be inconsistent because of some constraint
imposed by our brains.

If I misunderstood the claim, then my arguments are irrelevant. But
your arguments don't seem to be saying that I've got the claim wrong.

[AS]
> > 1. No known mechanisms could do it.
> > ...in the most general
> > case it is undecidable ... and even
> > decidable cases ... are combinatorially
> > explosive, and therefore intractable in general.

[PC]
> I've said this before, but Godel's incompleteness theorem
> (and the related Halting problem) only applies to notational
> systems that are unbounded.

The main fact I was (perhaps carelessly) referring to is that we are
able to think about the natural number system which is an unbounded
system: we can very easily formulate (and even believe) propositions
that we understand clearly and which, for all we know, may not be
decidable, e.g. "There is no largest pair of twin primes".

Contrast: "There is no largest prime number", which is also easy to
understand but was proved long ago by the ancients. Someone who has
never seen the proof might believe correctly that prime numbers must get
scarcer as you go along the number sequence and believe incorrectly that
eventually there comes a point beyond which they don't exist.

Although my comments are trite, they have implications for the power of
the mechanisms that would be required for a brain to be able to rule out
all inconsistent conscious belief contents of which humans are capable.

I don't think your remark about bounded systems has any bearing on this.

It's important to distinguish the boundedness of the brain (on which I
agree with you, obviously) from the boundedness of notational systems
used by brains. You and I use unbounded notational systems of various
kinds including natural and formal languages and most obviously the
everyday notation for numbers.

[[This is related to Chomsky's admittedly controversial distinction
between competence and performance (e.g. in Aspects of the Theory of
Syntax, circa 1963). I think Chomsky got this right and most of his
critics simply failed to understand what he was saying.

This is also loosely related to the fact that a computer can have the
(generative) capacity to construct symbols which just happen to be too
large to fit into its memory. By giving such a machine more memory,
without extending its basic abilities, you change its upper bound. The
*virtual machine* implemented in a computer may have bounds which are
larger than the limits actually supported by the implementation. In some
cases the implementation limits keep changing, e.g. when a process is
sharing physical memory with other processes whose requirements change
over time. Some virtual machines support indefinite precision rationals
and "bigintegers" which have no upper bound (E.g. common lisp).

However, they are typically implemented in systems which do have size
limits. E.g. most CPUs, unlike Turing machines, have an upper limit to
the amount of memory they can address: that follows from the requirement
for direct "random" access on the basis of *explicit* addresses held in
address registers of fixed size. Extending the addressing power of such
a machine requires not only extra memory but additional general
mechanisms, usually involving a mixture of hardware and software, which
would change the nature of the machine.

I have no idea whether brains use fixed size addressing mechanisms at
some important level. Whether they do or not, at another level we, like
the Common Lisp virtual machine, have the *potential* to construct
arbitrarily large numerals, e.g. in our case with the aid of external
memory. I don't know how many other animal brains have this feature, nor
how exactly it evolved. I have some speculations, but, that's a topic
for another occasion.]]

Anyhow, even if it is true that there is a biggest explicit numeral that
can fit into my brain (or any human brain, or even into the universe)
that does not imply any upper limit to the size of numeral about which I
can have a belief. E.g. I believe (and I expect you believe) that the
result of adding that numeral to itself is an even number which cannot
be explicitly encoded in my brain as a bit pattern.

For all these reasons, the boundedness of the brain proves nothing about
decidability of belief contents. Any adequate theory of how the brain
works must explain our undoubted ability to think about infinite sets of
various kinds -- discrete, continuous, linear, multi-dimensional,
tree-structured, network-structured, static, changing, etc.

Perceptual contents may be more limited! Why that's so is an interesting
topic for another occasion.

(We could easily get into a discussion of logical and semantic paradoxes
here, but I suspect the psyche-b list members would not welcome that.

A discussion of the biological advantages of having the abilities that
make possible thinking about infinite sets, and the developmental
processes that produce them in young children could be suitable topics
for another thread.)

[PC]
> If one has bounded string lengths, then the set of all operations on
  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> strings can be surveyed, and consistency can be assessed one way or the
> other.

That is perfectly correct, but if you think you can use this to prove
that arithmetic as you and I understand it is decidable, then I look
forward to seeing the proof, and, if possible, an answer to the question
whether there's a largest pair of twin primes!

(Of course there's a largest pair of twin primes that can be represented
in my brain. But that's irrelevant.)

I hope it's clear that there's an ambiguity in the underlined
antecedent: for it could refer to the syntactic properties of an engine,
i.e. it has a maximum memory capacity, or it could refer to the semantic
properties: i.e. there's a largest string it can refer to.

The latter does not follow from the former if you have a sufficiently
powerful language, including quantifiers.

Even with my poor, tired, finite brain I have no difficulty thinking
about the set of all even numbers, and noticing that only "half" of them
are divisible by four.

However, I agree with your implicit claim that many people misuse
Godel's theorem in philosophical discussions about minds and machines:
I've argued against people like Lucas and Penrose elsewhere, for
instance, including a long critical review of Penrose's Emperor's New
Mind in the AI Journal, 1992 Vol 56 (pp 355-396). I don't understand all
the issues I discuss and I may have got some details wrong: a of my
critique will be appearing soon in AIJ!

[PC]
> It's true that the number of alternatives increases
> combinatorically, but this is a matter of computational complexity
> rather than some inherent, absolute computability barrier.
> Ditto for the Halting problem if one has a finite length tape.

That's correct. The complexity/tractability argument is different from
the computability/decidability one.

Apologies if I did not make that clear. I was presupposing that the
difference was understood.

My claim was simply that *even* for decidable/computable cases
tractability can still raise its ugly head, and often does. There are
far cleverer mathematicians and computer scientists than I am who have
been trying with only very limited success to reduce the combinatorial
complexity of some quite important practical problems, e.g. deciding
whether a given number is prime.

Consider these numbers: 7 59 557 3343 10007 1000033 7777801 10000019
Someone who had made a mistake could easily believe that one or more of
these numbers was divisible by 131. To claim that human brains have some
sort of inconsistency detector which would always discover the
inconsistency in such a belief and prevent it consciously being held
would be very rash.

Actually I am coming round to the view that Baars really intended only
to rule out a very *simple* class of inconsistencies e.g. where the
contradiction is made explicit  because there are two percepts, two
sentences or whatever, whose syntactic form makes the inconsistency
trivially detectable.

I suspect he simply did not notice that what he wrote, if taken
literally, referred to a much larger class of inconsistencies which no
brain mechanisms could exclude.

That's an understandable mistake in this multidisciplinary morass.

The task of specifying precisely what claim he was trying to make
remains unfinished.

I don't think it's a simple matter to specify precisely, since as I
showed previously it's easy to ask questions which sound empirical but
actually are not because a positive answer is incoherent. (E.g. "Can
anyone see a cube and a sphere occupying the same region of space at the
same time?" The question is incoherent, and therefore not empirical.
Likewise Stan Klein's question whether it is possible to see my two
intersecting pencils at the same time in the same place without either
being "transparent" or "diaphanous".)

I suspect that there's no implementation-independent way of specifying
what sorts of inconsistencies the brain excludes. To see what I mean
consider a particular implementation which physically prevents certain
states, e.g. a bistable network allowing only group G1 of neurons all to
be active or group G2 of neurons, but not both groups. Then IF those
groups are used by the brain to represent particular semantic contents,
then those contents cannot be represented simultaneously. However, that
does not rule out group G1 and another group G3 being simultaneously
active even if that represents a semantic inconsistency.

In other words, certain mechanisms (e.g. low level sensory mechanisms)
may compile some semantic inconsistencies into physical
incompatibilities. Then THOSE semantic inconsistencies will be ruled
out, but not others, which have not (yet) been compiled into inhibitory
links.

Anyhow, until the precise type of inconsistency allegedly ruled out has
been specified, apparently empirical questions remain too unclear to be
capable of being settled by evidence.

Perhaps someone with more patience and tact than I have can finish the
task of clarification.

  From now on the discussion becomesmore obscure. I am not sure exactly
what Peter is arguing for or against, especially as we both reject
View A. So I don't know if anyone will find it worth reading on.
=======================================================================

Here goes anyway.

[PC]
> In general, the kinds of consistencies (or inconsistencies) that
> we observe regarding our own perceptions and concepts of the
> real-world do not involve huge numbers of properties and
> operations

NB: just because I say that issues about decidability or tractability
are relevant to *some* contents of consciousness (e.g. beliefs about
number theory or perception of complex Penrose/Escher pictures) it does
not follow that I am claiming that they are relevant to *all* such
contents.

I was arguing against a theory which as actually expressed was very
broad and covered many cases, without qualification. My
counter-arguments dealt differently with different cases, though I
probably did not make this clear enough.

[PC]
> ...(impossible figures are not very complex -- if they
> were it would take us much longer to recognize their global
> inconsistency).

Why do you say they are "not very complex" ??? As I suggested in a
previous message, they can be made as complex as desired. With patience,
you can create a Penrose triangle, square, pentagon, hexagon, etc...
Detecting the inconsistency gets harder and harder.

So I don't know what point you are making. I've always allowed that we
can detect *some* inconsistencies. I even allowed that where small
inference chains were sufficient, layered neural nets could do the
propagation required to find the inconsistency quickly.

My argument was only against a view that the brain can prevent *all*
inconsistencies which seems to imply the ability to detect all of them.

A separate argument is that even when inconsistencies are detected they
are not always eliminated (e.g. by suppression of one corner of the
Penrose triangle, or interpretation of the edges as curved.) That was
part of my argument 2 (the empirical argument) against view A. (Baars
notes the phenomenon, but for some reason doesn't regard it as a counter
example, which is why I now think he is using "inconsistent" in a very
restricted sense.)

[PC]
> This again is an argument for view B, limited
> consistency,

Yes

[PC]
> but it is at the same time another also
> argument against the relevance of computability.

No. It's irrelevant to the relevance of computability!

Just because *some* examples don't involve computability/decidability
issues, it doesn't follow that *none* do!

I was writing in a context in which the discussion (and the words in
Baar's books and papers) had already broadened the context beyond visual
perception.

Until someone convinces me that all possible conscious human beliefs
about numbers are decidable, I'll stick by my arguments against view A
in its unrestricted form. Even then I'll need convincing that the
decision task is tractable given available mechanisms in the brain.

[AS]
> > Even if the brain does not use propositional or predicate logic it seems
> > (in the case of humans) to have combinatorial capabilities both as
> > regards the variety of 2-D images that can be "parsed" and the variety
> > of 3-D percepts that can be formed. Motion merely adds to the
> > combinatorial complexity.

[PC]
> There are analog mechanisms that can handle the
> simultaneous satisfaction of huge numbers of constraints.

Agreed. Using soap bubbles stretched over a wire frame to find the
minimum stress shape for a roof over an irregular building is an
example.

Another nice example avoiding combinatorial search is the use of a
network made of bits of string to find the shortest route from A to B in
a network of roads: just build the network in the obvious way, then pull
the knots representing A and B apart as far as possible. The tautest
bits of string will then determine the shortest route. That's a nice
fast parallel analog computation that may be quite difficult to
implement using discrete symbol manipulation (though I expect it's
possible using discrete token-passing mechanisms in a concurrent
computer network.)

However, from the fact that there are lots and lots of cases where
special purpose devices can avoid combinatorial explosions you cannot
conclude that there's anything wrong with my argument. What you need to
show is that

(a) ALL decidable problems can be solved that way without explicit
    combinatorial searching (perhaps using new kinds of quantum gravity
    engines???)
or
(b) All the possible contents of consciousness (percepts, beliefs,
    intentions, plans...?) are constrained to lie in the class for which
    such analog mechanisms are available.

Either (a) or (b) would be a very interesting claim, and I would like to
see the arguments. Till then I remain sceptical.

[PC]
> Any large scale system that can come to
> equilibrium is performing something like this. The problem is that when
> we deal with information, we first think of encoding it symbolically
                            ^^^^^^^^^
> and computing on symbol strings,

"we first think"? Who exactly?
I for one have spent many years thinking and writing about other
options.

> ...but in general,
> this is not how biological evolution tends to solve these problems --
> things are much more like analog relaxation processes.

"...in general..." "...tends to..." ???

I expect there are *specific* classes of problems for which this is
*always* done. (In fact my previous discussion of winner-takes-all
networks was meant to be an example).

But that still leaves open the possibility that another type of engine
has also evolved in human brains which, for certain purposes (e.g.
planning), uses discrete, inherently sequential, mechanisms.

E.g. unless I've missed something, not all plans can be constructed from
scratch via continuous deformations of some initial physical
configuration. E.g. plans for proving mathematical or logical theorems
don't inhabit a continuous space. Likewise (I think) plans for building
a house.

I conjecture that some of the problems that require discrete, largely
sequential, resource-limited mechanisms depend on the use of memory that
can answer questions like: "if the situation part way through is like
this then what will happen if I act like that?". I don't see how an
associative memory that can answer questions like that can avoid working
in discrete question-answer steps (even if it is implemented using
highly parallel continuous low level mechanisms).

Perhaps I am ignorant of some important types of problem solving
mechanisms or associative memory mechanisms.

Maybe you know how to design a house-building planner that gets round
such constraints: I've been thinking about such issues ever since I
attacked logicist AI at a conference in 1971, but I still think that
different sorts of mechanisms are needed for different sorts of
problems. Some of them are approximately like AI symbol manipulators,
while others use different mechanisms.

If that's what you are saying, then we are in complete agreement: and we
need to find out how the different sorts of (virtual) machines are
implemented in brains.

[PC]
> Motion is only a problem if you are trying to encode things like images
> into pixel arrays.

No. I was not thinking of pixel arrays.

I was making the trivial point that sequences of complex structures at
any level of description typically have more complexity than individual
structures at that level. This is relevant both to planning processes
and perception of moving objects, e.g. assembly of a machine. (Did you
ever play with meccano? It's one thing to see a structure built from a
kit. It's another to find a sequence of operations which will produce
that structure.)

Your comments are concerned only with the *disambiguation* sometimes
facilitated by motion, which I don't dispute, though it's not easy to
design mechanisms with these properties:

> .... if our perceptual systems are set up
> to register ongoing and stable patterns of spatial cross-correlations
> (i.e. sets of spatial relations), then motion actually helps simplify
> the visual world, by segmenting it into stable "objects" (whose
> internal correlations are stable). Relations between moving objects
> are in constant flux, but relations within objects are constant.

Yes, and I hope you are trying to implement these ideas in a working
system. The results could be interesting.

However, detecting constant relations between objects of various sorts
moving with various kinds of motion (rotation, translation, flexing,
etc) is easier said than done.

[PC]
> When you have invariances like this, (the formation of stable objects),
> then a huge reduction in the complexity of the representation is
> achieved (data reduction).

Yes. But that's just a special case of the general point I made in a
previous message about the brain's ability to use schemata (Kant and
Hebb. I mistakenly wrote "Bartlett").

Even though complexity gets reduced by the use of subsumptive schemata
(abstraction), the combinatoric issues remain. The problems are reduced
but not eliminated. This does not contradict anything I wrote.

[AS]
> > ...Finding a
> > non-exponentially explosive (in space or time) way to check consistency
> > of an arbitrary propositional expression could bring fame and fortune
> > and answer one of the open questions in the theory of computation.

[PC]
> I actually believe that there might be ways to do this using analog
> electrical circuits that either settle down into a stable state or
> alternatively, oscillate, blow up or go chaotic.

I have an open mind. If you can do it for propositional calculus I'll be
happy to applaud. Maybe you'll get a Nobel prize?
                                                           N
Of course one can do it in parallel trivially by building 2 networks
for dealing with a formula containing N propositions, and trying each
combination of values in parallel.

This produces a space explosion instead of a time explosion.

I look forward to hearing the results of experiments with your proposed
circuits for detecting consistency.

[PC]
> But if we are simply discussing logical
> (syntactic) coherence, as in the consistency of a finite formal system,
> then there are no external semantics, so this isn't a problem.

Having a semantics doesn't help. I can formulate N propositions about
things in the world (houses, people, cities, fields, rivers, etc.) and
combine them in various different ways to make statements whose
inconsistency is not obvious, and requires combinatorial checking.

The confusion of logic with syntax is a deep mistake fuelled by the
development of syntactic mechanisms for addressing logical problems. The
mistake is widely encouraged by teachers who introduce logic solely via
syntactic rules (e.g. natural deduction).

It would likewise be a mistake to claim that all the mathematics used by
physicists is simply concerned with syntax because much of the power of
mathematics arises out of new forms of syntax and associated algorithms.
But that's a topic for another day.

[PC]
> ...
> There exist good strategies for designing computer
> programs that don't loop.

But only in special cases. E.g. where loops are known at compile time to
be bounded they can be "unwound" at compile time into non-looping
conditional constructs. In other cases it is possible to prove by
induction on the forms of inputs that an algorithm always terminates.
But not in all cases.

> It might be hard to prove the consistency of
> arbitrary computer programs, but it might not be so difficult to do
> so if there are strong programming structures imposed on things (this
> was part of the thrust of structured programming).

As long as the main questions of complexity theory remain open, we
should keep an open mind. Certainly the task of identifying important
*classes* of sub-problems with reduced complexity is worth while. What
it has to do with how brains work remains to be seen. Reality is full of
surprises.

If you are saying that there exist classes of problems for which brains
have developed fast complexity-defeating strategies, then I agree. Human
vision is a remarkable example.

This is not relevant to my arguments against an *unqualified* general
claim about a consistency constraint implemented in our brains.

[PC]
> Again, the big
> problem is semantic consistency, not syntactic consistency.

As indicated above: that distinction is not relevant.

Consistency is an inherently semantic notion (as someone else has
already remarked). Some semantic notions can be compiled into or mapped
onto syntactic ones. That could also be true of any special class of
complexity-reducing semantic relations. (I.e. what AI researchers have
often called "heuristics".)

[AS]
> > Some people may think that parallelism in neural nets can defeat the
> > combinatorics, but although I have not thought it through, I suspect
> > there is probably a theorem about limitations of perceptron-like neural
> > nets on this sort of task, analogous to the Minsky/Papert results
> > regarding parity and connectedness: such essentially global properties
> > require serial mechanisms (or enough perceptron layers to simulate
> > serial processing).
[PC]
> I'm a little surprised that these kinds of arguments are still being
> trotted out, even today.

Mathematical truth has no history.

[PC]
> Single-layer perceptrons are impoverished devices, even as neural
> networks go, so it was (and is still) incorrect to
> indiscriminately apply those
> arguments to all neural networks.

I suspect you didn't read what I wrote even though you quoted it. Look
at the parenthetical bit at the end. Neither my comment, nor, as far as
I remember, the Minsky&Papert book, was restricted to SINGLE layer
perceptrons.

[PC]
> Nobody today thinks the brain is
> a big single layer perceptron. I'm not sure anyone ever did, literally.

I cannot understand what you are getting at since I was not talking
about single layer perceptrons.

> The second general problem with these arguments

I suspect you are thinking of arguments you've heard from other people,
and misidentified with mine. (Likewise, people often mis-quote Minsky
and Papert.)

> ...is that,
> biologically- or psychologically-speaking,
> how important is the computation of parity?

Any counter-example, whether biologically or psychological important or
not, can be used to refute an unqualified generalisation.

Had Baars claimed that inconsistency is ruled out only in *specific*,
biologically relevant, contexts the argument would have been different.

[PC]
> .....Perceptually,
> it's obvious that we don't have the ability to pre-attentively
> discriminate between 55 and 56 objects ... It's a very
> artificial and contrived argument.

Against what?

I think you are rehearsing points analogous to my second type of
argument against view B, namely: Empirically there are counter examples.

View A:
    Brains can prevent any sort of inconsistent contents of
    consciousness
Sloman argument 1:
    Mechanisms for doing this in general don't exist and even when
    they do there are intractable sub-classes.
Cariani:
    But empirically we can't do it.

I don't see the relevance of your comment, given that I had already
acknowledged the empirical limitations.

[PC]
> ...I think what we want is
> some kind of broadcast distribution of information that is then
> operated on by far flung sets of neural assemblies that emit their
> own patterns upon activation, with these patterns reinforcing or
> mutually inhibiting each other, with their interactions activating
> still other assemblies.

I would not dispute that *sometimes* space (parallelism) can be traded
for time. I was merely objecting to a general claim apparently made
without any consideration of what sorts of mechanisms might have the
properties required, as if it were a purely empirical issue to be
settled by looking for evidence of what we can and cannot experience.

Like you, I suggested that a subset of cases can be handled by
multi-stable networks using mutual excitation and inhibition
implementing winner-takes-all strategies. (The idea is at least 20 years
old, probably a lot older)

What's needed now is an analysis of various types of mechanisms required
for different classes of biologically relevant tasks, and architectures
in which they can be combined effectively.

[PC]
> .... like those line drawings of scenes in children's books ...
> ...It's not that we go looking at
> each object one by one and deciding whether it fits -- the mismatch
> pops out.

Sometimes. Not always.

[AS]
> > Vision has to work fast, at least if you are a bird, squirrel, chimp,
> > tennis player, car driver, etc. .....

[PC]
> It could be that sensory systems do a rough analysis early on and
> elaborate on it as more information comes in.

This is the theory of perception I've always supported: with different
levels of analysis proceeding in parallel with mutual excitation and
inhibition between levels implementing a combination of top-down and
bottom-up processing, though not only using numerical values.

But in itself this design does not deal with the sorts of inference
chains required to detect inconsistency in Penrose/Escher figures,
especially those where multi-step high level inferences are needed.

> ...If you take a random
> sequence of clicks 5-10 seconds long ... and repeat the sequence
> ....and finally one
> gets the pattern. This can take 30-60 seconds

And longer if the initial sequence of clicks is longer. So?

Cheers.
Aaron
===
Aaron Sloman, ( http://www.cs.bham.ac.uk/~axs/ )
School of Computer Science, The University of Birmingham, B15 2TT, UK
EMAIL   A.Sloman@cs.bham.ac.uk
Phone: +44-121-414-4775 (Sec 3711)       Fax:   +44-121-414-4281


From Aaron Sloman Thu Dec 18 12:03:41 GMT 1997
To: PSYCHE-B@LISTSERV.UH.EDU
Subject: Motion after effect (was: Baars on Binocular Rivalry)

    Date: Wed, 17 Dec 1997 12:30:37 -0500
    From: Arnold Trehub <trehub@PSYCH.UMASS.EDU>

I think there's an interesting disagreement between Arnold and myself.
At first I thought it might just be terminology. But I now think there's
something deeper.

[AS]
> > Let's make sure we are talking about the same thing. I view a text
> > window with the text smothly scrolling up. Then when it stops I can see
> > downward motion even though I see no text or other visual object
> > actually changing its location either relative to the window frame or
> > relative to the direction in which I am looking or relative to
> > anything else. I.e. there's both visible motion and nothing moving
> > visibly, i.e. no change of absolute or relative location.

[AT]
> It appears that we are *not* talking about the same thing. I thought
> your claim was that in motion after-effects we see *simultaneously*
> motion and no motion in exactly the same location.

That was and remains my claim. It looks incoherent. Below I'll explain
why it isn't, by sketching a model of visual perception which could
produce exactly that effect, and which could have evolved for good
biological reasons.

[AT]
> The problem, I
> think, is in your equating the immediate perception of *motion*
> with a subsequent analytic judgment of *change of absolute or relative
> location*.

There's nothing subsequent about it! At one and the same time I have the
experience which I can only describe as "There's vertical motion here"
and the experience which I can only accurately describe as: "All objects
are experienced with unchanging relative (to one another) and absolute
(i.e. relative to my visual field) locations".

This is exactly like what happens in the other example I described when
flying in a plane that banks to left or right while I stare at the
bulkhead, except that in this case the motion experienced visually is a
rotation to left or right not a vertical translation.

Note that I am not the only person to claim to have these experiences.

This is one of many ways in which the theatre metaphor (like all
metaphors taken from our environment) breaks down as a model of mind.
You cannot have a bunch of actors on a stage all standing still while
motion occurs.

But consider the possibility that the contents of consciousness are not
so much like the contents of a stage but more like the contents of a
(globally accessible) database with information contents linked (another
metaphor requiring unpacking) to a host of different capabilities some
of which are activated at the time while others are ready to be
activated if needed. [I elaborate this a little below]

In that case it's perfectly possible for that "global database" to
include some representation of motion with a certain magnitude and
direction as a global feature of the current experience or some large
chunk of it, even though all the individual features and objects are
recorded as having unchanging locations.

(I.e. abstract information stores can be inconsistent in ways that are
not possible for, e.g. paintings and photographs.)

Our awareness of our own state of visual awareness may be partly
delusory in that we *think* our visual experience has all the qualities
of a physical picture, whereas in fact it is something much more
abstract and the spatial features result from how the contents are
related to the structure of a spatially organised intermediate database
derived from the current visual stimulation.

Examples of more abstract databases could be histogram-like records of
global amounts of various features, e.g. greenness, redness,
verticality, circularity, and of course opticl flow.

The whole point of an (illusory) after-effect is that something can get
into one of these databases via an abnormal process.

In the normal process current sensory input leads to a global record of
motion in a region via data-driven processes of analysis and
interpretation (e.g. continual change of location of texture fragments
and edges leads to global motion features being detected, presumably
via accumulation of information in something analogous to a histogram).

In the abnormal process, mechanisms involving fatigue, habituation, some
kind of "rebound" or whatever, probably relatively high up in the chain
of processing, can lead to an *illusory* record of motion of a certain
kind, simply because sustained data-driven recording of motion of the
opposite kind has just ceased.

I.e. global motion is recorded (and therefore experienced) while there's
no record of any motion of any parts (and therefore none is
experienced).

Someone in the grip of an oversimple theory of consciousness (e.g.
simplified physical metaphors of searchlights and actors on stages,
etc.) will find it difficult to comprehend how it is possible for the
current contents of consciousness to include a direct contradiction.

If we replace such simple metaphors with more apt and subtle metaphors
involving interconnected information stores of various kinds, the
difficulty disappears (though new difficulties arise regarding how the
detailed mechanisms actually work, how the semantics work, what the
differences are between these information stores and the unconscious
ones (e.g. those which continue to function in blindsight, etc).)

A possible partial mechanism (which has a great deal in common with much
of what Baars conjectures) could involve something like the following.

We know that a common feature of biological evolution is replication of
a pre-existing device, after which the two copies develop different
functions and possibly different structures. Vertebrae are a spectacular
example.

Suppose that at an early evolutionary stage organisms with multiple
sensors (either of the same kind, e.g. retinal cells, or of different
kinds, optical, auditory, tactile transducers) required a type of
coordinated processing of sensory data that involved a highly parallel
network of units receiving information from different transducers, which
could then be compared with one another synchronously and diachonically
so that huge amounts of local data were processed locally in order to
detect things like spatial contrast, local temporal
change etc. In the case of sound this might include an array of
processors simultaneously dealing with different frequencies.

The outputs of these processing units could be sent all over the place
to things that need them (e.g. posture control mechanisms in the case of
both vision and other sensors). As organisms evolved the ability to
detect, learn about and use higher level generalisations, the perceptual
mechanisms may have developed cascades of more and more abstract
detectors and recognisers and histogramming devices, used for a variety
of purposes including online control and decision making and also
feeding various longer term information stores, and rule-induction
mechanisms.

Now consider the problem of relating the outputs of all the different
kinds of more or less abstract detectors of features, objects,
relationships, etc. in a coherent usable way. This could be particularly
important for vision, where the retinal input is changing with enormous
rapidity (e.g. due to saccades as well as bodily motion) and yet the
perceived scene is largely static or changes far more slowly.

The problem of coping with both the spatially organised rapidly changing
visual input and the spatially organised relatively static perceived
scene could be achieved by making a copy of the mechanisms that had
originally evolved for low level processing of the sensory input and
modifying them so that they could instead be fed by the outputs of the
various high level recognisers. E.g. in the case of vision we'd then
have the following "pipeline" of types of information processors. (I'll
explain the motion compensation bit later.)

    Optic array
        -> lens
            -> retina
                -> feature detectors
                 |   -> high level recognisers/classifiers etc
                 |       -> spatially organised (e.g. 2-D?) map
                 |                   ^
                 V                   ^
                 (motion compensation)

Of course ideas like this are constantly being re-invented (remember
"cognitive maps", Kosslyn's graphical models, some of Marr's work on
2.5D sketches, and all sorts of models and theories concerned with our
ability to use spatial capabilities in understanding sentences, solving
abstract mathematical problems etc.).

But often the "last" stage is confused with one of the earlier stages,
i.e. as if the outputs of higher level recognisers are projected *back*
into the low level arrays closely linked to the retina. (I've fallen
into that confusion myself, in the past.)

There may well be lots of such back-projections for all sorts of
different purposes including top-down context-sensitive tuning of low
level feature detectors. But projecting information about relatively
static objects in a scene back into a constantly changing retina-driven
visual array would be of little use.

That's why I started by talking about making a COPY of the low level
mechanism and then giving it a different function.

If "here be motion" records are fed into that spatial map they would
generally be the result of location changes and optical flow patterns
recorded at lower levels. But mechanisms involved in illusory
after-effects (and other illusions, or proprioceptive detectors) could
also feed such information in, even when it is inconsistent with other
incoming information about the locations of features of objects.

One of the non-trivial changes in the function of the new information
array would have to be its ability to receive information conceptually
related to abstract long term knowledge, unlike the original information
array which merely records things like intensity, velocity, contrast,
colour, etc.

This could account for the sorts of experiences we describe by saying
that someone *looks* happy, for instance, or a bridge *looks* rickety.
There's no way happiness or ricketyness could be outputs of purely
data-driven low level feature detectors analysing an optic array. For
happiness and ricketyness are not features of optical patterns, even
though we may be able to learn (or may be innately programmed) to
associate them with such features.

On this model, far from consistency being a built in constraint on the
integrated information store (how could it be?) one might expect
occasional INconsistencies because of the varieties of routes by which
information is accumulated.

Of course, there might be some sorts of constraints, possibly operating
in lower level analysis/recognition/interpretation mechanisms, which
select between mutually inconsistent collections of items to feed into
this "map" (e.g. using the winner-takes-all mutual excitation/inhibition
networks which Peter Cariani and I discussed previously.)

That would rule out certain sorts of inconsistencies (in the manner
assumed by Bernie Baars, perhaps), but not all. In particular it would
not rule out the inconsistency experienced in a Penrose triangle or
devil's pitchfork, or the motion/no-motion inconsistency I've described.

Why did I add the motion compensated route by which low level
information could get into the "map"?

One reason is that it could be useful to have low level visual
information in registration with higher level more abstract descriptions
for a variety of practical purposes to do with planning, control of
actions, etc. I am sure this will have to be done for robots.

There's another reason to do with stereo vision.

I've often wondered how the binocular disparity computation can be done
when I'm looking a difficult and subtle random dot stereogram which
takes up to a minute to fuse - e.g. the spiral staircase stereogram or
the saddle stereogram.

While looking at the pictures my eyes are constantly moving, so if the
disparity detectors were using direct retinal outputs they would have an
enormously difficult job searching for a way of working out
different binocular disparities across the region. However if there's
some other place where a relatively unchanging "map" is being built for
each eye, where the outputs of feature detectors can be stored after
some kind of relocation to compensate for saccadic motion and perhaps
vergence changes, then IF the disparity detection (a *hard* problem in
general) is done on those maps the complexity will be much reduced.

For this reason, and other reasons, I've added to the diagram a route
through a "motion compensation" mechanism from the outputs of various
low level feature detectors to the postulated spatially structured
database of relatively unchanging information. But the implication of
the remarks about stereo is that there would be at least three of the
relatively static maps: one for each eye used as input to a binocular
fusion mechanism and one integrating all the various information derived
in various from the current visual scene.

How exactly that spatial integration is done remains a mystery. Simple
2-D arrays won't work for reasons that would take too long to spell out
here. I suspect an entirely new kind of information processing engine is
waiting to be discovered or invented by some genius.

A final comment:
If we found a portion of the brain implementing the "unchanging map", we
might not realise that it was peforming that function if the experiments
were being done on animals (or people) with clamped heads and static
eye-balls. For in those circumstances its use to record not the current
retinal input but the motion-compensated outputs would go unnoticed.
Has that happened in studies of V1?

Apologies if these amateur speculations and questions are wildly off the
mark and refuted by well established facts.

Aaron