Posted in 13 parts  20 Jul 1996
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Was Re: Non-uniqueness of external language)

    Control systems and philosophy of mind (Part 1)

I cannot keep up with all the people who contribute to this news group,
including andersw+@pitt.edu (Anders N Weinstein) who wrote on 13th July
in response to a long posting of mine a few days earlier, and has
written many things since then, which I have not had time to read.
Nevertheless I'll try to reply the article of 13th July, because it
involves many important themes.

Anders had been trying in a Wittgensteinian vein to link mental state
descriptions with social norms and ascriptions of rationality (which is
also a feature of Dennett's intentional stance, and Newell's knowledge
level).

I am grateful to him for helping me (or forcing me) to try to think more
clearly about all these topics. It also helps me understand better what
I think is so misguided about the linguistic, anthropocentric, type of
philosophy he has been defending.

My full response to him, of which this is the first part, is very long.
I shall install it as a single file at

    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

I shall post it in the form of a succession of instalments (which may or
may not be of general interest to readers of this news group other than
Anders!) I wrote the various parts at different times during the past
week, so I apologise in advance for inconsistency, repetition, etc.

In future messages I'll use the subject line:

    Control systems and philosophy of mind (Part N)

where N > 1. (This is part 1. There are 13 parts altogether.)

Because I have not been able to keep up with all the postings that
followed on from Anders' and my interchange, it's likely that some of
what I have written is now out of date either because others have said
it or because Anders has changed his mind. (How often to posters to this
group ever change their minds because they meet counter argumetns?)

I'll start with the point on which Anders ended, because it may betray a
deep diversion of views that explains some of the things Anders says in
other places.

-----------------------------------------------------------------------

I had been trying to point out that the realm of (normal adult western)
human minds (which I previously called the NPR - normal personal realm)
was part of a larger space of possibilities which also included
infants, brain damaged adults, people with senile dementia, and many
other animals as well as possible robots of the future.

I argued that any worthwhile philosophy of mind should:

(a) accommodate a much larger space than just the NPR, since other
animals, abnormal humans, and possibly robots of the future have mental
states and processes (e.g. perceiving, learning, deciding, suffering,
enjoying, noticing, being curious or angry or surprise, and many more)

(b) try to show how such things were possible (by relating their
existence and functionality to possible layers of implementation
including virtual machine architectures)

(c) allow the possibility that many of the concepts of mentality are
actually specific to particular architectures within this space of
possibilities so that different families of concepts (more or less close
to our concepts like "believe", "perceive", "desire", "intend", "learn",
"understand", "enjoy" etc.) might be shown to be generated by different
architectures.

In contrast, Anders
(i)   wants to restrict mentality entirely to the NPR,

(ii)  wants to restrict philosophy of mind to the discussion of the NPR

(iii) believes that mentality of the NPR kind is produced by a culture
    which includes a public language, and NO other explanation of how
    (normal) mental states can occur can possibly be relevant, although
    (like Ryle) he admits that if something abnormal happens then other
    explanations (e.g. referring to brain damage) might be relevant.

At times he writes as if he thinks other animals can have mental states.
However, one of his answers to my comments about other animals was
revealing

[AW]
> Perhaps when non-linguistic animals discover germs and cure diseases,
> send people to the moon, build computer networks, and so on, I will
> recant.

So you think the Orang utang's ability to leap through branches, or to
weave a collection of branches into a nest for the night with one hand
while cradling an infant with the other is too trivial to involve
mentality?

Likewise a magpie's ability to build a nest by assembling quite long
twigs into a tangled tree-top structure with enough rigidity to function
as a nest. (Try doing it yourself!).

Likewise a sheepdog's ability to get a group of sheep through a gate.

You seem, in this remark, though not all others, to be interested only
in those manifestations of intelligence that use high technology and
advanced science.

If having such capabilities is your definition of having a mental state,
then we are miles apart, and probably further discussion is a waste of
time. But on the assumption that the above is an aberration I'll
continue.

As a sample of Anders' objection to my attempt to embed the focus of his
interests in a broader scheme of things, here is the closing paragraph
of his message to me:

[AW]
> Indeed no
> non-linguistic animal could ever analyze the workings of its own
> control system as you want so much to do.

Note that I never claimed they could. He then goes on:

[AW]
> ...It is very surprising to me
> that someone who prizes science so highly should want to dismiss the
> importance of the most distinctive thing that make theoretical science
> possible.

I find this remark quite extraordinary, for I have never "dismissed the
importance" of language or science, although I did accuse him (and 1950s
linguistic philosophers) of adopting an anthropocentric view of mind
and I did claim that animals without language could have mental states
and processes, such as those listed above.

I don't see the point of labouring the obvious in this context:
of course human science, engineering, art, etc. have achieved remarkable
things, etc. etc. Of course.

And of course they do depend on human language.

But that in itself says nothing about whether
    (a) other animals without a public language have mental states and
    processes, and

    (b) whether the achievements that depend on a public language (and a
    culture) also depend on our information processing architecture.

Just because culture and language are part of the explanation it does
not follow that they are the whole explanation.

Neither do I see the relevance of my or Ander's evaluations (i.e. the
things we *prize* or the things we find *distinctive* or *important*) in
our philosophical or scientific work.

However, I AM concerned with things that are difficult to explain.

For instance, I think there are more things about human and animal
*visual* abilities that are mysterious than there are about human
scientific and mathematical abilities (contrary to what Penrose claims).

I say this because I have worked on vision, and it looks to me as if
none of the available explanatory models can account for, e.g. the
visual capabilities of a monkey or a magpie, let alone a human. (I've
outlined some of the reasons in a paper on a Gibsonian approach to
designing visual systems, in JETAI in 1989, available in our Birmingham
Cognition and Affect ftp directory:
    ftp://ftp.cs.bham.ac.uk/pub/groups/cog_affect/
in the file
    Aaron.Sloman_vision.design.ps.Z

By contrast the computational modelling and replication of many
*linguistic* capabilities seems to be much closer to our grasp (though
perhaps that's because I have not looked closely enough at alleged
achievements).

It may be that the above extract reveals something important about
Anders. Perhaps he is too obsessive about wanting to treat human beings
as *special*. If that's the case it may be blinding him to all sorts of
points that he seems to find difficult in what I've written, which
surprised me.

Could this be why he is so opposed to philosophical work that surveys a
broader scene than NPR. I don't know.

I do know that there is a long history of opposition to advances in our
understanding, where much of the opposition is based on the desire to
put human beings at the centre or top of something or other: the solar
system, the animal kingdom, etc. This sort of opposition also arises in
connection with the suggestion that there may be no deep division
between people and possible machines. (Weizenbaum, the original creator
of Eliza, is a particularly interesting example of this.)

Anders does not subscribe to all of these oppositions, so I find this
apparent obsession about human language and mentality surprising. While
I am not keen on ad hominem remarks I mention this possibility so that
he can show where I have got him wrong, if I have.

-----------------------------------------------------------------------

I'll include here another paragraph from the middle of the original
message, which seems to me to be in the same vein.

[AW]
> I wonder what motivates your seeming blindness to the
> nearly infinite power that only language makes possible, for example
> design stance science itself?

Why only *nearly* infinite?

We think about infinite sets and our linguistic capabilities do seem to
display infinite generative power as Chomsky (and others) have remarked,
e.g. because of the compositionality inherent in them. (Of course we
have performance limits, but they are easily explained.)

But what's so mysterious or special about this infinite power?

Once an information processing system has an architecture which supports
iteration or recursion in the creation of structures, infinite power is
no big deal.

You can't claim that the "infinite" human capability comes from language
because the infinite generative power of a language would not be
accessible to a machine without an appropriate architecture. (I assume
that we both discount the "inessential" limitations due to speed, memory
size and limited life spans, which make the capabilities finite in
practice).

Your emphasis on language, culture and norms as explaining everything is
like trying to explain the ability of an airliner to fly by talking only
about the flight controllers and pilots and the rules they obey.

I would not dispute that linguistic capabilities and cultural influences
give us a lot that other animals don't have. Who would dispute something
so obvious?


What I regard as a gross distortion of reality and an excessive
restriction of philosophy of mind, is the refusal to acknowledge ANY
continuity or overlap between human perception and chimp perception,
between human learning and chimp learning, between human pains or
enjoyment and chimp pains learning and enjoying (to choose but a tiny
subset of cases - the full range of which would include very young
children, people with brain damage, etc. etc.)

Or do you want to exclude perception, learning, pains and enjoyment from
the realm of the mental?

If you include them then you are obliged to allow the analysis of mental
concepts to go beyond what is normal for linguistically competent
humans. (Beyond the NPR)

I'll return to the issue of animal mentality near the end of Part 2.

I'll come back later to the question why information processing
architectures (not brain mechanisms) and the design stance are relevant
and important for understanding the full range of possibilities in this
larger realm of mentality (and therefore also the special cases covered
by NPR).

Part 2 will follow shortly. There are 13 parts altogether, of varying
importance. Part 2 is particularly important because I think it helps to
clarify the class of control systems I was talking about which I think
Anders had not grasped.

Because of the unreliability of propagation via net news, if someone
else wishes to archive this discussion in a Web directory that could
be useful, reducing reliance on trans-oceanic network links.

=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 2)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part  of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

We now get an interesting turn in the discussion. Anders tries to show
that for certain classes of (simple) information processing systems
descriptions of their capabilities are tied to their high level
behavioural capabilities and dispositions. Moreover I agree with him,
for simple cases, though not when the architecture is more complex.

His example is a simple calculator. On this occasion I quote the full
text. It is probably the most persuasive thing he has written in this
whole interaction, because it is an example that actually suits his
analysis. It is useful because it enables me to describe ways in which a
control system might be different and not suit his analysis.

[AW]
> Let me try another example. Suppose I have a simple calculator program.
> I type "3 + 5 * 2" and it comes back against my expectations with "16".
> I type "4 * 2 + 5" and it comes back with "28". After further
> experimentation I form the hypothesis that although it is otherwise
> a normal calculator, in contrast to the usual conventions,
>
>   H: it gives "+" higher precedence than "*".
>
> Assume this hypothesis is borne out in all future interactions.
> A possible consequence of this is:
>
>   C: it represents the string "3 + 5 * 2" AS "(3 + 5) * 2".
>
> If I wanted, I could even phrase C in terms of "seeing
> (hearing? reading?) as".
>
> Now I want to distinguish two interpretations of these statements.
> According to one, call it the dispositional sense of representing-as, if
> H predicts its answers in every case, then *nothing* you could
> possibly discover about the insides of the calculator could disconfirm
> it. What it is to give "+" higher precedence just is to evaluate the
> expressions that way. In this sense, the statements are behavioral ones
> and not hypotheses about what's going on inside the calculator AT ALL.
>
> But there is of course another set of questions one can ask, questions
> about the structures inside the calculator that make this possible.
> These give a different question, such as C': Is there a
> representation of the string as "(3 + 5) * 2" functioning inside it?
> Even here, it is not so clear to me what this means.  The calculator
> may evaluate opportunistically, replacing nodes with values as soon as it
> is possible to do so, so there is never a complete representation of
> the input string. In this case, I think one ought to say that C' is
> literally false, there never is a data structure that represents the
> string thus. If you say there is a virtual machine that represents
> the string thus, it seems to me you run the risk of
> of making the control system claim C' degenerate into the merely
> behavioral claim C.
>
> It is not so clear what it would mean for H' to be false. One might
> say:  perhaps it has two modules, a conventional parser in which * has
> higher precedence, and an evaluator which evaluates parse trees
> non-standardly so as to give the answers we observe.

In this move Anders at last shows a grasp of the importance of a
control system composed of multiple interacting modules.

[AW]
> .But then we face
> the problem -- why should we say the output of the parser means one
> tree rather than another, if that is not the structure that matters to
> the evaluator?
>
> Anyway, I hope this makes clear what I am concerned about. I am
> concerned about the confusion of the two senses of statements like H
> and C, the behavioral sense and the control system sense. My view is
> that intentional states are more like the unprimed, behavioral claims
> which are not claims about the control system at all. It is to the
> second sort of question that I direct the idea that a probe or
> instrument might be needed for confirmation, since those questions are
> ultimately about physical structures inside the device -- even if they
> are high level -- while the former questions simply are not.
>
> BTW, I hope you can see also why one might say that the calculator
> might be filled with straw as far as the truth of behavioral H and C
> are concerned.

I'll respond by listing points of agreement and disagreement.

1. I agree that there are some relatively simple high level virtual
machines that consist of a single module whose states and processes are
totally defined by relationships between its input and its output. Any
other facts about it will be facts about how it is implemented.

2. I do not agree that all those implementation facts are "ultimately
about physical structures" since there could be several layers of
implementation and at present I do not see any reason to believe that a
physical examination could show how many intermediate virtual machines
there are nor what structures exist within those intermediate virtual
machines. I think a designer of the system may know what's going on,
but, as I said previously, unless the system has been specially
instrumented (e.g. by tracing commands in the source code) to reveal its
internal operation at various levels there may be no way of ever finding
out exactly how it works without having access to the design and
implementation process.

3. If the claim about being filled with straw is another way of saying
that the truth conditions for the high level functional specification
carry NO implications about the particular internal mechanisms used then
I agree.

4, If the claim about straw is meant to say that the high level
functionality could emerge out of nothing, without having any physical
basis, then I disagree, for the high level requirements will *constrain*
possible physical bases.

For example, if the high level specification requires there to be at
least N distinguishable enduring states of the system (states in which
the relationships between input and output are different) then no
physical system that is incapable of having N significantly different
enduring states could be an implementation.

There are more subtle requirements about the ability to modify the
states and to access stored information for a variety of uses. These
additional requirements could rule out machines filled only with straw,
or water, or a swarm of bees, and many other possibilities.

Moreover, points like this are directly relevant to that part of
philosophy of mind which asks about the relationship between mind and
matter, and the requirements for a material world to be capable of
supporting mental states and processes as we know them. (I suspect
Wittgenstein just wasn't thinking about these issues when he made his
comment and he has thereby confused lots of readers.)

5. Your comments on the calculator, at the point where you say

> ...perhaps it has two modules, a conventional parser in which * has
> higher precedence, and an evaluator which evaluates parse trees
> non-standardly ....

shows me that you do grasp the possible implications of a complex
architecture, but you have not thought them through fully.

You clearly grasp the possibility of there being a number of coexisting
interacting information processing modules performing different tasks.

Now consider replacing your "simple calculator" with a calculator that
is part of a teaching mechanism. Perhaps its task is to help teach
mathematics teachers, by simulating students who only partially
understand the idea of precedence. The program can parse all arithmetic
expressions in the normal way, and perhaps it always does. But, in
certain teaching contexts, after parsing it reorganises the parse tree
so that "3 + 5 * 2" is first parsed as

        +            which is then               *
     3    *          changed changed to        +   2
        5   2                                3  5


giving the result that surprised you. If you did not know that it was
part of a suite of teacher training programs and you found the interface
on a screen, not knowing that the user had got to a point where the
program was checking out ability to detect precedence errors you might
wrongly conclude that the program did not have the correct parsing
ability.

Depending on how complex the program was and how many other modules were
at work within it you might or might not be able to work out what was
going on by interacting with it.

There is no reason to believe that you could always work out what was
going on by opening it up and attempting to decompile the physical state
of the machine, without knowing a great deal about the design of the
machine and its software. E.g. all the software in it, even the machine
code, could be stored in a highly encrypted form which can only be
decrypted when certain known users were logged in.

If instead of being a well designed teacher training program it was a
very much more complex suite of modules designed to implement a model of
a very intelligent but not very friendly or cooperative human, it could
be even more difficult for you to find out what exactly the program
knows and does not know, what its motives are at any particular time,
what features of its personality generate those motives, how it selects
strategies for interacting with others and so on. In fact it could be
impossible if the machine's state is constantly changing under the
influence of its interactions so far. E.g. because you don't know
something about its motive structure you could, at a certain point,
unwittingly activate an aspect of its personality which thereafter gives
high priority to the goal of confusing you while giving the impression
of trying to be helpful.

And for the same reasons as before, the fact that you cannot discover
all the semantic states from outside does not imply that they are things
you can discover by using physical instruments.


[AS]
> >I think you may have been using a very narrow notion of control
> >system (e.g. something you can inspect only using a brain scanner).
[AW]
> Not at all. But the control system is something physically inner, to which
                                                  ^^^^^^^^^^
> your epistemic relation is about the same as your relation to your
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> stomach.

Why "physically inner"? Have you completely misunderstood all the talk
about virtual machines?

Well if I have not yet been able to convince you how wrong your model of
a control system is, I guess I never will, and this whole discussion is
a waste of time. It may be that if you spend some time working on a real
information driven control system you'll see what's wrong with your
view. But maybe you won't.

If by the same epistemic relation you mean that in both cases you don't
know what's going on then that's true, but no different from what I was
saying. If you mean there's the same explanatory relationship with high
level mental states and processes (need I list them again) then you are
just wrong. (I suspect you know that, actually.)


[AS]
> >But explaining the possibility of a working instance of level A may
> >require reference to level B, and explaining some of the divergences

[AW]
> But explaining *what it is* to be a working instance of level A does not?
> That would be what I am pushing with regard to intentional systems.

In some cases it is possible to make a sharp separation between a
requirements specification and a design specification. (Though why you
should restrict philosophy to be concerned with only one of those levels
beats me.)

However, when the system in question is so complex in its functionality
that it consists of many coexisting, interacting sub-systems, for each
of which the others provide part of the environment, then, as in the
case of the system with an uncooperative module described above, I do
not believe it is possible to provide a complete requirements
specification without talking about the architecture.

(Though I agree it may sometimes be worth trying, e.g. when attempting
to understand a simple calculator: and if all you were doing was
*trying* I would have no disagreement - it's only your dogmatic
exclusion of everything else as necessarily completely irrelevant that I
am objecting to.)

My own conjecture about how our concepts of mentality work is that we
have already been programmed by evolution to use concepts that
presuppose a certain kind of architectural complexity in other agents
and that our mentalistic concepts are based on this innately determined
framework. The general framework could then be shaped and extended by
the culture the child is born into.

You can see why such pre-programming of assumptions about
architecture-based concepts might be biologically useful: otherwise each
individual infant would somehow have to invent from first principles the
idea of another mind having beliefs, desires, intentions, etc. and to do
so entirely on the basis of behavioural analysis of others. From my
observations of children it is clear that they have and use such
concepts long before they can talk about them. That is why from a very
very early age interactions between infants and parents can take on the
form of a political power struggle, which parents often lose.

(Some people assume the child learns about other minds on the basis of
extrapolation from the self to others. But that still requires the
infant to start with concepts of such mental states and processes as
applicable to itself, and for the sorts of reasons Kant gave against
concept empiricism I do not believe a new born infant could possibly
derive, induce or abduce such concepts by self observation. So even on
that theory some sort of INNATELY determined conceptual apparatus
concerned with describing, recognising, and making inferences about
mental states is required.)


[AS]
> >I don't think you appreciate the extent to which an external
> >language is but an extension of and depends on the vast
> >representational capabilities required for many non-human animal
> >capabilities, including perception, learning, problem-solving,
> >nest-building, hunting, etc.

[AW]
> And I don't think you acknowledge the extent to which an external
> language gives you a new medium to think in and makes possible new
> forms of thinking, such as those about quantum mechanics or even lesser
> affairs, one that is simply impossible without it.

I have never denied any of that. Why should I? It's so obviously true I
wouldn't bother arguing about it.

But the obvious fact that external languages EXTEND our ability to think
about minds, or even to have mental states, does not imply that the
external languages are NECESSARY for thinking about or having them.

[AW]
> Dennett's story
> about the innate control system capacities being yoked by social
> pressure into emulating a speaker is a good one, I think.
                ^^^^^^^^^

Why emulating? Surely that should be "becoming a speaker"

Of course the social pressures help to turn a human infant into a
speaker of English or Urdu or whatever. Who would deny that?

But what being a speaker amounts to is not a fixed unitary concept.

There's a whole space of possibilities, and without adopting the design
stance you will not understand that space and you will not see what
explains the possibility of our region of that space nor what it is that
mice lack that prevents them having our kind of social system.

(Also you will not see how to explain the possibility of such design
instances being implemented in the physical world - one of the old
problems in philosophy of mind.)


[AW]
> Even on a lesser level, very few animals have much in the way of tool
> use, for example. (Dewey once wrote that language as the "tool of tools
> is the cherishing mother of all significance") You could at least
> acknowledge that culturally acquired forms of representation --
> mathematical notations, say -- are like tools and are what make
> possible all of the science that you are so interested in. No animal
> control system alone could form a theory.

Why are you telling me all these things that are so obvious?

I don't see their relevance to anything we have been discussing, about
the nature of mental concepts like: believe, decide, perceive, learn,
desire, etc. etc.

[AW]
> Partly I wonder if your refusal here stems from refusal to keep very
> separate as I do between what an organism can do and what a control
> system can do.

Why bother to keep them separate?

Are you simply trying to retain something for philosophers to discuss
without having to learn anything not included in philosophy books?

If you are a complex control system of the sort I've talked about then
your separation may prevent your ever understanding much about yourself
apart from the relatively shallow stuff that is available to common
sense.

[AW]
> Some functions of the control systems can be analyzed
> as performing transformations on vectorial representations of spatial
> facts.  Still no non-linguistic animal can actually do linear algebra
> problems -- that is a very different thing, which requires mastery of a
> public technique for operating with the notation of linear algebra.

Of course the fact that a system uses algorithm X deep down inside
itself does not mean that the agent can use algorithm X to perform
externally posed tasks. Who would claim otherwise? I think you are
arguing against some sort of straw person that I have never met.

[AW]
> ..That
> notation is a public, cultural possession, and if you can do linear
> algebra it is not at all because of your control system, but because
                ^^^^^^^^^^                                     ^^^^^^
> the mastery of this technique was transmitted to you from the
> culture.

You repeatedly use "because" and similar constructs as if you were
giving the WHOLE explanation of something.

This is as silly as saying that the fact that 747 can fly is
*not*at*all* because of the aerodynamic properties of its wings, or the
mechanisms in its engines, *but*because* the pilot can drive it and the
flight control people can specify its take off time and flight path.

This is just a silly opposition. The oppositions you keep making strike
me as being similar.


[AW]
> So sure, beasts can solve spatial problems behaviorally, because their
> control systems do it for them,
                  ^^^^

I would not put it like that. Neither should you. It's inconsistent with
your previous division between what the whole agent does and what parts
of it can do, which I accept.

[AW]
> but they can't solve any such problem
> theoretically, as a scientist might, because they have not been given
> the language in which to conduct these reasonings.

As it happens I don't know what bonobos can or cannot do, and I suspect
you have no right to pontificate about it either. But even if you are
right about dogs and cats and mice (and I am sure you are) I cannot see
why you are bothering to say this.

What has it got to do with analysing and explaining the possibilities of
mental states and processes?

I assume you don't want to deny that other animals can see, or learn, or
remember, or be frightened, or notice things?

Or perhaps you do?

Do you really do want to say, like Descartes, that other animals have no
mental states and that when a dog is cut open the squeals and howls are
merely workings of a physical machine, for there is no pain?

If that's the position you are writing from then the whole discussion
has been pointless for that sort of person will not be influenced by
arguments of the sort I have been using.

=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 3)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is a continuation of my reply to Anders Weinstein
<andersw+@pitt.edu> who wrote on 13th July in response to my very long
critique of his restriction of philosophy to the NPR (normal personal
realm i.e. adult human western culture normal personal realm).

He wanted to link descriptions of mental states to norms and
standards of rationality. I objected:

> >[AS]
> >> >So if you tell me that you have a bad toothache, but you don't go to
> >> >the dentist .... your statement that you have the toothache
> >> > involves no commitment ... puts you under no sort of obligation...
> ....
[AW]
> ....The fact is: you take them to be in a
> state in which the rational thing to do is to go to the dentist, or
> otherwise seek relief.

I think you are confusing what's rational with what is the content
of a disposition.

A compressed spring has a tendency to increase its length. There is
nothing rational (or irrational) about that.

Similarly, a person who is in pain will have certain tendencies
and there's nothing intrinsically rational or irrational in acting on
them, or not acting on them.

A full account of pain (or pleasure) would be very complicated.
Here is a very superficial first draft partial account, elaborating on
the above, only in sufficient detail to give a hint as to what a more
complete account would look like.

A thing that is in pain stands in some causal relationships to possible
actions. If X is in pain then (roughly):

    1. X is inclined to terminate the state or reduce its intensity

    2. If X can identify what it was in the current situation that
       brought about the state then X will be inclined in future to
       avoid or prevent or terminate such situations. (In more
       primitive brains with learning mechanisms this can probably be
       expressed in terms of negative reinforcement.)

    3. X has a counterfactual (or "subjunctive") preference for the
       state not to have occurred. This is relevant only to
       conceptually sophisticated sufferers, and not, I suspect, to
       dogs, cats, birds, lizards, etc. What I mean by this is that
       a counterfactual conditional statement is true, namely that
       if X had been aware of the likelihood of the state being about
       to start then if he/she had the opportunity to prevent its
       starting the opportunity would have been taken (subject to a
       host of defeasibility conditions)

This is not meant to be a full account of what it is to be in pain.

These are merely illustrative of a class of dispositions which I take to
be typically involved (by definition) in states in which pain exists.
Particular pains will have additional contents. E.g. a located pain will
be a state with dispositions related to the location as well as semantic
content referring to that location.

All of these are statements about dispositions or inclinations in X
which can be overridden by all sorts of other states that can coexist
with the pain. Thus the descriptions are all defeasible. E.g. the pain
may be masochistic, or self inflicted as a test, or dominated by other
aspects of the current situation (like a wounded soldier while the
battle continues to rage), or so slight that an estimate of the amount
of effort required to change the state overrides the inclination to
change the state.

The above has nothing to do with rationality or social norms, though it
is concerned with causal powers inherent in a situation.

Rationality can be involved in states of pain in a subset of cases where
there is additional sophistication. Again the following is a first draft
provisional account of the additional requirements rather than a
complete analysis.

If the following conditions hold:
    (a) X is *aware* of the state which has features like 1, 2, and 3
       (which requires some sort of self-monitoring capability in X,
        together with the ability of this pain state to win over other
        potential attention attractors)
    (b) X *wants* the state to be terminated (which does not follow from
        1, 2, and 3, but requires additional motive generating and
        motive processing capabilities)
    (c) X has and is aware of the opportunity to terminate the state
then
    the rational thing for X to do is use the opportunity
(unless a typical defeating condition holds, such as that X wants
something else more, or X judges the cost of using the opportunity too
high, etc.)

Assessments of rationality require something more than mere states of
pain. I am sure a rat can have pains (and maybe also pleasures), but I
have no idea whether it can ever satisfy the additional conditions
required for its reactions to be rational (or irrational). I have
similar doubts about very young infants.

I suspect (but am not sure) that chimps and bonobos do satisfy the
additional conditions, e.g. because they have the required additional
self monitoring capabilities.

Whether a robot does will depend on the sophistication of the robot's
information processing architecture.

[AW]
> That is already a normative statement, in
> virtue of the fact that "rational" is already a normative term.

You are just assuming that the connection between having a pain and
taking action has something to do with rationality. A rat can be in
pain without there being anything rational for it to do or not do.

But if you really do think that mental states are not really present
except in agents that can use self-descriptive language, then, like
Descartes you probably don't believe rats, dogs, etc. can be in pain.

Is that your position?

[AW]
> I am a bit curious as to how you think of norms of rationality, their
> ontological status and relation to control system science.

Actually I don't think much about "norms of rationality". I regard
rationality as being a characteristic of a tiny subset of human
mental processes, namely those in which we have reasons for saying
or doing or thinking or wanting something. Then to be rational is to
act on those reasons and not to act without reasons.

This has nothing to do with social norms. A totally isolated individual
who wants X and knows he can achieve X by doing Y and has no reason not
to do Y, and who has no other current desire, is irrational if he does
not do Y. But all that means is that he has reasons and does not act on
them. (There could be many explanations for this, which might be couched
in terms of processes in the architecture.)

You seem to be of the opinion that rationality has something to do with
what society dictates, expects, teaches. This is wrong because it is
possible for an individual to be rational in the midst of a largely
irrational society in which people are indoctrinated to accept beliefs,
goals and norms without having any reasons for them, e.g. as a result of
religious or racialist indoctrination.

On the other hand, I think it is a myth that intelligence has very much
to do with rationality. One of the most powerful aspects of human
intelligence is the ability to see, i.e. to extract very rich
multi-purpose information from optic arrays. This has nothing to do with
being rational.

A huge amount of the training of intelligent mathematicians, scientists,
game players, etc. is the inculcation of rapid automatic cognitive
reflexes, which are useful and even correct, but which are produced in a
manner that is automatic and does not require rationality.
(E.g. memorising multiplication tables, memorising opening gambits in
chess, memorising rules and formulae so that they can be recalled and
applied without necessarily remembering WHY they work.)
=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 4)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

Anders wishes to make some sort of sharp division between a control
system and a set of capacities.

[AW]
>...
> The point about capacities *was* separate from the point about norms.
> It was preliminary to suggesting that what we think of as "the mind"
> is not in any way an inner control system but rather a set of capacities.
                                            ^^^^^^^^^^

This use of "but rather" is incomprehensible to me. There is no
contrast between a control system and a set of capacities.

You've probably misread "control system" as something that is rigidly
deterministic and inflexible. that might explain a lot of the things you
have written in previous mechanisms which I have found very odd, e.g.
pejorative labels applied to control mechanisms.

A sophisticated factory control system might be a very large collection
of capacities (capabilities, powers, skills, know-how, and knowledge
about constraints, requirements, high level objectives, preference
orderings and languages of various sorts) only a subset of which is
active at any time.

Some of these capacities could involve abilities to communicate with
humans involved in the factory, including managers, accountants,
delivery van drivers, security officers, trainee operators to whom the
system gives instruction and advice, and others.

Why do you think there's some opposition between a control system and a
set of capacities?

I suspect you are using some very rigid notion of a control system:
something like a deterministic algorithm or electronic circuit. You
don't have the notion that a control system might be a complex (virtual
machine) architecture consisting of a large number of different
capabilities and stores of knowledge, including motive generators,
motive assessors, planning sub-systems, perceptual mechanisms, etc.

In that sort of architecture some of the subsystems may have a
collection of capacities and dispositions that are relatively easily
definable in terms of the global behaviour of the subsystem. But the
architecture as a whole need not be like that. For example there could
be a sub-mechanism concerned with a certain linguistic capability that
never gains control for any one of many different reasons. One reason
might be that the conditions under which it is relevant may never arise.
Another reason may be that whenever it is relevant there is always some
other important goal that dominates. Another reason might be that this
capability is *suppressed* because the output has become very strongly
negatively evaluated: an extreme case would be someone who had taken a
vow of silence. A less extreme case would be a person who is so
embarrassed by a particular word that he knows well, that he can never
bring himself to utter it.

For all these different sorts of reasons you cannot define the
information processing capabilities inherent in the total system in
terms of the dispositions of the total system or in terms of
behavioural tests that can be performed on the total system.

Instead you HAVE to specify some features of the architecture: i.e. this
is an aspect of the state (e.g. a mental state) of the system that
requires the design stance for its specification. (I think the full
story is more complex than I have indicated.)

[AW]
> Principal among these, I happen to think, is the capacity to use words in
> accordance with public norms which give them their meaning.

Clearly that is one of the capacities of a typical normal adult human
being in our culture (not one brought up by wolves, for example). I.e.
it is part of the NPR

But that capacity depends on the individual's (virtual machine)
architecture including a very large store of information required for
such language use, just as a plant control system may need to include
information about a language for communicating with other parts of the
company, or the human operators.

In both cases, the capacity cannot be defined behaviourally.

If you think this could not be a feature of a control system, you just
don't use the word "control system" in the most general way. Then choose
another phrase to describe the sort of information processing system
described above as involved in running the factory. ("Administrative
process" "Management system"?)

>....
[AS]
> >And "make it possible" is a very important type of explanatory
> >relationship. The structure of a chlorine atom makes possible
> >various kinds of chemical compounds. But it does not suffice to make
> >any of them actually occur.

[AW]
> I am disheartened that we agree on one thing yet disagree so on its
> significance. We seem to agree that the study of sub-personal
> information processing is concerned with an implementation level.

You seem to want to use the word "implementation" as if it had some very
narrow, low level, connotation, whereas I am constantly drawing
attention to very high level forms of implementation where components of
the architecture being used have properties and capabilities that are
partly analogous to the things you ascribe to whole persons: e.g. they
may have information and be able under certain conditions to use it.

The ability to acquire, store, manipulate, recall, use and in some cases
communicate information is an aspect of the personal level, or animal
level, or robot level, that characterises WHOLE people, animals or
robots.

But it can ALSO be an aspect of a PART of such a system, a component in
the explanatory architecture in which the whole person is implemented.

The capabilities of such a part may be part of the implementation of
linguistic capabilities, mathematical capabilities, musical
capabilities, ethical standards, aesthetic preferences etc.

I.e. the explanation of capabilities (and states and processes involving
those capabilities) of the WHOLE person may include reference to what
you would describe as a SUB-personal level at which intentional
descriptions are still applicable.

I.e. the options are not, as some parts of your message suggest:
    EITHER whole person level OR brain/physiological/physical level

The third option is: an information processing level at which (virtual
machine) components have and use information.

However, I don't see any reason to think there's a sharp distinction between
the personal and sub-personal level, for reasons I've indicated
previously: e.g. what's personal or sub-personal can vary from one
individual to another or within an individual can vary from one context
to another. Things about your state of mind that you notice, or can
access at one time, you may be unable to detect at another: but that
doesn't mean they have ceased to exist (e.g. your anxiety, or your
knowledge of someone's name, or your knowledge of how a tune or poem
goes).

The situation is further complicated by the fact that there are aspects
of a person (e.g. jealousy, grief, a tendency towards vengeful violence,
the capacity for extraordinary self sacrifice in the interests of
unknown others) that may lie dormant as part of a personality for
a long time and be inaccessible by the person,
and yet take control when triggered by a context (seeing
something, being irritated by another driver on the road, consuming
alcohol or another drug which changes the internal balances of
processes, or being confronted by a child screaming for help in a
burning building).

[AW]
> I
> agree that explaining implementation is important

But I suspect you have a very narrow notion of that, even though you
say:

>...and that it involves
> a different sort of reduction than that found in other paradigms drawn from
> elsewhere in science.  I agree that one can intervene based on design
> stance knowledge in all sorts of ways. A lot of design stance science
> is, as you point out, interesting in its own right and very hard. I
> have *never* suggested it wasn't a worthy and important project.

Well there were occasions when you were very disparaging about it, e.g.
apparently claiming (like Ryle) that it was totally irrelevant to the
explanation of any normal human functioning: as if only aberrations
could be explained in terms of implementation level facts.

And you also suggested that it would be undesirable for a teacher to
make use of the design stance (or a therapist, or lover, or friend?)

This sharp division between personal level and design level and
disparaging attitude toward the latter is part of what I have been
arguing against all along.

Accepting what I say would involve a CHANGE IN OUR CULTURE. It would
involve a recognition of important human-like but sub-human processes
that we need to understand for a full understanding of ourselves, our
educational processes, our counselling processes, our therapies, etc.

=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 5)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

Anders doesn't like my claim that in order to get a clear view of the
nature and variety of the full range of human and non-human mental
states and processes including the ability to perceive and to learn and
to think to have beliefs, to have desires, to take decisions, and many
more, we need to adopt the design stance, and think about the variety of
information processing architectures (which will mostly be virtual
machine architectures, not directly implemented in physical or
physiological mechanisms).

I've claimed

(a) that there is a wide variety of architectures, some of them closely
    similar and others different widely

(b) that different classes of architectures make possible different
    classes of mental states and processes

(c) so that a full analysis of the whole range of concepts of mentality
    has to be grounded in the study of the range of architectures and
    their generative capabilities.

Moreover I've argued that many aspects of these architectures are both
sub-personal in that we are not aware of them and they are not directly
observable or testable behaviourally, and yet they can involve mentality
because they include information, decision making, inference, etc. I've
called this information-based use of the design stance "the information
level".

It is above Newell's and Simon's symbol processing level (in which there
are merely structures that are manipulated but no semantics) and below
their knowledge level and Dennett's intentional stance level.

I suspect, from some of his repeated pejorative comments that Anders
does not know what this level is: he confuses its contents with some
lower level sub-personal states and processes that could be checked out
in the brain using physical instruments, and he does not believe (does
not want to believe?) that anything below the personal level could
include intentional states, e.g. acquire, store, manipulate or use,
information.

I.e. he thinks everything mental is accessible at the level of the whole
agent, defined in terms of linguistic capabilities, shaped by a culture
and social norms, and subject to rational assessment, and he
thinks everything that does not work at that level must be free of
semantics and physically measurable.

Perhaps that's because he thinks that only physical systems can provide
implementations for anything. He also wants a very sharp and rigid
distinction between levels: nothing can straddle levels and it is
perfectly clear whether any state is personal or sub-personal and
therefore non-intentional.

I think another source of his difficulty is his belief that control
systems must be very simple and rigid, either mechanical or electronic
devices like steam engine governors and thermostats or else simple
algorithms, like the algorithm for taking in an arithmetical formula on
a calculator and computing the result. He doesn't understand the
possibility of an information processing *architecture*.

I don't deny that the type of analysis that he is interested in can be a
part of philosophy. In fact I think there is already a name for it
"Phenomenology". What I am claiming is that there's an important role
for philosophy of mind outside phenomenology.


[AW]
> ....my main point could be summed up as:
>
>    Control system science is *only* concerned with an implementation
>    level. Therefore it is irrelevant to philosophy of mind.

It cannot be ONLY concerned with the implementation level. That's
because it also has to be concerned with WHAT it is that that level
explains.

E.g. it explains things like your ability to do mathematics, to
perceive, to learn a language, to reason, to suddenly have a new desire,
to get into an emotional state.

Moreover, in some cases it is hard to be precise about what it is that
is being explained except in terms of an architecture.

(E.g. I think there are many uses of the word "emotional", and at least
one of them can be analysed only in terms of an implicit theory of the
architecture as one in which it is possible sometimes to be in control
of one's thought processes sometimes not. See the papers in the
Cognition and Affect project directory for more on that:

    ftp://ftp.cs.bham.ac.uk/pub/groups/cog_affect/
E.g. these files on how the architecture accounts for states like grief:
        Wright_Sloman_Beaudoin_grief.ps.Z
        Wright_Sloman_Beaudoin_grief.text
(The files will have to be withdrawn when the journal article appears,
in Philosophy Psychiatry and Psychology).

Since the philosophy of mind is concerned with what all of these
phenomena are, and how they might be explained, or what might bring
about their existence, philosophy of mind HAS to be concerned with the
implementation level.

That's why so much of what is written by professional philosophers
about mentality is so narrow and arid: it concentrates on shallow
features close to common sense, and ignores so much of what is deep and
important about human and animal, normal and abnormal, young and old
minds.

To me the distinctions you are trying to make all look like the last
ditch defence of an intellectually lazy philosophical community who just
want to sit in their armchairs and talk about what they already know in
virtue of having absorbed a language and a culture and a lot of common
sense. (The NPR - normal personal realm)

They don't want to find out how that relates to a deeper, richer
and more general reality.

[AW]
> Philosophy of mind, I take it, is concerned with conceptual questions,
> questions about the nature of mental states, in abstraction from
> questions about how one might implement them.

Define it that way if you wish, but don't expect all other philosophers
to accept your definition.

That's what I regard as narrow, shallow self-centred philosophy. I
try hard to make sure none of my philosophy students think like that.

You have simply defined out of existence an older tradition in
philosophy
(a) which tries to make sense of what sorts of mental states and
    processes there are in different sorts of creatures,
(b) which asks what sorts of machines could have them if any,
(c) which asks what sorts of relationships such states and processes
    might have to various sorts of physical systems, including brains,
(d) which includes in the nature of mental states and processes
    perceiving, noticing, learning, being afraid, deciding, wanting
    something, etc.

It's this broader philosophy of mind which I claim needs the design
stance (at the information level) and which should be able to show that
our ordinary concepts of mentality can be revised, enriched, extended
(though not discarded) by showing how to ground more systematic sets of
concepts in a theory of architecture.

Anders thinks that any attempt to analyse mental concepts in terms of
something at a lower level must be mistaken because that is confusing an
implementation level with what is implemented. I.e. he assumes that all
such distinctions are absolute:

[AW]
> I think this idea is analogous to the way someone who thought the mind
> was a production system might think that neuroscience, while important,
> was pitched one level too low for their concerns.  It could fill in the
> details about how to implement the production system, but would not
> tell you what is going on at the level of rule firing,
> conflict-resolution, etc.

I am now going to make a comment that may surprise you: sometimes a
philosopher will need to dig deeper into implementation levels than an
engineer!

(Can you guess my reasons before reading on?)


An engineer who just wanted to design an expert system and had a
language for expressing-condition action rules, could design, implement,
test, sell and (if she's lucky) maintain an expert system without ever
thinking about anything below the level of the production system
language.

A philosopher studying such systems should want to know what makes
the existence of such a system possible (remember all of Kant's
questions of the form "How is X possible?" See chapter 2 of A.Sloman
The Computer Revolution in Philosophy 1978, for more on this).

Answering that question requires talking about implementation levels,
including pointing out that there need not be a UNIQUE lower level
implementation: the same system could be implemented in different ways
in different lower level languages, different machine architectures etc.

For some virtual machines, the differences in implementation make no
difference at all to what the virtual machine is how it behaves (at its
own level, apart from differences in speed and perhaps overall
reliability). However, for some virtual machines, implementation can
make a difference.

Moreover, even when it doesn't, it's an interesting philosophical
question how the existence of such implementation hierarchies can
produce causal powers at the level of a production system interpreter.
I.e. philosophers can discuss matters that the engineer just takes for
granted without needing to analyse them.

Moreover, a broad minded philosopher, unlike a narrow minded engineer
just doing a job, might be interested in questions about what difference
it would make to the expert system virtual machine if the implementation
were different.

Looking at different possible implementation architectures may then
reveal a space of concepts for subtly different classes of high level
designs (just as the low level architecture of matter reveals space for
finer grain than the periodic table of the elements as a high level
conceptual framework for kinds of stuff: e.g. isotopes and ions become
possibilities).

For example: the machine hardware and operating system design may or may
not allow user-level interrupt handlers (as most modern systems do,
though not all did in the past and not all languages support this). If
these are allowed then a high level language can allow the definition of
interrupt handlers and this can give the system new interactive powers
including new debugging capabilities (as is commonplace in AI
languages).

Similarly if there's error correcting hardware and the implementation
allows "failsafe" duplication then this may mean that in some
circumstances the high level virtual machine will degrade gracefully,
instead of catastrophically crashing.

Once these points are understood we can see how the philosophical study
of mind includes a survey of the space of possible minds and the kinds
of architectures that can support them. (This is a space with no sharp
external boundaries dividing things that are from things that are not
minds, but which contains lots and lots of fascinating internal
structure including many design discontinuities).

[AW]
> I wish you would appreciate that I only think that design stance
> science is irrelevant to one *sort* of question, conceptual questions
> about the nature of mental states.

They are irrelevant to the conceptual questions that YOU happen to be
interested in. (And a subset of philosophers. E.g. your teachers?)

They are not irrelevant to a broader set of conceptual questions about
the nature of learning, perception, desires, beliefs, moods, etc. for
these are concepts (I claim) which implicitly refer to an underlying
architecture supporting a rich collection of states and processes.

When I find out more about what the architecture actually is I can then
find out that my concepts are in need of extensions, refinement, and
in a few cases correction.

(E.g. by comparing the architectural requirements for monocular and
binocular vision we can see that there are at least two kinds of
percepts of depth or distance and then seek introspective or other
forms of corroboration.)

This survey of alternative architectures and their implications gives a
better framework within which to know how to talk about mental states of
brain damaged people, young children, chimps, and what it's like to be a
bat (or a rock).

By never looking at any implementation level you forever isolate
yourself from such important self education regarding the inadequacy of
your own concepts of mental states.


[AW]
>....
> I must admit I also have another commitment, viz. that the answers to
> these conceptual questions can abstract from all details about the
> control systems inside people's heads that make them possible. This is
> certainly true of the philosophies of all the authors I cited.

They are all guilty of the combination of blinkered, lazy, ignorant
self-protection of which I am complaining.

Or put it another way: they are all focusing on one tiny sub-region of
the space of possible minds. Poor things.

[AW]
> I suppose if one thought the mind simply *was* the inner control
> system,

I wonder what the words "simply" and "inner" ar doing here?

Is water simply the hidden H2O molecule? Suppose I say it is H2O but not
"simply" H2O. A mind is a sophisticated control system, but it isn't
simply that, not least because being that is nothing simple: but
something very rich and complex. Moreover, your mind includes a lot of
very specific relationships to your environment, which includes your
culture.

The same would be true of any control system which is part of a
multi-agent system. Embedding in the larger context would change what it
was. The more information it had and used about the larger context the
more it would be changed. So it would not "simply" *be* whatever such a
control system might be if totally removed from the larger system.

[AW]
> then of course design stance stuff would be highly relevant to
> the nature of mental states.

As it is for all the reasons I've given, not least explaining some of
the variety of mental states and mental capabilities.

[AW]
> In that case, the study of design would
> *not* be merely an implementation level, something pitched one level
> down in the hierarchy from the questions that interest me.

So what? Why should the study of design be "merely" anything?

You do really like to carve up reality into disconnected segments it
seems?

> ...Rather, it
> would simply be the mental or intentional level itself,

There's that "simply" again.

Try re-writing everything you've written, and wherever you have used
"simply" or "merely" put in

    "among other things, and to a first approximation",

and see how you like the result. It would be much closer to what I'm
saying.

[AW]
> ...and there
> simply wouldn't be a higher level in which one talks about the whole
> person or organism.

Complete non-sequitur. I have no idea why you think this follows.

There's a level at which I can talk about a car engine as having a
certain power (e.g. 50 horse power). I can also talk about it as
satisfying legal requirements for emission, braking capabilities, and
other things. These are properties of the whole car.

Explaining how they are possible, and how they might vary from one car
to another, or how they might change in the life time of the car
involves talking about cylinders, pistons, carburettor, spark plugs,
distributor, exhaust system, etc.etc. But it remains the case that when
I have explained how the power, comfort, ease of driving, satisfaction
of legal requirements, etc. depends on those implementation level
features I can still talk about the horse power and other things as
features of the whole system. It is not a feature of the pistons, or the
spark plugs, etc.

In a partly analogous fashion I can tell you how language understanding
capability is in part composed of a collection of capabilities some of
which you don't know you have. But the ability to understand English is
still a feature of you, the whole person.

[AW]
> If this were so, the architecture of the control system would *be* the
> architecture of a mind.

Exactly. But remember that the architecture of an information processing
system is not necessarily fixed, like the physical architecture of a
car.

E.g. by absorbing a culture you change your collections of capabilities:
you change your architecture.

Thus only some of the architecture of a mind will be common across human
minds. This may mean that some minds may be capable of states that
others are not, because they lack the cultural prerequisites. For
instance I think there could be (and maybe there are) cultures (or
subcultures of our culture) in which it is impossible to have moral
indignation.

So you  have to distinguish the architecture presupposed in a particular
culture at a particular time, on which the concepts of mentality of that
culture are based, from the architecture that a deeper scientific theory
might reveal is part of a more general architectural framework within
which such cultures may be absorbed and within which other forms of
development could happen.

Within the deeper framework we can get deeper and more general concepts
of mentality. E.g. moral indignation will be a particular kind of mental
state, which is part of a more general family mental states
corresponding to a subclass of what we call emotions. A still broader
class of emotions (e.g. types of fear) will apply both to linguistic and
non-linguistic agents, though subsets of that class (e.g. fear of losing
one's job) will be language and culture dependent.

[AW]
> ....I wonder why you seem to
> agree with me that the control system is merely an implementation level
> for person-level capacities and abilities.

Your use of "merely" seems to me to express some confusion. I would
never use that word in that context.

What's "mere" about it?

I suspect you have a distorted notion of the relationships between
levels.

Later on in Anders' message a point arose about the breadth of the
design stance, which I have claimed is needed to unify our conceptions
of mentality in a wide range of contexts, including other animals and
people with brain damage. Anders seems to think that it would exclude
the study of brain damage.

[AS]
> >....The design
> >space of animal minds is not a totally smooth continuum, for there
> >are many discontinuities, but neither is it a dichotomy with some
> >major divide between us and the rest. Especially if, among us, you
> >include all the varieties of human beings of all ages, including
> >those with genetic brain defects, degenerative brain diseases, brain
> >damage, and all the pathologies that don't stem from physical
> >abnormality.

[AW]
> I would have thought design stance science covers correct design, not
> defective.

That would mean that you had to abandon the design stance as soon as you
were trying to improve a design, find bugs, fix bugs. I cannot imagine
anyone wanting to exclude such things from the design stance. Did
Dennett every make such a suggestion in his presentations of the
design stance?

Why on earth would one want to exclude them? They are normal parts of
the design process.

(Except for miraculously good designers?)


=======================================================================

Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 6)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

One of the problems Anders brings up is the old one of how it is
possible for a mental state to have semantic content, to refer to
something. Haugeland and later Searle called this the problem of
"original intentionality". Harnad later called it "the symbol grounding
problem" (and generated much confusion by leading people to think
meanings have to be *grounded*, an old but discredited idea found in
empiricist theories of meaning -- e.g. Lock, Hume -- slaughtered by
Kant.)

The problem of explaining how meaning bearers, e.g. symbols,
representations, control states or whatever, refer for a particular user
or owner of the meaning bearers is a difficult one. I have discussed it
at length in a succession of papers, largely arguing that the question
as normally posed is confused because it assumes that there is a single
dichotomy between things that can and those that cannot "understand" or
have intentional states, whereas there is a wide variety of cases,
corresponding to different architectures (and maybe different languages
and cultures and individual differences) not adequately describable and
distinguishable using our ordinary concepts, like "understands",
"refers", "intends".

My papers on this are in the Birmingham Cognition and Affect ftp
directory
    ftp://ftp.cs.bham.ac.uk/pub/groups/cog_affect/

including my IJCAI85 paper
    Sloman.ijcai85.ps.Z
        (What enables a machine to understand?)
A sequel is in the ECAI96 proceedings
    Sloman.ecai86.ps.Z
        (Reference without causal links.)

And there's more in my contribution to the (still unpublished)
Turing90 colloquium, which discusses the still tentative notion
of loop-closing semantics:
    Sloman.turing90.ps.Z
        (Beyond Turing Equivalence)

Anders wants to argue that only a whole organism that is part of a
linguistic culture can have semantic states that refer to things.
I think he is quite wrong, not least because non-linguistic animals can
have percepts, beliefs, desires, fears which refer to something in the
environment, even if their states differ in some detail from human
versions and even if they do not conceptualise the environment as we do.

[AW]
> Let me try one contemporary example. One might wonder, philosophically,
> how it is that my thought of a sheep gets to be about sheep. One crude
> attempt at an answer: it is a state typically caused by the presence of
> sheep. Way too simple, but a candidate answer, anyway.

It's such a silly answer I don't like having to discuss it. No theory of
meaning that regards causal links between referent and meaning bearer
can cope with our ability to refer to nonexistent things (like that six
inch wide spider crawling up your left arm, or the olympic medal I am
going to win in the year 2000). Nor can it cope with reference to things
in the distant past, or to abstract entities like numbers, or proofs, or
infinite sets (when did the set of all integers ever enter into a causal
relationship with anything?)

[One day, on a different thread I may give an answer to that....]

Moreover, having a regular causal relation is not sufficient either,
since there are many such links which are not meaning links (though
H.P.Grice did point out that at least in English they can sometimes be
described using the word "mean" in a different sense, e.g. "Those clouds
mean rain").

[AW]
> Now an answer
> like this doesn't depend on *anything* about how such a causal link
> might be implemented. It may in fact be a very hard and worthwhile
> engineering problem to build a sheep-detector -- say, a black box with
> a photo-sensitive surface that prints out "there's a sheep" in largely
> the right sort of circumstances.

Maybe your problem is that all the implementation level models that you
can think of are so simple and so obviously wrong that you (rightly)
just cannot believe that they have anything to do with what you wanted
to explain.

In that case I may have to say come back later when we have better
tutorials on more realistic explanatory architectures. (Some people seem
to be able to grasp the ideas easily on the basis of familiarity with
problems concepts and techniques of software engineering. Others find it
remarkably difficult.)

If you have some concept of reference which depends on a type of
implementation, and it turns out that there are several different types
of implementation all of which produced slightly different observable
phenomena in unusual circumstances, then that might show that where you
thought ONE concept of reference would suffice, FIVE or SEVENTEEN could
be needed, some of them instantiated in normal humans some not.

Given enough time I would expect to convince you of something like
that in relation to a number of concepts, like "believes" "wants" "is in
pain" "concept of itself" by showing you how slightly different
architectures produce different kinds of minds. This would not be an
empirical demonstration: it would be an analytical argument about the
properties of different designs. Then you could do an empirical study
and find examples of these diverse concepts in the world: e.g. in young
children, chimps, bats, normal adult western humans, etc.?

This sort of refinement of everyday concepts constantly happens as a
result of scientific advance. For example, newtonian physics separated
out notions of mass and weight. Also different notions of energy were
introduced, and separated from momentum. Further notions of energy
developed later on, as a result of the kinetic theory of heat.

Compare the impact of Freud on our common sense terminology (inferiority
complex, repressed desires, etc.)

Anders seems determined to argue that even if information processing
architectures within an agent are part of the explanation of the
possibility of the mental states of that agent, this has nothing to do
with philosophy of mind, and the features of the architecture cannot be
relevant to answering any philosophical question about minds.

[AW]
> Now no one is denying that there is a real question about what needs to
> be inside the box.                                        ^^^^^^^^^^

I would not have guessed, from some of what you wrote.

> ..But I think one can see that for the *philosophical
> question* we started with, it really doesn't matter at all
            ^^^^^^^^^^^^^^^

We started? Who? Not I.

Not the old philosophical tradition going back through Hume, and
Kant and Aristotle.

The restriction of philosophical questions about mind to a tiny subset
concerned with NPR is, I think, just a recent 20th century aberration in
anglo-saxon linguistic philosophy. You should abandon the restriction,
and enjoy the richer, broader, deeper philosophy including the
possibility of philosophy contributing to and learning from other
sciences and some forms of engineering, e.g. from psychology, brain
science, computer science, software engineering.

The narrow kind of philosophy should not be taught to philosophy
students because it is so intellectually stultifying: it
compartmentalises philosophy and encourages and justifies ignorance,
species-centredness and shallow theorising that gives explanatory weight
to social phenomena and ignores the diverse forms of information
processing infrastructure that makes different sorts of social systems
possible.

[AW]
> I don't actually have a causal theory of reference. But I believe that
> nothing inside the head is relevant to the explanation of how my words
> and therefore the thoughts that are expressible in those words get to be
> about things in the world.

I can assure you that by changing things inside your head I could
sever the connection between the noises you utter and things in the
world, without doing anything to the social system in which you are
embedded.

But part of the problem it that you have sensed something that is true
of a subset of meanings and you want to extend it to all aspects of
meaning.

E.g. when I refer to the Eiffel Tower, in my thoughts, intentions,
desires, beliefs, or utterances the existence of the Eiffel Tower and my
existence in an environment containing it is relevant the possibility of
my referring to it.

E.g. one of the reasons why I am not a methodological solipsist is that
I claim there is no way for someone to have a thought about the Eiffel
tower *solely* on the basis of having certain mental states. For
thoughts to refer to concrete (or steel) physical particulars and
certain other things some sort of causal link (possibly very indirect)
is necessary, as discussed by PF Strawson in Individuals.

This is why not ALL kinds of semantic content can be explained entirely
in terms of internal states. But there are other kinds of reference that
do not depend on anything in the environment.

(Another case of "some" not implying "all"). My IJCAI95 and ECAI96
papers discuss these issues.

By designing different kinds of robots with different kinds of things in
their heads, and studying different kinds of animals with different
sorts of things in their heads, and studying human beings at different
stages in their development, when there are different things in their
head (including perhaps a stage when you have senile dementia, which I
hope will never happen, but might) we can then find that where we
thought there was one simple notion of words having meaning for a word
user we may actually find that there are dozens of different cases which
are worth distinguishing: i.e. they define subtly different concepts.
I.e. there is no such thing as THE concept of a word referring to
something.

This will not be too different from finding that where we thought there
was just one concept of the element carbon, study of the architecture of
matter showed that there could be isotopes.

I had previously agreed that many mental states involved dispositions,
abilities, capacities, including some high order dispositions to produce
new dispositions to produce new dispositions (as Ryle had shown). Anders
wants to argue that ultimately all such states are behaviourally
detectable.

[AS]
> >Nope. Who said the tests have to be public? As Ryle remarked, you
> >can have a disposition (or for that matter capability, ability or
> >competence) whose manifestation can be triggered by completely
> >internal events, and whose manifestations can also be completely
> >internal. Knowing how a tune or poem goes is one of the examples
> >Ryle discussed.

[AW]
> They *can* be triggered internally, but that doesn't mean the *tests*
> for their presence aren't public.

We can and do use public tests. I don't dispute that. It doesn't
follow that ALL such tests are public. (Yet another case where
"some" does not imply "all").

I can check whether I remember a tune by giving myself a totally
private test of the sort Ryle discussed, and then tell you that I
remember how the first six bars go, and no more.

I can ask myself whether I remember what someone looks like whom
I've met but not seen recently, and answer yes or no on the basis of
an internal test. That's not the same as asking whether I would
recognize the person if he were in front of me. Of course, I might
say I remember, and then find that my memory was wrong when I meet
the person again.

But that's still different from not remembering anything at all about
the appearance.

I can be sure that I'll enjoy seeing for the umpteenth time a film that
I've always enjoyed in the past. I watch the film, and discover to my
surprise that I am not really enjoying it as much.

I can be sure that for certain classes of problems if confronted with
examples I'll find them intelligible (e.g. exercises in introductory
books on group theory). I can then test myself my looking at some of
them and see whether I find them intelligible. I then find that there
are many such problems that I do not find intelligible and several that
I do. No external test would be so decisive and quick. If I have other
linguistic capabilities you can then go on to test whether I really
understand them correctly or not. But (a) that's not the same as testing
whether I find them intelligible as I read them and (b) if my other
capabilities are damaged there may be no test that could work.

These are all internal tests of dispositional states which I think fit
well within the categories discussed by Ryle (though probably any
behaviourists reading this will be tearing their hair out, if they have
any left by now.)

[AW]
> I think Wittgenstein might provide supplementation to Ryle here. You
> can hear a tune in the imagination, but this capacity is still
> related to public criteria which show you know what the tune is.

This is such weak relationship, for the reasons I've given, that I
am unimpressed by it.

But in any case, I've never denied that SOME internal states may be
related to external phenomena. What I am objecting to is your
determination to link ALL of them to social and linguistic phenomena and
social norms and behavioural tests.

I argued that besides being conceptually relevant, internal states of
the information processing architecture could be directly relevant to
practical problems involving human mental states and processes,
including teaching, counselling, therapy. E.g. a teacher can try to
change a pupil's inference strategies, in part of his architecture.

[AW]
> This doesn't really affect my point. I didn't deny that knowledge of
> the control system might help one intervene in a practical context.
>
> However, even some of this, like "inference strategies" or a conceptual
> system, is basically behavioral

That's your claim. You can't make it stick if you mean externally
detectable. You should know enough to know that a computing system may
include inference strategies among its works that are never testable
from outside. (That's one of the reasons why debugging complete systems
is so hard, and good designers and implementors nomrally test modules
separately.)

The reason it is not externally testable is that it is only one of many
components in the architecture, and you may not be able to set up
testing situations that are sufficient to determine which of those
components dominates the interaction with the environment or an
interrogator.

I suspect you have not grasped this because you have thought of control
systems as essentially monolithic structures (like a parsing algorithm).
The opposite view is argued at length in Minsky's Society of Mind,
though it is very much older than that (Plato, Kant, Freud, faculty
psychology).

> ....and dispositional, and is misconceived
> if it is supposed to be about rules or representations in an inner
> control system.  That students have flawed strategies or concepts
> for example is publicly testable in what they do or say.

Well if you had real experience of being a teacher (or of designing
working models of human-like systems or other complex information
processing systems) you would know just how wrong that is.

My theories about the flawed strategies or concepts or similarity
judgements or misleading background associations in my students are very
RARELY directly testable.

They are *theories* not observations. I can use them as a basis for
practical or predictive strategies and then see if my strategies work.

But that's similar to concepts and theories about unobservables in
physics or chemistry. Even if the predictions come out right or the
pupils seem to have learnt in the end, that does not prove that my
theory about what went wrong previously was correct.

Like all the best scientific theories, my hypotheses about what's going
on in the minds of my students are constantly subject to revision.

[AS]
> >I may probe and explore to find out what's in there (which the
> >student may not know) and then use that to help me collaborate with
> >the student's own mind (or brain if you prefer) in that mysterious
> >bootstrapping process we call education.

[AW]
> I would prefer you to collaborate with the student.

No doubt. But I can't see the relevance of your preference in this
context.

We were discussing the nature of mind not the ethics of
interpersonal behaviour. (I say that because I think that ultimately
your whole position has a strong ethical component: you somehow
dislike the notion that we are information processing systems of a
type that is amenable to scientific study, maybe for the same sorts
of reasons as Weizenbaum, in Computer Power and Human Reason. I
suspect Ryle did not have this kind of ethical motive behind his
mistakes: he was simply ignorant of the conceptual framework we now
have, which you don't want to use. But, as a certain commentator we all
know would say, that's just my interpretation.)

In any case, what I was talking about might involve collaboration
with the student, just as brain damaged patients collaborate with
the neuro-surgeon in performing tests which help to identify the
type and nature of the damage.

The difference is that often with a pupil I am trying to identify
something that can be more easily fixed than brain damage, sometimes
with the explicit help of the student. I.e. I don't need to anaesthetise
the students!

But I may give them lots of practice examples that transform their
habits. This is commonplace in teaching music also.

Sometimes, in a tutorial context, I use tricks to remove high levels of
anxiety which I believe interfere with the attentional mechanisms that
are needed for the students to allocate resources to what I am trying to
teach them rather than to imagined (or real) potential threats in the
situation.

It seems to work quite well for many students. Later I tell them what I
was doing to them, and they don't mind.

My English teacher used to make pupils who had produced a grammatical or
spelling error write out the correct version five or ten or twenty
times, depending on how often they had made the mistake recently. He did
not do this as a *punishment*, as many inferior teachers would, but for
the same reason as a piano teacher makes a pupil practice scales.

Both are implicitly assuming there's a trainable information store --
though they don't really understand it so they cannot optimise their
teaching strategies except on a trail and error basis.

By acquiring a better understanding of the mechanisms involved they may
be able to devise even better "correction" methods.


Anders reveals a very strange misconception about information processing
control systems of the sort I am talking about. He seems to think they
might be detected and usefully manipulated by opening up the brain and
poking around in it with suitable physical instruments.

He seems to think that *everything* important about a person must be
either essential social or essentially physiological. He constantly
fails (or refuses) to see the third possibility.

[AW]
> Sometimes perhaps
> one needs to go behind their back and deal with their brain or their
> control system.

You are apparently using a very narrow notion of control system,
probably linked to neural circuitry. A good teacher can usually get
at a pupil's control system "from the front", just as counsellors,
parents and psychotherapists often try to do.

Are you assuming that all design-based manipulation requires brain
surgery?

[AW]
> Often "finding out what's in there" is a matter of such things as
> finding misconceptions, which is a matter of misusing or
> misunderstanding the rules governing the use of public words or an
> inner analogue of this. So in some sense when you diagnose this you're
> finding out what's "in there", but it may be merely a matter of
                                              ^^^^^^^^
> inferential dispositions, not the inner machinery that make that make
                            ^^^^^^^^^^^^^^^^^^^^^^^
> those dispositions possible.

Yet another "merely". What's "mere" about it?

Yet another false contrast:
    "inferential dispositions" vs "inner machinery".

There is no contrast. The inferential dispositions may be features of
just one component in a complicated architecture.

>...stuff deleted where I think I simply misunderstood Anders....
=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 7)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

Part of the confusion that Anders manifests is I think shared with a lot
of mid to late 20th century English speaking philosophers who were over
impressed by some features of first person present tense utterances
using mental words, e.g. "I believe that..."

Moore's paradox is (if I remember correctly) concerned with the apparent
contradiction between saying things like "p but I don't believe p". A
person saying that appears to have contradicted himself. And yet p could
be true while he does not believe p, so from that viewpoint there is no
contradiction.

[AW]
> Do you agree Moore's paradox is part of the
> logic or grammar of "believe"?

No. I think that there are differences between first person present
tense uses of a lot of mental state concepts and all the other uses.

The importance of first person present tense uses has been inflated
by philosophers causing themselves and others to be deeply muddled.

[AW]
> In the current issue of Mind Arthur Collins presents an argument that
> Moore's paradox refutes any theory of beliefs as inner realities, for
> example, a functional state theory.  He claims any such theory fails to
> explain why in saying "I believe that p and not p" I contradict myself,

[I assume that in this sort of example "and not p" is not supposed to be
in the scope of "believe", otherwise there's no problem, since "p and
not p" is self contradictory.]

However, the claim that there is a contradiction is wrong. You don't
contradict yourself in making such an assertion. It is logically
possible for the conjunction to be true. Therefore there is no
contradiction. E.g. consider this example

    I believe that 3 + 5 equals 9, and 3 + 5 does not equal 9.

I wrote that. I did not contradict myself.

I did, however, make a false statement, whose first conjunct is false,
the second true. But there's no contradiction. Similarly

    I believe that 3 + 5 equals 8, and 3 + 5 does not equal 8.

Again it's a false conjunction, this time with a true first conjunct.
But there's no self contradiction.

However there is something else going on, which might be described as a
"pragmatic infelicity".

I.e. in many contexts the fact that a person utters an indicative
sentence in all sincerity conveys two items of information, namely:
    (a) whatever the content of the sentence is, that which is
        explicitly asserted
    (b) the fact that the speaker believes that that is the case.

(a) is asserted. (b) is not asserted, but in a large collection of
communicative contexts the assertion of (a) is a reasonable basis
for inferring (b). This may or may not be part of the explicit
intention of the speaker. It rarely is.

Thus when a person says "I believe that p, and not p" there is a
conflict between what is explicitly stated in the first half and
what is pragmatically implied in the second half. But that's not a
contradiction in what is said: it is a contradiction between part of
what is said and what a normal listener would infer.

Problem dissolved.

More sophisticated listeners know that people don't always believe
what they assert.

(Not just because they sometimes tell lies: often an incorrect answer to
a question is more helpful to the asker than the exact truth. Sometimes
they forget what they believe, as can happen especially in philosophical
debates like this.)

[AW]
> As with anything in philosophy, there are responses one might give to
> Collins' argument. But I don't think the features of first person present
> tense idioms are trivial.

Well, they may not be *trivial* but people who inflate them to have
*central*importance* certainly get their concepts in deep muddle.

You keep making false dichotomies, on which you base unsound arguments,
like this:
    "Either it's trivial or its crucially important.
    It's not trivial therefore .... "

The premiss is false.

[AW]
> I don't think you are acknowledging the idea that the concept of
> intentional content is already, all by itself, a normative one.

Because it isn't.

That's what I have been saying all along, and giving various, diverse
reasons, based on examples of lots of different kinds of mental state
with intentional (referential) content, including perceiving, learning,
liking, etc.. which can occur in animals.

Your only response to that, so far, is to say how interesting and
important human linguistic capabilities are. So what?

And you keep trying to bring in social norms.

[AW]
> ...For
> example, if you say that some state has the content "Fido is a dog" you
> *thereby* determine worldly conditions under which that state is
> *correct* or in *error*.

I think you've missed a point in Frege's work. The truth values are just
two things. Any two things will do, including the numbers 1 and 0, or
the other way round: all we need is a function mapping situations into a
binary set. (For some concepts it may require a continuum, but lets
ignore that for now. I'll also ignore the point made by Frege that it
may be a partial function, i.e. there need not be a value for every
situation.)

So a propositional content divides classes of states of affairs into two
categories. The ability to make such divisions is very useful as is the
ability to make divisions into three, twenty or a continuous range of
cases.

Having got two truth values you can then adopt a convention that
uttering a sentence conveys the information that one of them (e.g. T, or
1) is the value of the propositional content in the current state of
affairs. You could swap them round and nothing would change, provided
you were totally consistent in your swaps. You don't need a notion of
correctness or error to get all the semantic content you need off the
ground.

(Within an information processing system all sorts of alternative forms
of representation of the difference might be possible. E.g. there could
be a database of "unvalenced" propositional contents all of which have
either "T" or "F" as attributes, and that determines the role they have
in planning, decision making, guiding perceptual processes, etc. There
need not be any "favoured" truth value in the system.)

You can then start talking about how agents of various kinds make use of
semantic contents in their dealings with one another and with the world.

And *then* notions of error and correctness can come in. The orang utang
that judges a branch to be able to support its weight can make an error
and go crashing to its death. Another makes the correct judgement and
survives.

But these notions of "error" and "correctness" in the allocation of
truth-values to propositional contents have nothing to do with social
norms. They are all to do with the roles of information structures in
control architectures concerned with survival (and with lesser goals).

Do you believe that a duck which never learns English can never make a
mistake?

Why then do hunters use decoys?

Anders tries to link semantic properties to the notion of inconsistency
(rightly) and then claims that inconsistency is a notion that depends on
social norms and the idea of a failing:

[AW]
> You also determine that the very same system
> cannot go on to deny "Fido is an animal" without *inconsistency*,
> another normative failing.

Again this has nothing to do with social norms. Moreover sometimes
inconsistencies are very good things: they show you that the proof
you were constructing is valid. (I assume you know enough logic to
recognize that).

If an inconsistency is stored in a certain sort of database it
might cause trouble. But not necessarily: for example it may be far more
convenient, when the architecture is sufficiently complex and the
information stores are sufficiently large, to tolerate quite a lot of
global inconsistencies than to get involved in paralysing exponentially
costly searches to weed out inconsistencies. (It's an NP complete
problem).

[AW]
> ...So in one sense it is merely descriptive to
> say there is such a state, but it determines all these normative
> consequences by virtue of rational inferential relations among contents.

Philosophers who don't think about architectures make the mistake of
treating these *normative* notions as somehow basic and then they cannot
explain how any kind of mind other than a normal adult human fully
rational mind works.

If you replace that sort of philosophy with one that's grounded in the
notion that complex systems actually have to *work* as organisms and
other machines do, you can turn a lot of recent philosophy upside down
in a very fruitful fashion -- and fruitfully abolish destructive
distinctions between philosophy and other disciplines, such as
psychology, ethology, AI.

[AW]
> Now this raises a question about the nature of the relevant norms.
> Perhaps they are natural and impersonal platonic facts about the realm
> of ideas, but one alternative idea is that they depend on tacit social
> conventions that determine what the rules governing the relevant
> concepts are. Perhaps they derive from some other source, or do not
> need to be explained. But you seem to be just taking it for granted
> that items can have representational content and stand in normative
> inferential relations.

No. The normative relations are secondary and arise out of deeper
semantic relations.

Your problem is that you just assume social systems exist as if that
were a brute fact that does not need to be explained and which can
somehow explain the existence of minds of individuals.

What makes it possible for some sorts of individuals to form social
systems and linguistic communities?

Could they do it if they did not include powerful information processing
architectures?

The nature of that architecture deeply determines what sort of social
system is possible (not in detail of course). That's why rats can't have
a society for the study of archeology.

I think I obviously have biology on my side. Powerful brains with rich
information processing capabilities were needed before social systems
could exist.

Of course, brains could continue to evolve under pressure from
requirements of social animals: so the influence is not all one way.

If you want to know how I think information processing capabilities,
with semantic content for the machine can exist independently of
social context the answer is very complex. My students and I have
several papers elaborating on partial answers in the Cognition and
Affect directory
    ftp://ftp.cs.bham.ac.uk/pub/groups/cog_affect/

[AW]
> It means: do you explain the intentionality of public utterances as
> derived from that of the inner states they express?

Does it all have to be derived from one thing or all from another?
(One of these false oppositions: there's too much binary thinking
around, as Minsky often points out.)

Once you have social animals engaged in public communication then
*some* of what they do obviously depends on the existence of the social
community. Who would deny that?

E.g. I cannot say "I promise to be there" if there's no institution of
promising, a social institution.

But just because SOME aspects of public utterances are impossible
without a social context it doesn't follow that ALL aspects of the
public or private use of any sort of information structure depends on a
social context.

For example a person, or animal or robot might use an external notation
for a diary, or for trail blazing, or some other purpose. This is
external and observable but neither social nor based on cultural norms
(unless you wish to talk about culture with one individual in it).

Because Anders thinks of control systems in such simplistic terms
(circuits in the brain, monolithic algorithms, etc.) he does not see how
there can be intentional sub-components in such a system. And that makes
it hard for him to understand how anything sub-personal in himself might
be mental, e.g. things of which he is not conscious.

[AW]
>...
> I can't understand why anyone would consider those processes mental.

I previously gave plenty of cases of mental processes of which you are
or were unconscious.

Some of the processes that start off conscious later become unconscious,
e.g. the perceptual, decision making and motor control processes
involved in driving a car, or processes involved in reading fluently,
which start off being conscious (and deliberate) and later become
unconscious (and automatic). These require very complex information
processing whether you are conscious of that or not. The information has
semantic content, e.g. referring to whether traffic lights are red or
not, whether to turn right at a corner, etc. I.e. these are intentional
states even if you are not aware of them while driving (though you could
be if your attention switched back to the details of the driving task).

Note: even if I am quite wrong in thinking that there's a subconscious
process that manipulates information gained from perception when an
expert performs, there is nothing conceptually wrong about it. You write
as if it is an incoherent suggestion. Thre's nothing incoherent about
it: the only problem is that you seem to be incapable of thinking about
complex information processing architectures some parts of which process
information in ways that are inaccessible to others.

There are other cases where what is achieved in people is clearly not
something physical, e.g. learning the grammatical rules of one's
mother tongue: a process that occurs in young children without their
being aware of them. That's another case where sophisticated
information processing occurs, including formation of new concepts,
abstraction of general rules, construction of complex information
structures from simpler ones.

(All of this is probably implemented in some mixture of neural and
chemical mechanisms. I am talking about higher level virtual machines
with properties that may be very different from those of brain
mechanisms.)

I cannot see any difference between these unconscious sorts of processes
(e.g. linguistic processes and inductive learning processes) and many of
the things you are conscious, of apart from the fact that you are
conscious of some of them but not others. That's a difference in
accessibility within the system. (There are many kinds of access:
clearly some processes and structures that you cannot access by
introspection can be accessed by other processes involved in producing
sentences.)

This difference in accessibility I take to be a matter of which internal
processes are amenable to certain kinds of self-monitoring. Whether they
are or not does not affect whether they involve the use and manipulation
of information, i.e. whether they have intentionality.

[AW]
> Processes in my adrenal glands affect what I say or do, but are not
> mental processes, are they?

It's strange how you concentrate on totally implausible counter
examples.

Assuming that is not wilful, I can explain this only on the assumption
that you have not yet understood what an information based control
system is.

If I found out that your adrenal glands were deeply implicated in the
analysis or interpretation of visual or linguistic input, in appropriate
ways, then I would say that some mental processes were involved. If they
merely speed up or slow down the processes or if they merely produce
hormones that alter mental processes through their effects in other
parts of the brain, then I would not say that processes in the adrenal
glands were mental.

There are chemical processes that are not concerned with anything
mental and there are processes concerned with words, concepts,
visual structures, plans, motives, and other information structures
etc. which are strong candidates for being described as mental
processes whether you are aware of them or not. They certainly are not
physical or chemical or neural (though they are implemented in physical,
or chemical or neural mechanisms).

If you insist that they are not mental if we are not aware of them, and
you cannot give any *other* interesting difference, then I regard that
as a purely definitional move, like refusing to call a circle an ellipse
because you've defined an ellipse to have two separate foci.

[AW]
> The focusing of light by the lens affects
> what I can see but it is not a mental process is it? I am wondering
> what it is that makes a process mental in your view.

Well I hope you now see the answer.

Focusing of an image is part of the processing of information. So it is
a limiting case that's not very interesting.

Finding edges, or regions of similar optical flow is a bit more
interesting, and a stronger candidate for being on the mental side of a
mental/non-mental divide (especially as you can pay attention to the
results, in some cases). Detecting larger scale visual features that cue
into long term memory stores is a still stronger example.

I don't think there's any very sharp divide: though it is not a
*continuum* of cases because there are many discontinuities in design
space.

NOTE: I suspect that this over-simple contrast between mental and
non-mental is one of those dichotomies that will be replaced by a finer
grained and more useful conceptual framework when we have a deeper
understanding of the possible architectures and the variety of types of
processes that can occur in them.


Anders keeps coming back to what the agent is aware of and the global
properties, dispositions, etc. of the whole agent. He doesn't like the
idea of applying mentalistic language to components of the architecture,
i.e. sub-personal systems.

So he denied that mental processes occur that he is unaware of, e.g.
planning processes involved in sentence production.

[AW]
> >> ...This happens when
> >> I simply conduct an argument or even speak aloud or write equations and
> >> notes to myself without *in any way* thinking or planning what I am saying.

[AS]
> >Your faith in your own beliefs about what's going on inside you is
> >touching. And misplaced.

[AW]
> I made no claims about what went on inside me. I said I didn't think
> or plan what I said.

Depends whether you are making the strong claim that there is NO
thinking or planning of which you are unaware, or whether you are
making the weaker and obvious claim (which is implied by what I said)
that the thinking or planning is not being done consciously by you.

[AW]
> ....Perhaps some *other* entity inside me does something
> like planning but that is irrelevant to the claim.

I took it that your phrase "*in any way*" was meant to rule out ANY
sort of thinking or planning.

If you are happy to allow that thinking or planning that you are unaware
of may be part of the process of speech production or writing then we
are not disagreeing. That's all I was saying: mental states and
processes can in you occur of which you are not aware.

[AW]
> ...the point concerned what happens "in
> my mind", and there there can be a complete blank.

So you want to exclude from your mind all the things that you are
not aware of?

If this is just a linguistic move (i.e. you define them not to be part
of your mind, and I define them to be part of your mind) then we have no
disagreement of any substance. [It's like the empty verbal dispute over
whether a circle is or is not an ellipse, when it is not clear which of
two concepts of "ellipse" is in question.]

In that case those of us who try to learn things about what's in our
minds that we are not aware of are simply not talking your dialect of
English.

If you really believe there could be a complete blank and NO explanation
in information processing terms for the phenomena of speech production
then you have a concept of process that I find mystifying.

I assume you don't believe linguistic performance is produced by
inexplicable magic.

So perhaps you have a very strong theory of direct implementation of
high level linguistic capabilities in neural mechanisms without any
intermediate virtual machines operating on semantic contents, making
plans, taking decisions, following rules.

I'll be very surprised if you can make such a theory work, but I am
willing to listen. I know there are some people who are trying to show
how minds are, for instance, dynamical systems and mental states are
analysable in terms of trajectories of state vectors, while others are
trying to show that there are neural implementations of consciousness,
etc.

But it does not sound as if that's what you are getting at. So I am
mystified. Maybe you are one of those believers in brute facts that need
no explanation and are simply inexplicable?

=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 8)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

Anders denies that anything going on at the sub-personal level can be
relevant to what it is to use symbols with specific meanings. In an
earlier message he wrote:

[AW]
> >> In these cases, we can say the meaning is all visibly carried in the
> >> overt activity with the public symbols,

I responded
[AS]
> >You lucky man. Never to have been totally misunderstood by those who
> >watched and heard you talk.

[AW]
> No but when I am misunderstood I try to clarify by further public
> explanations. I don't say: look at my control system, there you will
> find out what I really meant.

It read to me as if you were saying "just look at my overt activity.
There you will find out what I really meant". What else does
"visibly carried" mean?

Sometimes I can guess what a student means who is failing to express
himself clearly, because I have had experience of other students
getting confused at that stage of learning. I can then offer the
suggestion to the student, who then recognises it as fitting what
was in his mind.

The fact that we communicate about that in public does not mean that
I am talking about essentially public and "visibly carried"
phenomena. I.e. I think you've made in invalid inference.

[AS]
> > What does "meanings derive from X" mean anyway?
[AW]
> ....
> One answer is: an assertion derives its meaning from the meaning of inner
> mental states or of inner mental processes which precede it. This is
> what I am trying to reject. If there need be no mental processes preceding
> it, than it can not derive its meaning from these.

The real problem is that you are trying to answer a question about a
VERY complex system using a simple formula all in one go
instead of trying to see how it depends on layers of sophistication that
are themselves very complex.

You postulate two incredibly oversimplified answers as if they exhausted
the options, reject one then choose the other.

I don't believe either. I've given pointers earlier to some of my papers
about the architectural grounding of semantic capabilities which give
different more complex answers.

The important point for now, is that there's no way there could be a
social system with a rich public language unless there were individuals
within that system that had sophisticated information processing
architectures.

Some of the kinds of meanings in those systems might be sufficiently
primitive to depend only on the architectures. (E.g. think of perceptual
states in a fly?)

Others could depend on relationships between the contents of the
architecture and the embedding environment. Yet others depend on an
embedding social system.

The whole story is very complex, and requires a study of the
evolution of intelligence across species and the development of
language and intelligence within the young of our species and the
evolution of linguistic cultures.

Any one-line answer such as
    "meanings depend on states of the internal system"
    "meanings depend on the social environment"
is just oversimplifying obfuscation.

I've previously tried to point at the kinds of complexity required in
what I thought you might find a familiar class of system

[AS]
> >E.g.a computer controlling a chemical plant.

[AW]
> Still a completely mindless mechanism.

Have you ever thought about what's required to control a chemical plant
or a factory?

Have you thought about what would be needed to extend it to be a mindful
mechanism?

As explained previously, I am talking about a very sophisticated control
system or management system: including a large collection of capacities
(capabilities, powers, skills, know-how, and knowledge about
constraints, requirements, high level objectives, preference orderings
and languages of various sorts for internal and external communication,
e.g. with managers, operators, etc.).

One of the important points is that in such a complex system not all the
components can be active simultaneously. There may be competition for
control between components and which one wins at any time may depend on
obscure combinations of circumstances (including previous learning done
by the system). This is why external monitoring, checking and control
can be very difficult.

(Psychotherapy and teaching are very hard to do well.)

Again oversimplified theories are opposed here:

[AW]
> I take a mentalist theory which Ryle criticized to say: the performance
> is intelligent or not according as it has the right inner cause, e.g.
> proceeds from an intelligent plan. I take the alternative to be: the
> performance is intelligent or not according as it makes sense, e.g.
> presents a cogent argument, something that is autonomous and
> independent of its inner causes. And that there need not be any inner
> cause, for example no prior plan.

Both answers are so shallow as to deserve immediate rejection. Try
another alternative.

Try harder to understand the sort of control system I have been pointing
to. Don't just focus on the trivial and irrelevant examples.

=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 9)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).


Anders raises some points about abstraction and levels.

I wrote
[AS]
> >Nothing that I know about is totally autonomous: reality may come in
> >layers or levels, but there are deep interconnections between them.

[AW]
> There are all sorts of connections. For example coins are made of
> metal and will melt if heated. Still you can see why someone might
> claim that economic facts form a new stratum which in some way is
> autonomous -- independent of details of material realization.

Depending on the case, this could be right or it could be wrong.

For example, depending on the medium used for money you could have a
system that works very well or one that is very cumbersome (e.g. because
the "tokens" are too big and heavy), or which breaks down through fraud
(because the tokens are too easy to copy).

It can be very tempting to think that some abstract domain is totally
independent of its implementation and forget that even if that is true
in a general way particular cases may allow coupling between levels.

BASIC programs that include "peek" and "poke" are one type of example.

Another would be low level interrupt handlers or microcode routines that
are allowed to invoke user programs written in a high level language.

I don't think human minds and human language use completely separable
implementation levels: the effects of drugs on high level
decision making processes are an example.

(I know I have occasionally written in the past as if the levels were
completely separable. But I was wrong or simply being careless.)

[AW]
> Perhaps we need to get clear on what paramechanizing is. In my mind,
> any sort of control system explanation at any level short of the
> intentional one is literally mechanical.

Well then, either you are not using the word "literally" literally (a
common problem) or, more likely, you are not literally using the word
"mechanical" as many physics students and others are taught to use it.
E.g. in this usage mechanical systems are different from electronic
systems, and both would be different from virtual machines implemented
in software. A grandfather clock and spring driven watches would be
mechanical in this sense but not digital watches.

If your (literal) usage is so broad that it includes operations in a
lisp virtual machine, then things can be mechanical (in your sense) up
to a very high level in the implementation hierarchy.

If you allow virtual machines in which not just symbolic structures but
also information about things is processed (e.g. office information
systems, robotic systems) to be mechanical, then in that sense the mind
is literally mechanical.

I am not sure I see the point of using the word like that, but I won't
argue about it. It's yet another unimportant semantic point.

There is another (partly metaphorical?) sense of "mechanical" in which
people are sometimes mechanical (e.g. getting dressed when you are
familiar with the processes). I presume you did not mean "mechanical" in
that sense.

=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 10)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

Anders has argued that somehow the personal level is inherently
linguistic, and if it has any features that need to be explained they
are explained by social facts, by cultural norms, by canons of
rationality. Moreover, and most importantly, he takes first person
present tense utterances like "I believe that..." "I intend that..."
which may generate ethical and other commitments or obligations as
somehow central to and defining concepts like "believes that"
"intends..."

I've argued that this is simply an anthropocentric and distorted view of
mentality. It can be seen to be so in the light of all sorts of
biological facts, such as that many animals that don't use language
perceive things, can suffer, can be afraid, can be undecided, can be
curious, etc. It also excludes human infants and those with brain
damage.

[AW]
> I don't see that your observation is relevant to the idea that what is
> special about us is our ability to communicate in language. Sure some
> human beings are defective and can't develop it fully or at all.

I am not at all interested in what is special about us. If you are, then
why are you bothering to say all that in a philosophy of AI news
group, which is clearly concerned with something broader than human
mentality, and broader than linguistic capabilities?

AI has since the earliest days been concerned with the study of a wide
range of capabilities including vision, motor control, planning of
actions, and other things that have nothing to do with linguistic
communication. Of course, the study of architectures and mechanisms
underlying linguistic communication is part of AI also -- there are AI
natural language processing journals, conferences, workshops, conference
streams, labs, but all of them exist alongside other kinds of AI
activities.

Moreover, I have no idea what gives you the right to claim for the
philosophy of mind only those things that relate to linguistic
capability.

[AW]
> Your argument seems to me to have the same logical form as this: there is
> nothing special about human beings because human beings are made of
> the same chemicals as the rest of the world. As far as chemistry is
> concerned they are just one variety of complex aggregates of molecules,
> nothing particularly distinguished.

On the contrary: I don't, and AI doesn't, and philosophy of mind doesn't
define the subject matter in terms of chemistry or linguistic
capabilities, but in terms of a large (and still partly ill-defined)
collection of capabilities that I keep repeating: language, perception,
learning, motivation, emotions, concept formation, problem solving,
decision making, and many more.

And there is no reason to suppose that the use of a public language or
being part of a social system is central to or the basis of all those
capabilities.

If you want to focus on linguistic capabilities to the exclusion of
things that don't depend on them, then that's up to you. But don't claim
that everything else is excluded from philosophy of mind as studied by
others.

[AW]
> This is fine if your interests are in chemistry. Your argument is fine
> if your interest is control systems.

I've shown clearly that control systems do not define the topic but the
the sorts of capabilities the control systems are invoked to explain:
the ability to perceive, learn, notice, enjoy, etc. Control systems (in
a very general sense, which includes computational systems) are part of
an explanatory framework, not the definition of what is to be explained.

[AW]
> My interest, however, is in language, intentional states and
  ^^^^^^^^^^^
> philosophy of mind.

Well if you wish to have private and esoteric definitions of
"intentional" and "mind", excluding non-human animals and all the other
cases I keep listing, then that's a private oddity of your own.

It's not shared by AI, by psychology, nor by the vast majority of
philosophers of mind that I have encountered in the flesh or in books
and articles.

(After all, nobody, as far as I know, argued in response to Nagel's
paper "What is it like to be a bat?" that it's nothing because bats
can't talk. Nor did people say that was not philosophy of mind.)

Curiously, Anders goes on later to say:

[AW]
> I didn't say animals don't have mental states.

But you want to exclude the study of what makes such mental states
possible, or any other analysis of the types of mental states they can
have, from philosophy of mind?

[AW]
> I said there's a
> vast difference in the type and variety.

Of course there's a difference. Why do you think anyone would dispute
that? Call it "vast" if you will: I don't mind.

You also wanted to exclude the study of animal mentality from philosophy
of mind.

Certainly a COMPREHENSIVE philosophy of mind should explore whether all
things with mental states are exactly the same or nearly the same, or
vastly different, and also whether there is something that they all, or
various substates, have in common. Chemical compounds can differ vastly,
and yet they are all instances of a common physical architecture in
which a relatively small number of types of atoms exist which can bond
together to form larger structures using the same general principles.

Likewise mental processes might differ vastly despite all being
explained by information processing architectures of various kinds.

(Considering the stress Wittgeinstein put on anti-essentialist
arguments, e.g. there's no essence common to all games, it is very
surprising that such a staunch Wittgensteinian tries, apparently, to
argue that some aspect of human linguistic ability is the essence of
mentality.)

[AW]
> Don't you agree that thinking
> about quantum mechanics is simply impossible unless one has internalized a
> technique for the manipulation of its signs?

Yes -- but so what?

We were talking about philosophy of mind, and being able to do quantum
mechanics is not a necessary condition for having mental states and
processes. I know lots of youngsters who cannot think about quantum
mechanics: but they certainly have thoughts.

If you want to study the philosophy of science, or the philosophy of
language, or the philosophy of abstract thought or the philosophy of
adult western minds that's fine. Just say so, and others with different
concerns can ignore you, or they'll listen if you say something
interesting.

But defining those topics as circumscribing the philosophy of mind is
not something you can expect the rest of the world to agree to.

[AW]
> I wonder why you have no use for the idea that culture transforms us
> by "reprogramming"

Who said I have no use for that idea? I've said several times that I
think there may be different sorts of minds in different cultures.
Moreover, the use of even a metaphorical notion of reprogramming here is
consistent with the idea that the individuals in the culture are
information processing systems.

You seem to want everything to be simple: the complete explanation is
this, or it is that that, and no half measures, no complex combinations,
no subtle mixtures of levels. It's all social or it's all mechanical.
Why?

[AW]
> -- it is very prominent in Dennett's book on consciousness,
> and he's a friend of design stance, after all. As a scientist, don't you
> think that new notational devices are a marvelous thing?

I've written several papers on varieties of notational devices,
including ones you would not be interested in because you are only
interested in the *overt* public ones that are part of a set of social
means of communication.

Several of my papers on representation are in the Cognition and Affect
project directory:
    ftp://ftp.cs.bham.ac.uk/pub/groups/cog_affect/

The reason why I've thought and written so much about this is that I
think that understanding the varieties and forms of representational
systems is crucial to understanding the varieties and forms of
information processing systems.

But that includes all sorts of representational systems, not just public
languages used for communication between humans. It includes forms of
representations used in many types of virtual and physical machines.

However, I have no idea how you think the marvels of new notational
devices are relevant to the question whether animals have mental states
and processes.


[AW]
> These are all
> cultural items, like tools, which you can use mainly because you were
> taught to.

You have a very narrow notion of notational device. An animal that can
do a mating dance had better have some internal form of representation
that controls its behaviour when it does that dance. Similarly, a bird
that can bury 50 or more nuts, remember where it has buried them and
later on remember which ones are no longer there because it has eaten
them, must have some internal form of representation of the spatial
environment and what is located where in it. (I did not invent this
example: I attended a lecture about it by the Oxford biologist Krebs a
year ago.)

We don't yet know much about those forms of representations, but they
are certainly not cultural items, and as far as I know birds are not
taught to use them.

(They may be taught other things by watching or listening to other birds
such as forms of bird song, or how to get milk by pecking through foil
milk-bottle tops on doorsteps. But they are not taught how to use
systems implemented in their own brains.)

[AW]
> ..Your in-born control system gives you the innate potentiality
> of mastering these skills, but it sure doesn't account for your possession
> of them.

Of course not: Not on its own. Who would be so silly as to claim it did?

The cultural processes, external teaching processes, do not account for
your possession of new notations either. You can try teaching a dog or a
flea algebra till you are blue in the face and you'll get nowhere. The
cultural processes work only with individuals who have suitable
information processing architectures. Different kinds of learning
capacities can be explained in terms of those architectures.

But I agree that the innate architecture on its own does not achieve
everything. Why are you bothering to assert something so obviously true
as if it had been denied?

Especially when part of what you say is a recognition of what I am
talking about:
    "Your in-born control system gives you the innate potentiality
    of mastering these skills,"

Perhaps you don't understand what you have agreed to there, for you seem
to think it's like agreeing that the physical constitution of the brain
gives it its potentiality.

Everything physical has a physical constitution: yet not everything
physical can learn or perceive. Not everything physical implements an
information-processing control system.

(And not all information-processing control systems are defined in terms
of linguistically expressible self-descriptions as you seem to argue at
times.)

=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 11)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part  of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

We had a disagreement as to whether everything that's in the mind can be
expressed in behaviour.

> >[AW]
> >> ..we can say: animals can have those mental states that are
> >> expressible in behavior;
> >
[AS]
> >You cannot express in your behaviour anything remotely as rich as
> >your mental state on standing on a bridge watching the swirling
> >rapids of a river below. You can't even describe it except at a
> >fairly coarse level (J.L.Austin: "Fact is richer than diction").
[AW]
> That depends on what you mean. I can point at the swirling stream
> and say "it looks like that" and someone else who can see it can
> know what I mean.

Then the contents of the mental state are not expressed in your
behaviour but in the stream's behaviour.

I thought that in your original statement you were referring to the
animal's behaviour.

It may be worth noting that some of the very rich internal processing is
possible and useful because of the very high bandwidth transducers that
we have for input: the optical system in particular.

We do not have comparable high bandwidth transducers for output (though
posture, gesture, tone of voice and facial expression go a long way to
adding extra output bandwidth in subtle ways.)

The auditory system, less obviously, also absorbs and processes
information faster than any available output. E.g. you can listen to and
enjoy a whole symphony orchestra, but not output anything as rich as the
auditory experience.

(A few people, e.g. Mozart, seem able later to write down what they have
heard. Most cannot.)

[AS]
> >It's not the *fine* grained stuff that we express in language. ..

[AW]
> I meant it enables us to make fine-grained distinctions among concepts,
> for example among co-extensive ones. We can say that an owl is after
> a field-mouse, but there are limits if we start to ask just how the
> owl conceives of the field mouse (as a mouse? as dinner? as tasty?).

I agree with that. Anyone who wants to do owl anthropology the way we do
human anthropology will be a loser. Some of the reasons are in my half
serious paper: What is it like to be a rock? in the directory
    http://www.cs.bham.ac.uk/~axs/misc/

If you think there's no way ever to discover or describe the kind of
conceptual framework used by another kind of animal then you are simply
a scientific pessimist.

It may be difficult, but there is no reason to think it's impossible.

It will be impossible to *translate* owl concepts to human concepts, but
that's another matter. You can produce a metalinguistic description of
the structure of a set of concepts that you cannot translate into your
own.

One of the requirements for a comprehensive cognitive science (or AI),
not yet met by a long way, is production of a conceptual framework for
talking in a detailed and deep way about conceptual frameworks in other
kinds of agents, including ones with which we cannot communicate and
never will be able to except in the shallow way we already communicate
with cats and dogs, for instance. (Much richer communication is possible
with a bonobo, e.g. things like "Take the ball outside" "Put my shoes in
the refrigerator")


More denial of the relevance of an internal control system to explaining
mental capabilities followed later on, after I had pointed out that we
could be either ignorant or wrong about our some of our own mental
states, e.g. not noticing that you are infatuated or angry.

[AW]
> I *was*, however, denying that the fact in question was a fact about
> your "control system" which cognitive science might discover, in the
> way it discovers that your visual system exploits binocular disparity.

This is a curious opposition, like many that Anders makes. The detection
and use of binocular disparity in depth perception IS part of the
functioning of an information processing control system.

[AW]
> The fact that you really love someone may simply be implicit in your
> behavioral dispositions. The fact that you deny it may be implicit in
> your verbal behavioral dispositions. When there is a mismatch it is a
> mismatch between what you say and what you do (the manner in which you
> behave).  That is *not* a mismatch between what you say and an internal
> representation.

Well that's your theory.

On my theory the behaviour and dispositions to behave are to be
explained (partly) in terms of information of various kinds within a
sophisticated control system architecture. And the internal states may
be richer in their content than any behaviour or even any behavioural
dispositions of the whole system, because aspects of the information
content of sub-systems may be relevant to what's going on in your mind
(e.g. rich perceptual processes which cannot be displayed
behaviourally).

The information processing involved in the use of binocular disparities
to infer depth information is not inherently of a different kind from
information processing involved in selecting patterns of behaviour in
response to the smiles of the object of infatuation.

[AS]
> >Exactly: most of what's in your mind is inaccessible to you. ...

[AW]
> This is crazy; that would show it is not in my mind at all (in the
> sense I use this phrase).

OK We've already agreed that you have a lot of idiosyncratic concepts
(including "mind").

Most other people are quite happy to talk about someone not knowing his
own mind, or about people discovering things about what they really
feel, what they really believe, what they really want.

It is also commonplace that there's lots we know that we cannot access.
If you don't want to call that part of your mind or say it's in your
mind that's up to you.

[AW]
> It might be in my control system, but
> that is granted by everyone.

Is it? That's news to me.

[AW]
> ....You have to learn to distinguish genuinely
> subjective states from control system states, it seems to me.

You have to learn that there is more in common between the states you
call mental and the states you don't than you have hitherto appreciated
(e.g having information content, having control functions) and that this
commonality is part of an explanation of how mentality is possible,
and that the possibility of different kinds of mentality can be
explained by different sorts of underlying control system architectures.

This does not dispute the claim that some aspects of some sub types of
mentality depend on a social system.

[AS]
> >flautist can put his/her fingers on the flute and play a top G sharp
> >but if you ask which fingers have to be up and which down when that
> >happens may not be able to tell you.

[AW]
> Indeed this is knowing how, not knowing that. So what?

So both involve having stored information which is accessible and
usable in a practical context if not in the context of answering a
question.

The difference between what you can access at will and what you cannot
is far less important for a unifying explanatory theory of mentality
than the commonality.

The explanation of varieties of types of knowing how is part of
philosophy of mind.

[AS]
> >But that doesn't mean the information is not stored there in a form
> >in which it is useful for performance (e.g. when sight reading new
> >music which includes a top G sharp).
[AW]
> No one denied it might in some sense be stored in there.

So - semantic content can exist even though you cannot access it.

Good, that's progress. I was sure you denied exactly that.


[AS]
> >                        That you find such information about
> >yourself alien does not imply that anyone else does. ....

[AW]
> Information about how the digestion of food is effected in my stomach is
> interesting. It has an important role in my life and even affects my
> mental life. I may be very curious about it and take a passionate interest
> in finding out about it.  But I would never say it is something that is
> going on "in my mind" or is a mental process.

Why should anyone say it is? Is there some store of information about
how the digestion occurs? Is that information accessed, transformed,
updated, used? If not, it is not an example of what we are talking
about.

Actually, there is a control system involved in digestion and some of it
depends on cooperation with bacteria that you contain but which are not
part of you. So not all the control systems within you are part of the
architecture I am talking about. Roughly speaking, they are part of the
maintenance system of the physical vehicle in which the control system
is implemented.

(More work needed here.)

[AW]
> The computations performed by my visual system have a similar role in
> my mental life. I can take an interest in them, they have effects on
> my mental life, but they are not elements in my mental life, I would have
> thought.

They are not elements in your *conscious* mental life and we are all
agreed on that. However they are different from chemical processes in
your colon because they include manipulation of semantic information
(e.g. about depth, texture, optical flow).

If your digestive system manipulates information at all then it is only
as a relatively separate subsystem. Whereas the perceptual information
processing is deeply connected with other things you know, need to know,
want to do, etc.

Moreover under some circumstances you can access a lot of information
about the contents of your visual representations (e.g. such as the
acute and obtuse angles in your visual field corresponding to the right
angles at the corners of the 3-D table top.

I am not claiming that there is any *sharp* separation between the
information processing in you that is part of your mind and that which
is not. There may be many reflexes on the periphery for which there's no
clear answer using our current conceptual framework, which did not
evolve to be part of a comprehensive theory.

I have moved some of Anders' discussion of digestion to follow on from
here. It originally came much later on.

[AW]
> If someone were to call the knowledge that the colon digests food
> knowledge of the mind, I think you would agree they were confused.
> If someone were to call knowledge of the computations of the visual
> system knowledge of our mind, I say they are confused in the same
> way. But perhaps you could explain what makes the latter "mental".


> Can't you understand the claim that those computations do not
> form part of my mental life, in the way that being conscious of
> my office clearly does?

If by "my mental life" you mean "what I am aware of by myself" then
obviously what you say is true.

However (a) you have no right to try to restrict philosophy of mind to
that.

Isn't the word "phenomenology" already there for that that subset of
philosophy of mind?

(b) Nothing follows as to whether your *concepts* of belief, desire,
perception, decision, intention, etc. can or cannot be analysed solely
in terms of things you are aware of or know about.

Anyone who rejects simple-minded empiricism will know that our concepts
can have deep structures that are not necessarily accessible to us.

[AW]
> Almost anyone who thinks there is a "problem of qualia" produces nothing
> beyond a circular structure of absolutely worthless Cartesian intuitions.
> They philosophize as if Wittgenstein never existed and still worry that
> science must in some way "explain" consciousness. If one understands the
> giants of the 1950s one sees that there is no problem, no mystery, and
> *nothing* for science to explain here.

I agree with you that most of the writing about qualia and consciousness
is thoroughly muddled, but not for these reasons. See the files
        consciousness.lecture.ps
        consciousness.rsa.text
in the directory
    http://www.cs.bham.ac.uk/~axs/misc

Unlike you (and Dennett) I think there is something to be done about
qualia which is in the area of overlap between science and philosophy.

Namely there is something about our mental states and processes which
people are (in a very confused way) trying to talk about when they talk
about qualia.

The job to be done is to come up with a correct analysis and to show why
it is INEVITABLE that any well designed intelligent robot with human
like capabilities will have qualia.

I.e. for such a system the "it could be a zombie" form of argument is
just wrong.

(This is where I part company with Dennett. He tries to brush all this
under the carpet by saying there's no such thing as qualia, just because
a lot of what's said about them is confused.)

[AW]
> In seriousness, I have said many times I think that control system
> science is a real and worthy pursuit. Unlike philosophy, one can get
> results there. I expect the main reasons philosophers jump ship to the
> empirical disciplines is that the light is better there, not because
> they really have what they're looking for.

Why do you label adopting the design stance "jumping ship to the
empirical disciplines"?

There's a deep sense in which the study of possible designs and their
implications is no more empirical than mathematics or philosophy. It is
the exploration of what's possible, which can go far beyond the
empirical study of what happens to exist in this world. To that extent
it has far more in common with philosophy than with physics. Of course,
theoretical physics also has an element of this exploration of what's
possible and that's why there has always been an overlap between physics
and philosophy.

But I expect you'll refuse to call any such work philosophy because it
does not fit your narrow definition of philosophy.

[AW]
> I am only wondering about
> the *relation* of this pursuit to the study of *mental* states proper.

I.e. what's "proper" is what fits YOUR preconceived mode of analysis.
Everything else is "empirical", "not philosophy", "irrelevant".

Well feel free to ignore it, but don't waste your time attacking it.
Moreover, come back some time later and maybe you'll see how it is able
to provide deep explanations of some of the fine structure of what you
regard as relevant, which you are content to accept as inexplicable
fact.


[AW]
> I do think the idea that *computers* might think is as confused as
> thinking one day maybe base metals could be transmuted into gold -- it
> betrays a misunderstanding of the relevant concepts. But of course a
> robot might think. Perhaps more attention to the real problems of robotics
> that operate in real environments would direct the field into more
> valuable lines of research.

More attention than what? A lot of people in AI do think about these
problems and many of them who don't work on actually building robots
know a lot about what the problems are and work on fragments of them
including fragments that can be studied in totally simulated
environments. Meanwhile some of those who claim to be studying the
"real" problems because they work with "real" robots are merely studying
toy side issues because of the technical limitations of currently
available mechanisms.

But that's another long story.


=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 12)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part  of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

Anders brought in the concept of responsibility as central to his
concept of mental life.

[AW]
> I happen to think the relevant notion of mental life -- indeed the
> notion of consciousness itself -- is largely defined by the concept of
> responsibility.

No doubt given time you could define such a concept. Feel free.

But if you define such a concept and it excludes lots of aspects of
mentality that other people have been talking about for years (e.g.
perceptual processes that we may share with other animals), then don't
expect your preference for a new definition to carry much weight with
others.

It might if you could show it was part of a deeply explanatory theory.
But, like W. you seem to eschew explanatory theory because of your
concern to stick to a very narrowly defined practice of philosophy.

[AW]
> In practice I bet you recognize the distinction I am
> referring to. I have a certain sort of responsibility for my doings and
> even for the correctness and cogency of my private reasonings. I cannot
> be held responsible for a failure of my visual system; at most, for
> failing myself to take due care of my visual system or due notice of
> its liabilities. It is for this reason I say the goings on in the
> visual system are alien -- I don't "own" them the way I do my mental
> states.

Yes, I think I recognize the distinction.

Like many others you are assuming that there is a sharp distinction
between what you are and what you are not responsible for which, it is
well known, crumbles under close investigation, because you can always
shift the responsibility further back in time or deeper into the
mechanisms or back into the social system that produced you, etc..

On the other hand within the framework of an explanatory architecture,
and only within such a framework, I think I can provide an account of
those actions and decisions which flow from your own desires interests,
preferences etc. as opposed to those which are out of your control.
(Dennett's book, Elbow room, begins to give an account of this.)

In any case, this notion of responsibility has nothing to do with the
general notion of mentality. It's an old idea that we might be
responsible for some of our thoughts and desires and not others.

[AW]
> I think this notion is is the basis of the difference in our attitudes
> towards people or animals and mechanisms.

The basis of your attitudes perhaps. I know lots of people who praise
and blame their cats and dogs, and sometimes even punish them in anger
exactly as they punish an errant person.

The basis of my attitude to those I regard as normal responsible
individuals is the assumption of a type of architecture, functioning in
a certain way, which makes it possible to distinguish actions that come
from their desires and preferences from those which do not.

For people who don't have a concept of such an architecture the notion
of acting freely or of being responsible can and often does evaporate.

[AS]
> >students sign up for degrees in psychology because they (mistakenly)
> >think they will learn lots of deep things about how their minds work
> >which are not directly accessible to them.

[AW]
> Now *I* get to say: so what, if these students are mistaken?

Of course they are mistaken. The crucial point is that if you were
correct it would be incoherent to what to learn those things. It isn't
incoherent. It's just not what they are taught, on the whole.

[AW]
> But seriously: what the mistaken students are after is *not* information
> about how distance is computed from binocular disparity, is it?

Well that may be part of what the brighter ones want. It's certainly
what many of them get. (Have you looked at the content of degree courses
in psychology?)

Some will also want to know things about how motives are formed, about
how attitudes, personality, mood and other things develop and interact
with one another.

All of that requires a theory of the underlying architecture within
which those states coexist and interact. Unfortunately many
psychologists try to translate all these things into behavioural
characteristics that can be measured and expressed in tables and graphs.

They don't understand what a design for a behaving system is.

Anders thinks that what is not behaviourally definable must be
physically verifiable by observing the brain using appropriate
instruments. I objected that this would not enable us to find out about
the higher level virtual machine operations.

[AS]
> >What kinds of instruments would enable you to discover that the
> >computer on your desk is running a prolog program that is parsing
> >French sentences and interpreting them as querying a database?
> >
> >What kinds of instruments would tell you that it was running an
> >artificial neural net that implements a highly parallel content
> >addressible memory that allows items to be fetched very rapidly on
> >the basis of partial matches between queries and stored information.
> >"Suzie had a little goat, its fleece...."

[AW]
> It would be hard to *discover* these fact using only electronics testers.
> But the facts you cite are *realized* in electronics and so ultimately
> verified by the state of the electronics, albeit under a high-level
> description.

I think you are confusing the notion that the implementation is there in
the electronics with the notion that they are *verified* by the state of
the electronics. The latter implies the ability to check a relationship
between the contents of a high level virtual machine specification and
the physical implementation. If you don't have access to the compiler
you may not be able to do that.

Even if you do have such access you may not be able to do it if the
stuff is stored internally in an encrypted form and only decoded as
necessary.

[AW]
> .That is all I mean when I say instruments are required --
> presumably you *can't* conclusively verify these claims by looking at
> input-output behavior alone.

I don't think you can *conclusively* verify them any way at all. But
that doesn't bother me because I don't accept the verifiability theory
of meaning. Do you?

You are making another invalid opposition between mental states that
can be checked behaviourally and control system states that can be
checked by examining the brain.

On the one hand mental states cannot be checked behaviourally except in
limited special cases, and on the other hand high level virtual machine
states cannot be checked by examining brains.

This again reduces the difference you keep trying to erect between the
two.
=======================================================================
Newsgroups: comp.ai.philosophy
References: <4rcir5$ioh@usenet.srv.cis.pitt.edu> <4s01eg$pin@sun4.bham.ac.uk> <4s0uos$p6l@usenet.srv.cis.pitt.edu> <4s4blh$kkj@sun4.bham.ac.uk> <4s71cm$i83@usenet.srv.cis.pitt.edu>
Subject: Control systems and philosophy of mind (Part 13)

The full text of which this is part is at
    http://www.cs.bham.ac.uk/~axs/misc/control.systems.and.minds

This is part  of my reply to Anders Weinstein <andersw+@pitt.edu> who
wrote on 13th July in response to my very long critique of his
restriction of philosophy to the NPR (normal personal realm i.e. adult
human western culture normal personal realm).

The discussion of the nature of high level virtual machines continued
with some interesting points.

[AW]
> I should reiterate my query about virtual machines from another post.
> If you take a very high level view, you can say that the fact that I
> understand English is a behavioral fact about me at which *every*
> detail of my control system is irrelevant.

But part of the detail is which lexicon you have stored, and which items
of information are associated with different lexical items, and which
grammatical and other rules you use and which strategy you use for
parsing complex sentences etc. All of these are details of your control
system (in the general sense which includes the high level information
processing virtual machine implemented in your brain) and all of them
are relevant to identifying and explaining your linguistic capability.

They are not part of the information that is directly available to you,
and they are not easily read off behaviour (which is why linguists and
psycholinguists are in disagreement about their contents). Their status
is partly similar to that of theories about unobservables in physics.
But they are mental, in that they are concerned with information and
intentional content (e.g. what you refer and know about, whether you
know you know it or not, can play a role in the processes). They
definitely are not physical or physiological.

[AW]
> I know it because I have
> nearly enough the right dispositions to use English words in public
> circumstances.

But you do not know which words you know or what you know about most of
them or what grammatical words you know, etc. etc. E.g. most English
speakers have and use knowledge about tag questions. However, when they
are told they have it, they are pretty surprised.

[AW]
> I communicate with other speakers because we have been
> shaped by socialization to share enough of these dispositions.

What sort of *because* is this? Is it meant to provide a complete
explanation? Or is it meant to explain the difference between those
humans that can talk and those who can't?

It's a bit like saying a 747 flies because the pilot switched on the
engine and manages the controls. It ignores the mechanisms and physical
principles that make it possible for such a machine to fly at all.

[AW]
> .This is
> the Quine-level notion of "understands English" after all, a level at
> which nothing inside the head is relevant.

You keep saying that nothing inside the head is relevant.
That does not make it true. Even Quine's agreement does not make it
true (despite the hero-worship he receives in some quarters).

I've given you several examples of what's inside you (though not a
physical object in your head) and which you cannot access directly and
which is relevant to exactly what you know about English and how you use
the knowledge.

You can find out a lot more by trying to do research on natural language
processing in humans and machines.

[AW]
> But *you* are evidently talking about a lower level, a control system
> level which *enables* the dispositions that constitute understanding
> English.

I keep wondering whether the sole problem is that your notion of
possible types of control system is too limited. I find that hard to
believe because of your familiarity with computers and software.
But maybe that is the sole problem.

Anyhow I am talking about a complex control system inside which
includes factual information and control information and mechanisms and
strategies for manipulating them, of which you are mostly unaware.

I am saying that without taking account of these components and the
information they contain, i.e. without adopting the design stance, you
cannot give an adequate analysis of the space of language-capable
agents.

More generally without adopting the design stance you cannot give an
adequate analysis of the space of possible types of mental states and
processes.

If, for purely personal reasons concerned with what interests you,
you do no not want to go beyond the surface level characterisation of
normal adult western human language and mind then nobody would force you
to pay attention.

But if you are trying to argue that everything else is outside the
territory of philosophy of mind then your arguments have failed, unless
you simply define philosophy of mind in your own private way.

Moreover if you are trying to argue that everything outside the realm
that interests you is irrelevant to explaining how things within that
real work, even when they work normally, then that's just wrong. There
are existence proofs in linguistics, psychology and AI.


Next Anders again tries to appeal to a dichotomy: everything is either
concerned with dispositions of the whole system or else physical and
conclusively detectable by physical instruments. He excludes complex
information processing virtual machine architectures of the sort I am
talking about.

It begins to look as if his reasons are to do with a desire for
conclusive verifiability of theories. Either things should be
conclusively verifiable by behavioural tests or they should be
conclusively verifiable by physical examination of the internals.
E.g. he writes

[AW]
> If you
> are not making a claim about a virtual machine constituted entirely by
> the dispositions, then the dispositions alone only provide indirect
> evidence for its existence and some sort of probe or instrument is
> needed to conclusively confirm hypotheses about it, I would have thought.
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Well if you think that such complex theories can ever be *conclusively*
verified, and if you think there is a requirement for *conclusive*
verification in science (or philosophy) then I think you just have an
incorrect epistemology.

A full discussion of that will take us far afield, but the topic is well
discussed in many text books on philosophy of science. Most scientific
theories are neither conclusively verifiable nor falsifiable, and most
interesting scientific concepts are not definable in terms of
conclusively verifiable tests for their instances.

Could it really be that your whole position is so closely tied to a
naive verificationist theory of meaning?

[AW]
> Even if the instruments alone cannot generate those hypotheses. This is
> why I say knowledge of control system facts -- unlike knowledge that someone
> understands English -- would require instruments. It is because the
> control system is hidden, not because I want to describe it in
> low-level terms.

Not everything that's hidden can be checked by physical instruments. If
I present you with a previously unknown computing system on which there
is no available documentation, and I ask you whether it contains an
algorithm for computing square roots of complex numbers then there may
be no behavioural test nor internal physical examination that can
possibly answer the question.


It may be that internal observations will HELP you check out claims
about control system facts. That's certainly possible, provided that you
already know a great deal about how the system works, how its memory is
organised, etc.

But that should not be confused with conclusive verification.

Moreover, in general checking out what's happening in a high level
virtual machine is not possible unless the machine is "instrumented" at
its own level, e.g. in the case of software that often means having
additional tracing and debugging aids at the level of the programming
language's virtual machine (so called "source level" debuggers are crude
examples -- interpreted languages often allow more sophisticated and
flexible instrumentation).

But sometimes the instrumentation changes the behaviour. So it then
remains a matter of inspired guesswork to track down what's actually
going on in the original system, and you may never be absolutely sure
you've got it right (unless you were the original designer and
implementor)

(That's part of the reason for the so-called software crisis).
[end]