Course 287 Lecture 10: Processes and Proofs

Iterative Processes and Tail Recursion
A proof that fact_iter produces the same result as factorial
Recursive and iterative processes for factorial.
Automation of such proofs
The Fibonacci Numbers - A simple definition gives exponential complexity
Evaluating (apply f list) function explicitly applies a function f to a list of arguments

Abelson & Sussman

The material on linear processes is discussed in section 1.2, [p.31ff.]. This includes the computation of the factorial function and of Fibonacci numbers.

Iterative Processes and Tail Recursion

A computational process is "the pattern of actions a computing device takes". A program directs these actions. Let's compare the following definitions of the factorial function:

(define (factorial n)
   (if (= n 0) 1 (* n (factorial (- n 1)))))

(define (fact_iter n product)
    (if (= n 0) product
        (fact_iter (- n 1) (* n product) )

(define (fact_1 n) (fact_iter n 1))

Both are recursive functions. factorial is a linear recursive process, that is, there are deferred computations that grow linearly in n.

fact_1 is an iterative process. there are no deferred computations. It is a linear iterative process.

A function, written recursively, is an iterative process if each recursive call occurs at the top level of a conditional. This arises from the fact there is no need to defer a computation, or alternatively, the working-space to hold the values of variables can be reused. Consider:

    (factorial 4)
    ==> (* 4 (factorial 3))
    ==> (* 4 (* 3 (factorial 2)))
    ==> (* 4 (* 3 (* 2  (factorial 1))))
    ==> (* 4 (* 3 (* 2  1)))
    ==> 24

because the recursive calls of factorial are nested inside a computation, we have to remember what that computation was in order to carry on with it.

On the other hand, with:

    (fact_1 4))
    ==> (fact_iter 4 1)
    ==> (fact_iter 3 4)
    ==> (fact_iter 2 12)
    ==> (fact_iter 1 24)
    ==> (fact_iter 0 24)
    ==> 24

the apparently recursive calls are not nested, do there is no need to remember what we were doing, and we can re-use the same space.

A proof that fact_iter produces the same result as factorial

This proof assumes that the Scheme objects which represent numbers do in fact behave like numbers, so that for example (* x 1) = x. For Scheme integers this is a reasonable assumption, since as we recall, they are of arbitrary precision.


For all integers n>=0, (factorial n) = (fact_iter n 1)

Discussion. This theorem relates the functions factorial and fact_1. However the work of computing the factorial is done by fact_iter. So, in order to prove this theorem, we are going to have to prove something about fact_iter, and, since it is a recursive function, we are going to have to prove it by induction. The most obvious thing we might try to prove is that

       (fact_iter n 1)  = (factorial n)

However this clearly is not going to work, since in the computation of (fact_iter n 1) for a given n we are going to call (fact_iter i j) where i ranges from 1 to n and j ranges from 1 to (factorial n). So we need to prove something more complicated. Well, it we think about it, (fact_iter n m) is going to multiply m by n, (- n 1) (- n 2) ..... 1, that is it is going to multiply m by (factorial n). So we can formulate the following lemma (a lemma is a theorem whose main purpose is to support the proof of a more important theorem - just as fact_iter is a helper function for fact_1 so our lemma is a "helper-theorem".


For all integers n>=0, m>=0
(fact_iter n m) = (* (factorial n) m)

Proof: by induction on n

Base case: n=0

         (fact_iter 0 m) = m = (* (factorial 0) m)

Inductive Step: Suppose for some k, all m

        (fact_iter k m) = (* (factorial k) m)
      Consider: (fact_iter (+ k 1) m)
    = (fact_iter k (* (+ k 1) m))       by definition of fact_iter
    = (* (factorial k) (* (+ k 1) m))   by inductive hypothesis
    = (* (* (factorial k) (+ k 1)) m)   by associativity
    = (* (factorial (+ k 1)) m)         by definition of factorial
                                            and commutativity of *
So the result holds for (+ k 1)

Hence by induction, the result holds for all m. Hence the Lemma holds.

Proof of Theorem:

    (fact_1 n) = (fact_iter n 1)    by definition of fact_1
    = (* (factorial n) 1)           by Lemma
    = (factorial n)                 by rules of algebra.

[Note, here is a proof of the inductive step in a notation closer to the traditional mathematical notation]

Suppose for some k, all m fact_iter(k,m) = k!*m


    = fact_iter(k, (k+1)*m)      by definition of fact_iter
    = k! * ((k+1) * m)           by inductive hypothesis
    = (k! * (k+1)) * m           by associativity of multiplication
    = (k+1)! * m                 by definition of factorial, commutativity
                                 of multiplication.

Automation of such proofs

The proof above is quite capable of being automated. The hard part is guessing the right lemma, and that may require human intervention.

The Fibonacci Numbers - A simple definition gives exponential complexity

The Fibonacci numbers are a well known sequence of numbers:


The first two Fibonacci numbers are 0 and 1.

Each Fibonacci number (after the second) is the sum of the two preceding Fibonacci numbers. We can quite easily write a function to compute the n'th Fibonnaci number using this property:

(define (fib n)
    (if (< n 2)
        (+ (fib (- n 1)) (fib (- n 2)))

Note that we are counting from zero, so

    (fib 0) = 0
    (fib 1) = 1
    (fib 2) = 2

However, consider the calculation:

    (fib 5)
    ==> (+ (fib 4) (fib 3))
    ==> (+ (+ (fib 3) (fib 2))  (+ (fib 2) (fib 1)))
    ==> (+ (+ (+ (fib 2) (fib 1)) (+ (fib 1) (fib 0)))
                           (+ (+ (fib 1) (fib 0)) 1))

    ==> (+ (+ (+ (+ (fib 1) (fib 0)) 1) (+ 1 0))
                           (+ (+ 1 0) 1))

    ==> (+ (+ (+ (+ 1 0) 1) (+ 1 0))
                           (+ (+ 1 0) 1))

    ==> 5

Observe: At each stage we are doubling the number of deferred computations, so that to calculate (fib n) we would perform 2n deferred computations.

Also: We are recalculating (fib n) for each n many times.

We can recast this computation in iterative form by building up the sequence of Fibonacci numbers successively from the beginning until we have reached the one we need. This clearly takes n operations. We will need only to remember two successive Fibonacci numbers at each stage. We use two accumulators which are (fib (- n 1)) and (fib (- n 2)).

(define (fib_it n acc1 acc2)
    (if (= n 1) acc2
        (fib_it (- n 1) acc2 (+ acc1 acc2))

And now we can define fib_2 which uses fib_it as a helper function to compute the n'th Fibonacci number:

(define (fib_2 n)
    (if (= n 0) 0
        (fib_it n 0 1)

Consider the evaluation of:

  (fib_2 5)
==> (fib_it 5 0 1)
==> (fib_it 4 1 1)
==> (fib_it 3 1 2)
==> (fib_it 2 2 3)
==> (fib_it 1 3 5)
==> 5

We could also improve the performance of fib by memoisation, which will be discussed later in the course. This is quite painless for the progammer:

    (define fib (memoise fib))

and can achieve O(n log(n)) complexity, not quite as good as the O(n) complexity of the iterative reformulation above, but it involves no recasting of the original algorithm.

Currying allows arguments to be specified one at a time.

In Scheme, a function with n arguments must be given all its arguments if it is to work correctly.

(define curried_+
    (lambda (m)
        (lambda (n) (+ m n))))

(define add_2 (curried_+ 2))
(define add_3 (curried_+ 3))

(add_2 56)  ==> 58
(add_3 4)   ==> 7

Note - we are forced into this definition of curried_+ by the peculiarities of Scheme. The Haskell language, named after Haskell Curry, makes no distinction between the curried and un-curried forms. Neither does the lambda calculus. POP-11 provides an intermediate capability - for example maplist(%sqrt%) is the POP-11 version of the map function with the sqrt function curried in, that is maplist(%sqrt%) will map a list of numbers into their square-roots.

The apply function

Scheme has a built-in function apply. This is defined so that:

      (apply f x1 x2....xn l)

is equivalent to

       (f x1 x2.....xn l1 l2 .. lm))

where l1 .. lm are the elements of the list l.

(example '(apply + 3 4 5 '()) 12)
(example '(apply + '(3 4 5)) 12)