re: Unbound variable

[Robert W. Kerns <rwk@crl.dec.com>]

    Date: 02 Aug 91 23:04:06 EDT
    From: Louis Van Zandt <76234.1133@compuserve.com>
    Regarding your question on:

	  Welcome to Macintosh Common Lisp Version 2.0b1!
	  ? (foo x)
	  > Error: Unbound variable: X
	  > While executing: SYMBOL-VALUE
	  Is it normal?

    Yes it is normal. When you hit the return key the read-eval-print loop takes
    over. Before any arguments are passed to the function or macro they are
    evaluated. Since your x was unbound an error had to be generated. If you had x
    bound, it would have been evaluated and that value would have been assigned to
    the function or marco parameter.

    Before macro expansion can take place it must have valid parameter values. If
    it didn't it would expand in to an unstable form.

I'm afraid this explanation is rather confused and erroneous.

When evaluating a form, the FIRST thing done, in ALL versions of
Common Lisp (and virtually all other Lisp dialects), is to examine
the CAR of the form, to see if it is a macro or special form.

The arguments to a macro are *NEVER* evaluated before passing them
to a macro expander function.  (Generally, interpreters won't evaluate
any arguments to special forms, either; instead, they leave that up
to the individual function which implements that particular special
form.)

If the CAR is not a macro or special form, *THEN* it evaluates
all the arguments, and passes them off to the function.  More on
this later.

I really have been unable to figure out what kind of thing you would
consider an "unstable form", so I can't address that directly.

But the original form, in its, er, original, unaltered form, is the
first argument to any macro expander.  (The lexical (syntactic)
environment is the second).  Whether an argument to a macro is evaluated
or not is purely a matter of what the macro expansion does with it.  If
it's to be evaluated, it will be placed somewhere in the expansion in a
position to be evaluated.

For example

(defmacro ichiban (cons)
  `(car ,cons))          ; = (list 'car cons)

[The names are deliberately obscure; look at the code instead.]

places the CONS argument in a place to be evaluated in the expansion.

(By the way, things like this, which could be functions, generally
SHOULD be functions, not macros, but it makes a simple example).

But consider:

(defmacro hakoniirete (contents)
  `(make-instance 'box :contents ',contents))

 ;; = (list 'make-instance ''box ':contents (list 'quote contents))

This does not evaluate contents, because it places the CONTENTS
argument to the macro in a context where it will not be evaluated,
i.e. inside a call to QUOTE.

That is:  (hakoniirete (hana mushi))
==> (make-instance 'box :contents '(hana mushi))

thus putting the flowers and bugs into the box without
evaluating them.

If the model you proposed for lisp evaluation were correct, then
there could be no IF, COND, or QUOTE.  And macros would be useless.

Well, back to the original question.  If Lisp has to look at
the CAR of the form first, then why didn't he get an error about
an undefined function?

Well, what is going on is this:  The interpreter is being
consistent with what the compiler does.  And the compiler
does something slightly different than you might expect,
for efficiency.  (The language definition allows this).

(In fact, in some environments, the interpreter may CALL the
compiler.  But that doesn't matter here).

To understand WHY it *APPARENTLY* didn't check the function
until after it evaluated the arguments, we need to understand
a bit about compilation.  Bear with me, it's not hard, and
it will get back to the real topic soon.

In compiling Lisp, one of the things which happens is that
macros and special forms are handled by the compiler.  Macros
are expanded, and special forms are turned into special machine
code.  For example, IF is turned into conditional branches.

(IF (< X 5)
    (PRINT 'SMALL)
  (PRINT 'BIG))

becomes something like:

(TAGBODY
   (IF-FALSE (< X 5) (GO FALSE))
   (PRINT 'BIG)
   (GO END)
   FALSE
   (PRINT 'SMALL)
   END)

So in compiled called, all that remains are sequences of instructions,
and function calls.  The compiler does not know whether a particular
function will be defined later (although it may warn you if it thinks
it might not be).  So the checking for undefined functions happens when
you run the function, not when you compile it.

But checking for macros and special functions happened much earlier,
back when you compiled the function.  So this is still consistent
with what I said earlier, about checking for macros and special
forms *FIRST*.  But this wasn't the compiler, it was the interpreter,
right?  Well, bear with me a bit more; there's one more step.

How does the compiler compile calls to functions?  Well, this varies
depending on the particular machine.  Common Lisp allows two different
techniques, which are slightly visible to the user.

The more obvious technique, from the user's point of view, is strictly
left-to-right.  First, it gets the function, then it evaluates the
arguments, and then it applies the function to the arguments.  In
Lisp, this looks like:

(funcall (symbol-function 'PRINT) (EVAL 'BIG))

and indeed, many interpreters work this way.  But in the actual
compiled code, it looks a bit different:

SYMBOL-FUNCTION'PRINT
PUSH D0
PUSH BIG
CALL @-1(SP)  ;; Where the PRINT function got pushed.
POP

Some Lisps compile this way.  But on many machines, there's a faster
way:

PUSH BIG
SYMBOL-FUNCTION PRINT
CALL (D0)

That is, in many situations, it is one or two instructions FASTER
to not look at the function until after the arguments are evaluated.

Furthermore:  Even if the function *IS* looked at first, the error
may not occur when the compiled code does SYMBOL-FUNCTION.  Instead,
it may not occur until later, when the CALL instruction is actually
done.  Often, unbound functions are marked with routine, which when
called, signals an unbound-function error.

So, back to our interpreter question.  We can start to see what
happened:

1)  The interpreter looked at the CAR, and determined it was not
a macro or special form.

2)  It wasn't, so the interpreter evaluated the arguments.  Or
at least tried to, but one of them got an error.

3)  If step 2 had succeeded, it would have tried fetching and
invoking the function, and gotten an error on that.

Since this is the same thing the compiler would have done, we
can't know whether the interpreter invoked the compiler to do
this, or just did the same thing, unless Apple tells us, or we
look under the hood.  It doesn't matter; since compilers are
allowed to not signal the error until after evaluating, so are
interpreters.

Yes, the error is a bit confusing.  However, we felt that the
difference in performance was enough to warrent leaving this
up to individual compilers.

Anyway, I'm hoping to give a talk on Macrology (the Art and Science
of Lisp Macros) at the Lisp Users and Vendors conference this
fall.  (It's not definite yet, but I have the outline).  I hope
you'l come.

I won't be talking about implementation-level stuff like how compilers
work, like I did here.  That was just necessary to explain why the
language is the way it is, and not how to use it.

Instead, I'll be talking about how to use macros well, in ways which
make your code more readable, and give you more programming power with
less work.

I'll start with how to think about backquote to make it easy, and
work my way up to powerful techniques, style conventions, and how
to keep it all clean and understandable.

As I said, I have an outline already, I'm willing to change it.  If
anyone has issues they'd like to see addressed, or areas they have
always found confusing about macros, or best of all, favorite examples,
please send them my way.