19. Macros

19.1 Introduction to Macros
If eval is handed a list whose car is a symbol, then eval inspects the definition of the symbol to find out what to do. If the definition is a cons , and the car of the cons is the symbol macro , then the definition (i.e. that cons) is called a macro . The cdr of the cons should be a function of one argument. eval applies the function to the form it was originally given. Then it takes whatever is returned, and evaluates that in lieu of the original form. Here is a simple example. Suppose the definition of the symbol first is
(macro lambda (x) 
         (list 'car (cadr x)))
This thing is a macro: it is a cons whose car is the symbol macro . What happens if we try to evaluate a form (first '(a b c)) ? Well, eval sees that it has a list whose car is a symbol (namely, first ), so it looks at the definition of the symbol and sees that it is a cons whose car is macro ; the definition is a macro. eval takes the cdr of the cons, which is a lambda expression, and applies it to the original form that eval was handed. So it applies (lambda (x) (list 'car (cadr x))) to (first '(a b c)) . x is bound to (first '(a b c)) , (cadr x) evaluates to '(a b c) , and (list 'car (cadr x)) evaluates to (car '(a b c)) , which is what the function returns. eval now evaluates this new form in place of the original form. (car '(a b c)) returns a , and so the result is that (first '(a b c)) returns a . What have we done? We have defined a macro called first . What the macro does is to translate the form to some other form. Our translation is very simple--it just translates forms that look like (first x) into (car x) , for any form x . We can do much more interesting things with macros, but first we will show how to define a macro. Macros are normally defined using the macro special form. A macro definition looks like this:
(macro name  (arg )
    body )
To define our first macro, we would say
(macro first (x)
    (list 'car (cadr x)))
Here are some more simple examples of macros. Suppose we want any form that looks like (addone x) to be translated into (plus 1 x) . To define a macro to do this we would say
(macro addone (x)
   (list 'plus '1 (cadr x)))
Now say we wanted a macro which would translate (increment x) into (setq x (1+ x) . This would be:
(macro increment (x)
    (list 'setq (cadr x) (list '1+ (cadr x))))
Of course, this macro is of limited usefulness. The reason is that the form in the cadr of the increment form had better be a symbol. If you tried (increment (car x)) , it would be translated into (setq (car x) (1+ (car x))) , and setq would complain. You can see from this discussion that macros are very different from functions. A function would not be able to tell what kind of subforms are around in a call to itself; they get evaluated before the functions ever sees them. However, a macro gets to look at the whole form and see just what is going on there. Macros are not functions; if first is defined as a macro, it is not meaningful to apply first to arguments. A macro does not take arguments at all; it takes a Lisp form and turns it into another Lisp form . The purpose of functions is to compute ; the purpose of macros is to translate . Macros are used for a variety of purposes, the most common being extensions to the Lisp language. For example, Lisp is powerful enough to express many different control structures, but it does not provide every control structure anyone might ever possibly want. Instead, if a user wants some kind of control structure with a syntax that is not provided, he can translate it into some form that Lisp does know about. For example, someone might want a limited iteration construct which increments a symbol by one until it exceeds a limit (like the FOR statement of the BASIC language). He might want it to look like
(for a 1 100 (print a) (print (* a a)))
To get this, he could write a macro to translate it into
(do a 1 (1+ a) (> a 100) (print a) (print (* a a)))
A macro to do this could be defined with
(macro for (x)
  (cons 'do
	(cons (cadr x)
	      (cons (caddr x)
		    (cons (list '1+ (cadr x))
			  (cons (list '> (cadr x) (cadddr x))
				(cddddr x)))))))
Now he has defined his own new control structure primitive, and it will act just as if it were a special form provided by Lisp itself.
19.2 Aids for Defining Macros
The main problem with the definition for the for macro is that it is verbose and clumsy. If it is that hard to write a macro to do a simple specialized iteration construct, one would wonder how anyone would write macros of any real sophistication. There are two things that make the definition so inelegant. One is that the programmer must write things like "(cadr x) " and "(cddddr x) " to refer to the parts of the form he wants to do things with. The other problem is that the long chains of calls to the list and cons functions are very hard to read. Two features are provided to solve these two problems. The defmacro macro solves the former, and the "backquote" (" `  ") reader macro solves the latter.
19.2.1 Defmacro

Instead of referring to the parts of our form by "(cadr x) " and such, we would like to give names to the various pieces of the form, and somehow have the (cadr x) automatically generated. This is done by a macro called defmacro . It is easiest to explain what defmacro does by showing an example. Here is how you would write the for macro using defmacro :

(defmacro for (var lower upper . body)
  (cons 'do
	(cons var
	      (cons lower
		    (cons (list '1+ var)
			  (cons (list '> var upper)
				body))))))
The (var lower upper . body) is a pattern to match against the body of the macro (to be more precise, to match against the cdr of the argument to the macro). defmacro tries to match the two lists
(var lower upper . body)
and 
(a 1 100 (print a) (print (* a a)))
var will get bound to the symbol a , lower to the fixnum 1 , upper to the fixnum 100 , and body to the list ((print a) (print (* a a))) . Then inside the body of the defmacro , var, lower, upper, and body are variables, bound to the matching parts of the macro form.
defmacro Macro
defmacro is a general purpose macro-defining macro. A defmacro form looks like
(defmacro name  pattern  . body )
The pattern may be anything made up out of symbols and conses. It is matched against the body of the macro form; both pattern and the form are car 'ed and cdr 'ed identically, and whenever a symbol is hit in pattern , the symbol is bound to the corresponding part of the form. All of the symbols in pattern can be used as variables within body . name is the name of the macro to be defined. body is evaluated with these bindings in effect, and is returned to the evaluator.
Note that the pattern need not be a list the way a lambda-list must. In the above example, the pattern was a "dotted list", since the symbol body was supposed to match the cddddr of the macro form. If we wanted a new iteration form, like for except that it our example would look like
(for a (1 100) (print a) (print (* a a)))
(just because we thought that was a nicer syntax), then we could do it merely by modifying the pattern of the defmacro above; the new pattern would be (var (lower upper) . body) . Here is how we would write our other examples using defmacro :
(defmacro first (the-list)
    (list 'car the-list))

(defmacro addone (form)
   (list 'plus '1 form))

(defmacro increment (symbol)
   (list 'setq symbol (list '1+ symbol)))
All of these were very simple macros and have very simple patterns, but these examples show that we can replace the (cadr x) with a readable mnemonic name such as the-list or symbol , which makes the program clearer. There is another version of defmacro which defines displacing macros (see LINK:(displacing-macro)). defmacro has other, more complex features; see LINK:(defmacro-hair).
19.2.2 Backquote
Now we deal with the other problem: the long strings of calls to cons and list . For this we must introduce some reader macros . Reader macros are not the same as normal macros, and they are not described in this chapter; see LINK:(reader). The backquote facility is used by giving a backquote character ("←` ", ASCII code 140 octal), followed by a form. If the form does not contain any use of the comma macro-character, the form will simply be quoted. For example,
 '(a b c) ==> (a b c)
 `(a b c) ==> (a b c)
So in the simple cases, backquote is just like the regular single-quote macro. The way to get it to do interesting things is to include a use of the comma somewhere inside of the form following the backquote. The comma is followed by a form, and that form gets evaluated even though it is inside the backquote. For example,
(setq b 1)
`(a b c)  ==> (a b c)
`(a ,b c) ==> (a 1 c)
In other words, backquote quotes everything except things preceeded by a comma; those things get evaluated. When the reader sees the `(a ,b c) it is actually generating a form such as (list 'a b 'c) . The actual form generated may use list , cons , append , or whatever might be a good idea; you should never have to concern yourself with what it actually turns into. All you need to care about is what it evaluates to. This is generally found to be pretty confusing by most people; the best way to explain further seems to be with examples. Here is how we would write our three simple macros using both the defmacro and backquote facilities.
(defmacro first (the-list)
    `(car ,the-list))

(defmacro addone (form)
   `(plus 1 ,form))

(defmacro increment (symbol)
   `(setq ,symbol (1+ ,symbol)))
To finally demonstrate how easy it is to define macros with these two facilities, here is the final form of the for macro.
(defmacro for (var lower upper . body)
  `(do ,var ,lower (1+ ,var) (> ,var ,upper) . ,body))
Look at how much simpler that is than the original definition. Also, look how closely it resembles the code it is producing. The functionality of the for really stands right out when written this way. .C New part starts here. If a comma inside a backquote form is followed by an "atsign" character ("SAIL@ "), it has a special meaning. The "SAIL,SAIL@ " should be followed by a form whose value is a list ; then each of the elements of the list are put into the list being created by the backquote. In other words, instead of generating a call to the cons function, backquote generates a call to append . For example, if a is bound to (x y z) , then `(1 ,a 2) would evaluate to (1 (x y z) 2) , but `(1 ,@a 2) would evaluate to `(1 x y z 2) . Here is an example of a macro definition that uses the "SAIL,SAIL@ " construction. Suppose you wanted to extend Lisp by adding a kind of special form called repeat-forever , which evaluates all of its subforms repeatedly. One way to implement this would be to expand
(repeat-forever form1  form2  form3 )
into
(prog ()
    a form1 
      form2 
      form3 
      (go a))
You could define the macro by
(macro repeat-forever body
       `(prog ()
            a ,@body
              (go a)))
Advanced macro writers sometimes write "macro-defining macros": forms which expand into forms which, when evaluated, define macros. In such macros it is often useful to use nested backquote constructs. The following example illustrates the use of nested backquotes in the writing of macro-defining macros. This example is a very simple version of defstruct (see LINK:(defstruct-fun)). You should first understand the basic description of defstruct before proceeding with this example. The defstruct below does not accept any options, and only allows the simplest kind of items; that is, it only allows forms like
(defstruct (name )
     item1 
     item2 
     item3 
     item4 
     ...)
We would like this form to expand into
(progn
 (defmacro item1  (x)
      `(aref ,x 1))
 (defmacro item2  (x)
      `(aref ,x 2))
 (defmacro item3  (x)
      `(aref ,x 3))
 (defmacro item4  (x)
      `(aref ,x 4))
 ...)
Here is the macro to perform the expansion:
(defmacro defstruct ((name) . items)
     (do ((item-list items (cdr item-list))
	  (ans nil)
	  (i 0 (1+ i)))
	 ((null item-list)
          (cons 'progn (nreverse ans)))
       (setq ans
	     (cons `(defmacro ,(car item-list) (x)
			   `(aref ,x ,',i))
		   ans))))
The interesting part of this definition is the body of the (inner) defmacro form: `(aref ,x ,',i) . Instead of using this backquote construction, we could have written (list 'aref x ,i) ; that is, the ",', " acts like a comma which matches the outer backquote, while the ", " preceeding the "x " matches with the inner backquote. Thus, the symbol i is evaluated when the defstruct form is expanded, whereas the symbol x is evaluated when the accessor macros are expanded. Backquote can be useful in situations other than the writing of macros. Whenever there is a piece of list structure to be consed up, most of which is constant, the use of backquote can make the program considerably clearer.
19.3 Aids for Debugging Macros
mexp
mexp goes into a loop in which it reads forms and sequentially expands them, printing out the result of each expansion. It terminates when it reads an atom (anything that is not a cons). If you type in a form which is not a macro form, there will be no expansions and so it will not type anything out, but just prompt you for another form. This allows you to see what your macros are expanding into, without actually evaluating the result of the expansion.
19.4 Displacing Macros

Every time the the evaluator sees a macro form, it must call the macro to expand the form. If this expansion always happens the same way, then it is wasteful to expand the whole form every time it is reached; why not just expand it once? A macro is passed the macro form itself, and so it can change the car and cdr of the form to something else by using rplaca and rplacd ! This way the first time the macro is expanded, the expansion will be put where the macro form used to be, and the next time that form is seen, it will already be expanded. A macro that does this is called a displacing macro , since it displaces the macro form with its expansion. The major problem with this is that the Lisp form gets changed by its evaluation. If you were to write a program which used such a macro, call grindef to look at it, then run the program and call grindef again, you would see the expanded macro the second time. Presumably the reason the macro is there at all is that it makes the program look nicer; we would like to prevent the unnecessary expansions, but still let grindef display the program in its more attractive form. This is done with the function displace .

displace form expansion
form must be a list. displace replaces the car and cdr of form so that it looks like:
(si:displaced original-form  expansion )
original-form is equal to form but has a different top-level cons so that the replacing mentioned above doesn't affect it. si:displaced is a macro, which returns the caddr of its own macro form. So when the si:displaced form is given to the evaluator, it "expands" to expansion . displace returns expansion .
The grinder knows specially about si:displaced forms, and will grind such a form as if it had seen the original-form instead of the si:displaced form. So if we wanted to rewrite our addone macro as a displacing macro, instead of writing
(macro addone (x)
   (list 'plus '1 (cadr x)))
we would write
(macro addone (x)
   (displace x (list 'plus '1 (cadr x))))
Of course, we really want to use defmacro to define most macros. Since there is no way to get at the original macro form itself from inside the body of a defmacro , another version of it is provided:
defmacro-displace Macro
defmacro-displace is just like defmacro except that it defines a displacing macro, using the displace function.
Now we can write the displacing version of addone as
(defmacro-displace addone (form)
   (list 'plus '1 form))
All we have changed in this example is the defmacro into defmacro-displace . addone is now a displacing macro.
19.5 Advanced Features of Defmacro
(To be supplied.) (The basic matter is that you can use &optional and &rest with defmacro . The interactions between &optional 's initialization, and the fact that the "lambda-list" in defmacro can be arbitrary list structure are not clear. If you need to use this feature, try it out.)
19.6 Functions to Expand Macros
The following two functions are provided to allow the user to control expansion of macros; they are often useful for the writer of advanced macro systems.
macroexpand-1 form &optional compilerp
If form is a macro form, this expands it (once) and returns the expanded form. Otherwise it just returns form . If compilerp is t , macroexpand-1 will search the compiler's list of intenally defined macros (sys:macrolist ) for a definition, as well as the function cell of the car of form . compilerp defaults to nil .
macroexpand form &optional compilerp
If form is a macro form, this expands it repeatedly until it is not a macro form, and returns the final expansion. Otherwise, it just returns form . compilerp has the same meaning as in macroexpand-1 .