Re: Issue: DOTTED-MACRO-FORMS (Version 1)

[goldman@vaxa.isi.edu]

First, thank you guys for getting this issue considered and keeping the
ball rolling.  Since I guess I am the one who suggested that it should
be dealt with,  I think I should at least state why I thought dotted
macro forms should not be outlawed out of hand.  

1) Like Larry, I don't like macro extensions that encourage source code
programs to have a "dotted list" representation.  But I don't think I
understand why I don't like them well enough to feel comfortable
making them officially "discouraged".  It may be that it is only the
PRINTED REPRESENTATION I don't like -- that pretty printers I have used
don't seem to format them the way I would choose to, and that the "."
doesn't stand out sufficiently and I miss it when looking at the
printed representation.

2) The situation that brought this issue to my attention had to do with
programs generated not by people, but by an "automatic programmer"
(or a hairy set of macros, if you prefer a more earthy characterization.)
The dotted macro form was not intended to ever be written or seen by a
human programmer; just processed by the compiler if it happened to be
produced by these other macros.  Here is a synopsis.  See if you think
it meets Larry's characterization as "occasionally ... just the right thing.":

Our automatic programmer produces and compiles high-level queries to a
database.  A query is representated by a well-formed-formula (WFF).  A
primitive WFF consists of a list -- (<relation-name> arg1 ... argn) --
e.g. (GRADE student course grade).

These primitive WFFs can be negated, combined with AND and OR, and
embedded in quantifiers.  So a compound WFF might be:
"did s1 outperform s2 in any course?"
 (exists (course g1 g2) (AND (GRADE s1 course g1)
	                     (GRADE s2 course g2)
	                     (GRADE> g1 g2)))

The syntax we chose for posing a query to the database from a lisp program
is (?? . WFF).  Notice that the syntax for WFFs guarantees that they are
all pure lists, so (?? . WFF) will never be a dotted macro form.
[WFFs are also used in many other parts of our embedded language.]

We also have a "simplifier" for wffs.  It is logically possible for
a wff to be simplifiable to the point that its truth is independent of the
contents of the database -- e.g.,
  (OR (GRADE s c g) (NOT (GRADE s c g)))
is tautologically true, while other WFFs might be provably true or false
based on reasoning about constraints known about the data.  We needed
to represent two logical constants, and we chose to represent them by
the symbols TRUE and FALSE.  It turns out that these logical constants made
sense wherever WFFs made sense; in particular, 
  (?? . TRUE)
is a perfectly fine query; it is one that no programmer would ever
write DIRECTLY, but might be produced by some automatic programmer tool.
We defined the macroexpander for ?? so that (?? . TRUE) macroexpands to T,
and (?? . FALSE) to NIL.  But now we had introduced the possibiltiy of
dotted macro forms, and discovered, when testing, that various implementations
were inconsistent (sometimes even self-inconsistent) in their treatment of
these forms.


3) I don't remember whether I ever expressed my opinion on how this issue
would be best resolved.  My apologies if the following is a repetition of
an earlier submission.

  I do think that support for portable dotted macros should be provided.
I don't think it would be so bad (in fact, I think it would be GOOD) if
the "normal" case (the macro cannot be correctly used with a dotted
argument list) caused dotted lists to be caught and an error signalled by
Lisp's macroexpander, rather than expecting the programmer's expansion
code detect it.

  Pragmatically, I believe that most macros written with &rest are NOT
prepared to deal with "dotted" lists in the &rest variable (or atomic, 
non-nil, values of the &rest variable).  If invoked with the &rest variable
so bound, these macroexpanders will almost invariably either

a) produce an expansion which will propogate the dotted argument list
   to another macro or function (postponing detection of the error), or

b) cause an expansion-time error by doing car/cdr on a non list,
   which the programmer will trace back to being caused by the dotted argument
   list, or

c) (by poor programming) terminate cdring down the &rest list with a consp
   test  nstead of an endp check, masking the error entirely.

If LISP were REQUIRED to signal an error when an &rest macro parameter
"matched" a dotted list, or non-nil atom, then macro writers would not
have to check the condition themselves to get good error messages.

However, a declaration could be provided:
      (defmacro foo (x &optional y &rest z)
		(declare (allow-dotted z))
	...)
in which case an error would NOT be signalled on (foo x1 y1 z1 . z2)


I have not seen many actual uses of &WHOLE.  Certainly the same 
declaration could be used to control whether or not and &WHOLE parameter
could bind to a dotted list.  Is there anything about the use of &WHOLE
that would lead you to expect its "normal" usage would make a default of
"dotted lists are an error" less reasonable than for &rest ?
For instance, if it is mostly used in the idiom:
   (defmacro MW (&whole form)
     <record something about form>
     `(MW-INTERNAL ,. (rest form)))
where MW-INTERNAL is another macro, then it would be
better to let MW-INTERNAL be responsible for controlling whether
dotted arglists were acceptable, and never complain about binding them
to &WHOLE parameters.

----------------------------------------------------------------
Neil