DECLARE-MACROS (Version 2)

[Masinter.pa@Xerox.COM]

I responded to Moon's suggestions for wording correction. I added an
explicit endorsement at the end.

In the interest of shortening the issue, I removed the argument about
destructive MACROEXPAND hooks, which, I think, is more controversial
than the support that it raises.

In order to avoid the controversy of what is a declaration, a declare
expression, a declare form, or a list whose car is DECLARE, I reworded
the proposal to be more explicit as to the intent, which is to negate
one specific paragraph in CLtL.

I run some risk of making things worse, at least in your eyes, so you
might want to cast yours on the PROPOSAL section and respond to me if
you are unhappy.

!
Issue:        DECLARE-MACROS
References:   Declaration Syntax (p154)
Category:     CHANGE
Edit history: 22-Oct-87, Version 1 by Pitman
               9-Nov-87, version 2 by Masinter

Problem Description:

  It is permissible for a macro call to expand into a declaration and be
  recognized as such. This linguistic feature provides some useful
  flexibility, but has a number of disadvantages:

  * Operations on the executable portion of a body of code inside a 
    binding form (such as inserting an additional form) is a complicated
    operation. This is because one or more trial macro expansions must be
    done in order to pass over any declarations or documentation string
    and find the beginning of the body.

  * In order to find the end of the declarations, MACROEXPAND must be
    called until a non-macro form is seen or until a macro does not expand
    into a macro. In some interpreters which do macro expansion on the fly,
    this may cause inefficiency because macro expansion of the first form
    in a body must be done twice. In implementations where this is 
    optimized, the implementor may resent the fact that an optimization is
    needed in the first place.

  * Various code analysis tools have been shown to have been significantly
    slowed down by the need to expand macros in order to determine whether
    a binding is SPECIAL when analyzing a variable binding form. This is
    particularly serious when macro invocations are deeply nested; the
    number of macro expansions can be exponential in the depth of nesting
    unless a macro-expansion caching mechanism is added. 

  * User macros must be very careful about finding declarations in a body
    of code by doing proper macro expansion. Often, however, naive users
    don't realize this and so unknowingly write buggy code. This problem can
    be (and is) defined away as simply a programmer error, but this is a
    place where we could fairly straightforwardly redefine the language to
    better accommodate what has been shown to be a common expectation of the
    naive user.

Proposal (DECLARE-MACROS:FLUSH):

   Under this proposal, it would not be "permissible for a macro call to
   expand into a declaration and be recognized as such, provided that the
   macro call appears where a declaration may legitimately appear." (CLtL
   p. 154). Macros could not legitimately expand into declarations; the only
   valid declarations would be a list whose CAR is the symbol DECLARE.

   It would still be possible for a macro call to expand into a PROCLAIM
   form, however.

Rationale:

  The advantages provided by the ability to have a macro form expand into
  a declaration have been shown in practice to not be worth the price paid
  elsewhere in the language.

Current Practice:

  Most or all implementations support the old behavior even though few
  user programs probably need it.

  Some user macros are careful about finding declarations in a body of code
  by doing proper macro expansion, but some probably cheat and look just
  for explicit uses of DECLARE. The cheat probably works most of the time.

Adoption Cost:

  The nature of this change is such that implementations which did not
  choose to change would simply be supporting an implementation-dependent
  extension (except for some `minor' worry about destructive modification
  due to macro expanding while *MACROEXPAND-HOOK* is set to something
  which implemented displacing macros).

  In any case, there might be several places in which the interpreter,
  compiler, and system macros had knowledge about doing macro expansion
  before declaration processing. The change is not trivial, but most of
  its complexity is likely to be in finding the places which need change
  and not in making the actual change.

Benefits:

  The efficiency of some tools may be improved.

  User macros which must do minor surgery on bodies of code will be
  easier to write.

Conversion Cost:

  Most users probably do not write macros which expand into DECLARE forms
  so most users are probably not affected.

  Users who do exploit this feature probably know that they do. In any
  case, compilers could be made to detect cases where this feature is
  being exploited and warn about it.

  Rewrites must be devised on a case-by-case basis. A common sort of
  rewrite would take the form:

   Old code:  (DEFMACRO SPEEDY () `(DECLARE (SPEED 3) (SAFETY 0)))
   	      (LET (..bindings..) (SPEEDY) ..body..)

   New code:  (DEFMACRO SPEEDY-LET (BVL &BODY FORMS)
		`(LET ,BVL (DECLARE (SPEED 3) (SAFETY 0)) ,@FORMS))
	      (SPEEDY-LET (..bindings..) ..body..)

Aesthetics:

  This change simplifies the semantics of the language slightly and
  probably tends to better support the assumptions of naive programmers
  writing macros.

  In some cases involving complicated extensions to declarations, it
  may be slightly harder to express such extensions in a modular way.
  Experience thus far has shown such cases to be rare, however.

Discussion:

  Symbolics took an in-house poll of people who take advantage of the
  feature and it was generally agreed that in most cases where this
  feature is used at all, that it would be just as easy to work around
  using the sample rewrite techniques cited above.

  Moon `credits' Pitman for (against some opposition) pushing this
  `feature' down everyone's throats in the original CL design process.
  Pitman admits this was an expensive mistake. Moon and Pitman support
  this change as an important simplification to the language.

  The cleanup committee unanimously endorsed this proposal.