Issue: COMPILER-LET-CONFUSION (Version 6)

[Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>]

I agree this is looking better, but I think I might want to make a few
more changes [perhaps even add an option! -- see below] before sending
this out to X3J13. Rather than edit the proposal back and forth, I
though I'd just hit a few hot issues separately and see
what you think.
----------

  ...
  Having macros side-effect variables bound by COMPILER-LET is probably
  not a good idea in most situations anyway, since the order in which macros
  are expanded or the number of times they are expanded is not guaranteed.
  ...
  
In some implementations, the order of processing might be well-defined
(as an extension), and so assignment might be useful in such implementations.
Consider that if processing were known to be left to right, it might want
to guarantee behavior such as the following:

 (DEFMACRO WITH-FOO (VALUE &BODY FORMS)
   `(COMPILER-LET ((*FOO-USED* NIL))
      (MACROLET ((FOO ()
		   (DECLARE (SPECIAL *FOO-USED*))
		   (SETQ *FOO-USED* T)
		   '(LOCALLY (DECLARE (SPECIAL *FOO*)) *FOO*))
		 (COMPOSE-FOO-USER ()
		   (IF *FOO-USED*
		       `(LET ((*FOO* ,',VALUE))
			  (DECLARE (SPECIAL *FOO*))
			  (DO-IT))
		       `(DO-IT))))
	(FLET ((DO-IT () ,@FORMS))
	  (COMPOSE-FOO-USER)))))

 (LIST (SETQ X 1) (WITH-FOO (INCF X) (LIST X (FOO) (FOO))))
 => (1 (2 2 2))

 (LIST (SETQ X 1) (WITH-FOO (INCF X) (LIST X (FOO))))
 => (1 (2 2))

 (LIST (SETQ X 1) (WITH-FOO (INCF X) (LIST X)))
 => (1 (1))

Given our macroexpansion model, though, I think it's reasonable for us to
clarify that SETQ is forbidden for the sake of code portability. I couldn't
think of any non-trivial way to take advantage of SETQ given a non-left-to-right
expansion.

This will allow us to make some of the sections shorter since we needn't
get side-tracked talking about assignment.

  ...
  I believe this restriction causes unnecessary hair for implementors 
  and doesn't buy anything for users.  We'd be better off just coming
  out and saying explicitly that the behavior is unspecified.
  ...

The example you cite is not the best for seeing the issue. It doesn't fix
a problem, but it makes it much less visible. The case of
interest is not relying on a program special variable from within a
macro, but rather having a COMPILER-LET variable visible from within
a macro that isn't part of the same lexical scope. For example:

If LOOP does
  (COMPILER-LET ((*INSIDE-LOOP* T))
    ...expansion...)
and LOOP-FINISH does
  (DEFMACRO LOOP-FINISH ()
    (IF *INSIDE-LOOP* 
        `(GO LOOP::LOOP-FINISH)
	(ERROR "You are not inside LOOP.")))
then your error message will be poor if you do
  (LOOP ... (EVAL '(LOOP-FINISH)))
because *INSIDE-LOOP* might be T in the interpreter.

I admit the screw case is still there, but it's much harder to get to
this way. You can only get to it by doing the EVAL call within the body
of a macro or macrolet, so it's much more rare.

I also note that making LOOP always do MACROLET of LOOP-FINISH will
get around the problem reliably, but I point out that the expense may
be non-trivial if there are a zillion little macros that LOOP (or
whatever) needs to bind and that may never end up getting called.
COMPILER-LET is therefore more efficient for cases like this.

----------

The following interesting compromise situation occurred to me, though.
I'm still thinking about it, but I'll mention it just as an option and
maybe we can pursue it as an alternative if people like it...

Suppose that instead of referencing the value as a special, you had 
an operator COMPILER-SYMBOL-VALUE to get its value. [That could be a 
function, macro, or special form.] I wouldn't have a problem calling
a function to get this data because I still get to use the same basic
`shape' of code. Also, uses of COMPILER-LET are rare enough that a
bit of extra syntax is not overwhelming.

 (DEFMACRO mac (...) 
   ... (COMPILER-SYMBOL-VALUE name) ...)

 (new-COMPILER-LET ((name val))
   ...)

I don't know how other supporters of COMPILER-LET would feel about this.
It looks ok to me, though.

I can't think of any implementations in which this would be prohibitively
expensive to implement.

----------

  ...
  If COMPILER-LET were not part of the language, people wouldn't think in
  terms of rewriting COMPILER-LETs as MACROLETs; instead, they'd think of
  how to use MACROLET in the first place to solve their problems.
  ...

I disagree with this. By analogy, if LET didn't exist, people would think
in terms of using FLET to solve their problems. The problem is that people
don't think that way. People think in terms of containers (variables)
as places to put things, and procedures (functions) as things that do things.
They don't like to think of a container as being represented by a function
whose meaning is might be locally changed in some contexts. It's just not
natural. That doesn't mean there aren't people who can't think that way --
just that I think it varies a lot.

----------

The question about whether the extent of an &ENV argument is indefinite or
dynamic is interesting. Is there a cleanup to cover this? I agree that
CLtL is not technically ambiguous -- that is, I don't recall ever having seen
anything to make me assume it's dynamic -- but I've always assumed it anyway.
I think Genera implements it as dynamic. Is there any good reason to want
it to be otherwise? Is an issue on this topic perhaps warranted? We could
certainly make it be dynamic if people were going to use it, but if they're
not going to use it it seems a bit like needless overhead...