[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
issue COMPILER-LET-CONFUSION, version 2
- To: cl-compiler@sail.stanford.edu
- Subject: issue COMPILER-LET-CONFUSION, version 2
- From: sandra%defun@cs.utah.edu (Sandra J Loosemore)
- Date: Fri, 7 Oct 88 11:18:56 MDT
Here is a slightly revised writeup on this issue.
Issue: COMPILER-LET-CONFUSION
References: CLtL p. 112
Category: CHANGE/CLARIFICATION
Edit History: V1, 27 Sep 1988, Sandra Loosemore (initial version)
V2, 04 Oct 1988, Sandra Loosemore (add another example)
Status: **DRAFT**
Problem Description:
The description of the COMPILER-LET special form in CLtL is confusing
to many people. There are no examples provided to make it clear how it
is supposed to be used.
The description of how COMPILER-LET works in the interpreter only
makes sense for implementations which perform macroexpansion in
parallel with evaluation. In an implementation which performs macro
expansion in a prepass and follows CLtL literally in making
COMPILER-LET behave "exactly like LET with all the variable bindings
implicitly declared SPECIAL", it would not work at all for its stated
purpose, "communication among complicated macros".
Subtle bugs can be introduced because of the different handling of the
variable bindings in the interpreter and the compiler. In compiled
code, the bindings are only lexically visible during the expansion of
macros at compile time, while in interpreted code the bindings have
dynamic scope and may also be seen during ordinary evaluation if
evaluation and macroexpansion happen concurrently.
Further compatibility problems can result from the value forms being
evaluated in a null lexical environment in the compiler and the
ordinary lexical environment in the interpreter.
Proposal COMPILER-LET-CONFUSION:CLARIFY-STATUS-QUO:
Change the documentation of COMPILER-LET to include more discussion of
its intended purpose, including examples.
Change the description of COMPILER-LET's behavior in the interpreter
to indicate that it is not "exactly like LET with all the variable
bindings implicitly declared SPECIAL". Instead, clarify that
processing of COMPILER-LET by the interpreter happens at the same time
as macroexpansion; the value forms are evaluated sequentially
in a null lexical environment, bound to the special variables, and
that these bindings are available during the expansion of macro calls
in the body.
Mention that there may be compatibility problems between compiled and
interpreted code if the interpreter performs macroexpansion in
parallel with evaluation.
Rationale:
This proposal formalizes what appears to be the behavior of most
implementations.
Current Practice:
COMPILER-LET is rarely used.
Cost to implementors:
Minimal.
Cost to users:
None. Any code that would break under the new rules for the interpreted
behavior of COMPILER-LET would already be broken when compiled.
Benefits:
The description of COMPILER-LET would make more sense for implementations
that perform macroexpansion during a prepass in the interpreter. Some of
the compatibility problems between interpreted and compiled code would be
removed.
Proposal COMPILER-LET-CONFUSION:REQUIRE-PREPASS:
Disallow interpreters from performing macroexpansion concurrently with
evaluation. Macroexpansion must be performed in a preprocessing
codewalk.
Clarify that COMPILER-LET is processed by the interpreter or compiler
(the "processor") at the same time that macros are expanded. The
processor sequentially evaluates each of the value forms in a null
lexical environment, and the special variables are bound to these values
during the expansion of any macro calls in the body.
Rationale:
This proposal forces the interpreter to treat both macroexpansion and
COMPILER-LET in the same way that the compiler does.
Current Practice:
COMPILER-LET is rarely used.
Cost to implementors:
For implementations that do not implement their interpreters using a
prepass, this would be a fairly substantial change.
Cost to users:
Strictly speaking, this is a compatible change to the language. Since
CLtL makes no guarantees about the time of macroexpansion, any user
code that depends upon macroexpansion happening concurrently with
evaluation is nonportable. Some users, however, dislike
implementations with preprocessing interpreters, and want to be able
to freely redefine macros as well as functions.
Benefits:
An area of incompatibility between compiled and interpreted code is
eliminated. This proposal would also address some other compatibility
problems beyond those relating to COMPILER-LET (such as issue
LOAD-TIME-EVAL).
Proposal COMPILER-LET-CONFUSION:ELIMINATE:
Remove COMPILER-LET from the language.
Rationale:
Some people think that COMPILER-LET is ugly. Removing it from the
language is a much simpler solution than an elaborate proposal to
assign it consistent semantics in the interpreter and compiler.
Current Practice:
COMPILER-LET is rarely used.
Cost to implementors:
Minimal. Implementations could continue to support COMPILER-LET as
an extension.
Cost to users:
People who use COMPILER-LET would have to rewrite their programs to use
some other construct. Most uses of COMPILER-LET for communication between
macros can be handled using MACROLET instead.
I have been able to do this quite easily for all of the examples which I
have seen so far. For example:
(defvar *local-type-declarations* '())
(defmacro local-type-declare (declarations &body forms)
`(compiler-let ((*local-type-declarations*
(append ',declarations *local-type-declarations*)))
,@forms))
(defmacro typed-var (var)
(let ((type (assoc var *local-type-declarations*)))
(if type `(the ,(cadr type) ,var) var)))
(defun f (x y)
(local-type-declare ((x fixnum) (y float))
(+ (typed-var x) (typed-var y))))
can be rewritten as:
(defmacro local-type-declare (declarations &body forms)
(local-type-declare-aux declarations forms))
(eval-when (eval compile load)
(defun local-type-declare-aux (declarations forms)
`(macrolet ((typed-var (var)
(let ((type (assoc var ',declarations)))
(if type `(the ,(cadr type) ,var) var)))
(local-type-declare (new-declarations &body new-forms)
(local-type-declare-aux
(append new-declarations ',declarations)
new-forms)))
,@forms)))
The MACROLET versions are usually somewhat more complicated than the
COMPILER-LET versions, but not substantially so unless there are a large
number of macros involved.
Another approach for converting old code is to define a
COMPILER-LET-like macro to explicitly make the special variable
bindings available during the expansion of a specified set of macros.
The following macro FAKE-COMPILER-LET provides semantics fairly close
to those of proposal COMPILER-LET-CONFUSION:REDEFINE (below), the
exception being that it handles only macros defined with DEFMACRO and
not MACROLET.
;;; Imitation COMPILER-LET. Second argument is a list of macros which
;;; are to see the special bindings; these must have been defined using
;;; DEFMACRO, not MACROLET (because expansion takes place in null
;;; lexical environment).
(defmacro fake-compiler-let (binding-forms macros &body body)
(expand-fake-compiler-let binding-forms macros body))
(eval-when (eval compile load)
(defun expand-fake-compiler-let (binding-forms macros body)
(let* ((vars (mapcar #'(lambda (b)
(if (consp b) (car b) b))
binding-forms))
(vals (mapcar #'(lambda (b)
(if (consp b) (eval (cadr b)) nil))
binding-forms))
(binders (mapcar #'(lambda (var val)
`(,var ',val))
vars vals))
(defs (mapcar #'(lambda (m)
`(,m (&whole w)
(let ,binders
(declare (special ,@vars))
(macroexpand-1 w))))
macros)))
`(macrolet ((fake-compiler-let (binding-forms macros
&body body)
(let ,binders
(declare (special ,@vars))
(expand-fake-compiler-let
binding-forms macros body)))
,@defs)
,@body)))
)
;;; Example to illustrate nesting behavior
(eval-when (eval compile)
(defvar *depth* 0)
(defmacro current-depth ()
*depth*)
)
(fake-compiler-let ((*depth* (1+ *depth*)))
(current-depth)
(format t "First value = ~s~%" (current-depth))
(fake-compiler-let ((*depth* (1+ *depth*)))
(current-depth)
(format t "Second value = ~s~%" (current-depth))))
Benefits:
Having one less special form would simplify the language. An area of
incompatibility between compiled and interpreted code would be
eliminated.
Proposal COMPILER-LET-CONFUSION:REDEFINE:
Clarify the description of COMPILER-LET to emphasize that it is used to
make bindings of special variables available when macro calls appearing in
the body are expanded.
Clarify that COMPILER-LET is processed by the interpreter or compiler
(the "processor") at the same time that macros are expanded. The
processor sequentially evaluates each of the value forms in a null
lexical environment, storing the variable/value pairs in the
environment.
Extend the description of environment objects captured with &ENVIRONMENT
to state that they also contain information about bindings indicated with
COMPILER-LET.
Clarify that MACROEXPAND and MACROEXPAND-1 use the information stored in
the environment argument to actually perform the special bindings of
the variables to the values. Clarify that calling a macro-function
directly may result in incorrect behavior.
Clarify that the special bindings indicated by COMPILER-LET are also
visible during the evaluation of value forms of lexically nested
COMPILER-LET constructs. The special bindings are not present during
any other compile-time evaluation of the body, such as evaluation of
(EVAL-WHEN (COMPILE) ...) forms.
Rationale:
This proposal provides consistent semantics for COMPILER-LET in both the
compiler and interpreter. It will work just as well in interpreters that
perform macroexpansion in parallel with execution, as in those that perform
macroexpansion in a prepass.
Current Practice:
COMPILER-LET is rarely used.
Cost to implementors:
Implementors will have to change their interpreters and compilers to
have COMPILER-LET store things in the environment. The representation
of environment objects may need to change slightly. Implementations
in which &ENVIRONMENT does not return a complete environment object
will have to be changed. MACROEXPAND and MACROEXPAND-1 will have to
be changed to rebind the variables from the environment.
Cost to users:
User programs that perform code walks may have to be changed; the
changes would roughly parallel those listed above.
Benefits:
The treatment of this special form by the interpreter is made consistent
with its treatment by the compiler.
Discussion:
Can any of you implementors who have an interpreter that perform
macroexpansion in a preprocessing phase comment on how you handle
COMPILER-LET?
Proposal COMPILER-LET-CONFUSION:CLARIFY-STATUS-QUO is the alternative
which is the most consistent with current practice and requires the
least change for both implementors and users. However, it addresses only
some of the problems which can lead to incompatibility between compiled
and interpreted code.
Proposal COMPILER-LET-CONFUSION:REQUIRE-PREPASS would ensure
consistent semantics for macroexpansion as well as processing of
COMPILER-LET. On the negative side, it is a substantial change for
some implementations.
Proposal COMPILER-LET-CONFUSION:ELIMINATE is by far the simplest
alternative, but it would also has the greatest impact on users.
Proposal COMPILER-LET-CONFUSION:REDEFINE also guarantees consistent
semantics for COMPILER-LET between the interpreter and compiler. The
disadvantages of this proposal are that COMPILER-LET remains
complicated and potentially confusing, and it requires changes in both
user and system code.
Proposal COMPILER-LET-CONFUSION:ELIMINATE appears to have the most
support among members of the compiler committee; JonL (and others at
Lucid), Loosemore, Van Roggen, Dalton, and Perdue have all expressed
support for the idea of getting rid of COMPILER-LET. Since this is an
incompatible change, it is felt that a substantial effort should be
made to help users convert old code.
COMPILER-LET-CONFUSION:REQUIRE-PREPASS was suggested by Pitman and
COMPILER-LET-CONFUSION:REDEFINE by JBarnett.
-------