environment arguments and compiler contexts and ...

[Kim A. Barrett <IIM@ECLA.USC.EDU>]

To: cl-compiler
Re: environment arguments and compiler contexts and ...

This is kind of a "toss out some problems and see if anybody has any ideas"
kind of message.  It's probably going to be a bit stream-of-consciousness.

PROBLEM 1

While in general I think the change that CLOS implicitly made, requiring a
distinguishable &environment argument for the compiler, and basing certain
lookups based on it, there are some problems with this in actual use.  For
example, if I want to implement a DEFGENERIC macro, the obvious ways to do so
are 

    (defmacro defgeneric (spec lambda-list &rest options &environment env)
      (when (and (compiler-environment-p env)
                 (compiler-at-top-level-p env))
        (apply #'ensure-generic-function
	       spec
	       lambda-list
	       :environment env
	       (parse-defgeneric-options options)))
      `(apply #'ensure-generic-function
	      ',spec
	      ',lambda-list
	      (parse-defgeneric-options ',options)))

or

    (defmacro defgeneric (spec lambda-list &rest options &environment env)
      `(progn
	 (eval-when (compile)
	   (apply #'ensure-generic-function
		  ',spec
		  ',lambda-list
		  :environment ',env
		  (parse-defgeneric-options ',options)))
	 (eval-when (load eval)
	   (apply #'ensure-generic-function
		  ',spec
		  ',lambda-list
		  (parse-defgeneric-options ',options)))))

Now both of these definitions have some problems.

The first macro has at least the following problems:

(1) Its expander function causes side effects, which is just generally
considered to be not a good thing, though you can often get away with it.

(2) The compiler-environment-p predicate is necessary to prevent improperly
clobbering the running system, since we are assuming that
ensure-generic-function will "do the right thing" when the environment is from
the compiler.  But this means that this macro (and any like it) can't just call
the appropriate side-effecting function.  They have to themselves distinguish 
environment types, which means that this predicate needs a more official status
than it currently has.

(3) A program analyzer of the type Sandra mentioned sometime back (reads a
file, doing macroexpansions and decorating/transforming the results, then
writes out the resulting forms using print) currently doesn't have a way to
generate a distinguishable compiler environment object, so it has to
explicitely know about every such macro and have a way to perform the
compile-time magic itself in such a way that the rest of the system will
understand.  This is clearly bogus.

The second macro puts the environment argument into the expansion form as a
hard-coded constant.  There are lots of reasons this might lose.  Just to name
a few that I can think of off the top of my head

(1) If there are EQ dependencies involved in manipulation of environments,
Issue QUOTE-MAY-COPY makes this bad.

(2) This may lose in an implementation which stack-conses environments.  The
PCL code walker talks about this problem and goes to some trouble to avoid
being bitten by it.

(3) Sandra's program analyzer example is going to try to print the environment
to a file which will later be read back in in a meaningful way.  Good luck.


PROBLEM 2

Here's another problem exercised by Sandra's program analyzer example (gosh,
this thing sure seems to be a useful tool for thought experiments).  The way
she described it, it would do many of the same actions as a compiler.
Specifically, it would perform the same action a compiler would when
encountering and eval-when form specifying a compile situation, eval'ing the
body forms when appropriate.  Unfortunately, in at least one implementation 
(IIM Common Lisp), this is almost certain to lose.  The reason is that what we
put in these (eval-when (compile) ...) forms is often code which assumes that
it is really being called by the compiler.  The compiler sets up certain
context information, without which the code in the eval-when's will at best
cause immediate errors, or worse, might just silently appear to have worked
when in fact they have done something fairly unpredictable.


A solution to this might be something like a WITH-COMPILATION wrapper macro,
which will do things like setting up any necessary contextual information, and
provide you with an initial compiler-type macro environment.  Of course, to be
useful, accessors and update functions on the environment object are still
needed.  There is already an issue that addresses that,
SYNTACTIC-ENVIRONMENT-ACCESS.

kab
-------