Re: Interpreter messages inside a PROGV

[kab (Kim Barrett)]

> I have a problem with some interpreter warnings that intrude into my program
> output:
> 
> Interpreter warnings :
> ;   Undeclared free variable ^R-CLT, in an anonymous lambda form.
> ;   Undeclared free variable ?R-CLT, in an anonymous lambda form.
> 
> The code appears to work correctly, so the only problem is that these
> messages make a mess of the output, being very voluminous.
> 
> Where these are coming from:
> 
> I have a production-system interpreter.  The rule actions are
> interpreted by code in a PROGV that is handed a list of variable
> names and variable bindings computed elsewhere.  These names and
> bindings are to be in effect during the action execution. The
> code inside the PROGV gets each action, which is a LISP form, and
> EVALs it.  The action form is normally required by the rest of 
> the code to return NIL.
> 
> The message appears when one of the actions eval'd is a macro, such as:
> 
> (DEFMACRO FRF-SM-PM (IN OUT)
>     `(LET () (SETQ ,OUT (FETCH-REF-FORM ,IN (QUOTE SM))) NIL)
> )
> 
> So for example, the context is something like this:
> (PROGV SYMBOL-LIST VALUE-LIST
> ...
>    (DOLIST (ACTION ACTION-LIST)
> ...   (EVAL ACTION)
> 
> where SYMBOL-LIST would be (^R-CLT ?R-CLT),  VALUE-LIST would be (NIL FOO) 
> and ACTION would be (FRF-SM-PM ^R-CLT ?R-CLT)

The warnings are correct; this is erroneous code.  I'll explain further below, 
and also suggest a possible restructuring that might meet your goals.

> I believe the unhappy interpreter here is the one behind EVAL.  I
> thought that the scoping and binding provided by the PROGV
> function (which is intended for such applications) would have
> kept the LISP interpreter  happy - no problems with the exact
> same code in Franz Allegro CL. 

Since MCL's EVAL is (by default) implemented in terms of COMPILE, the fact that 
you are using EVAL isn't really significant for this question.

Unfortunately, there is a very large variation in the error checking done by 
different implementations.  Having a program compile/run without complaint in 
one implementation is no guarantee that the program is correct.  (A common 
problem is incorrect declarations that happen to be ignored in one 
implementation causing an apparently working program to fail in mysterious ways 
when ported to some implementation that believes those declarations).

> Can anyone:
> 
> 1. Tell me how to shut off the message, at least? Run time is critical, so
> if there isn't a flag variable to do this, I'll have to do something else.

I don't know of a way to do this.

> 2. Explain why MCL is unhappy with my PROGV variables? I thought PROGV in 
> effect made the variables special, and so I would expect that EVAL would 
> treat them as if they were special, so the message is confusing.  Such 
> variables are used as arguments to other action functions interfaced with 
> macros, so the only distinctive thing here might be that the outermost layer 
> of the macro is a LET.  (Is this where the anonymous lambda form is coming 
> from?)

The problem arises because the process of transforming the source (action) form 
into executable code is occuring in a lexical environment in which those 
variables are not declared special.  The fact that it is done in the context of 
a PROGV form that dynamically bound those variables doesn't matter.  It would 
not even matter if you were able to somehow bind those variables using LET 
(instead of PROGV) with appropriate SPECIAL declarations, since the code 
processing done by the call to EVAL is done in the null lexical environment.  

This issue of different times at which certain actions occur is also critical 
to your apparent misunderstanding of the semantics of PROGV.  PROGV has nothing 
to do with special declarations.  It's only effect is to establish dynamic 
bindings at runtime.  Consider the following (erroneous) code fragment:

  (progv (list-headed-by-foo) '(5) foo)

If list-headed-by-foo happens to be an accurate name then you might expect that 
this should simply return 5, and in fact most implementations will do just 
that.  However, many of them will also (under at least some conditions) signal 
a warning due to the undeclared free reference to FOO.  This is because the 
semantics of such a free reference is undefined.

Because the list of symbols to be dynamically bound by PROGV is not determined 
until runtime, it can't affect the lexical environment in which the body code 
is processed.

By the way, the LET doesn't have anything to do with the problem.  Also, the 
anonymous lambda occurs because in MCL (EVAL form) is (by default) more or less 
equivalent to (FUNCALL (COMPILE NIL `(LAMBDA () ,form))).

> 3. Based on #2, if #1 can't be done, is there a workaround?  It is important 
> to be able to use macros in this context, so getting rid of them is not a 
> good solution.  Using APPLY instead of EVAL won't handle the macros.
> 
> Note that it is important that the binding be lost outside of the PROGV, so 
> solutions that declare the variables top-level special won't work either.
> Perhaps I should proclaim the variables special when their names are 
> computed?  I thought PROGV in effect did this.  But how long would the 
> proclamation take (run time is critical), and would it have the desired 
> effect inside a PROGV?

I'm not sure what you mean when you say "solutions that declare the variables 
top-level special won't work".  I will assume you mean that you don't want to 
specify a global value for these variables by doing something like
  (DEFVAR ?R-CLT 'FOO)
and I agree that this wouldn't do what you want.  PROCLAIM'ing the variables 
special when their names are computed would do what you need (as I said above, 
PROGV does not declare anything).  I can't say with certainty that the calls to 
PROCLAIM won't affect your performance, but I would bet that any time spent 
there is going to be in the noise compared to the calls to EVAL.

Another possibility is to use the symbol list to build a special declaration 
form and augment the action forms with it.  Specifically,

    (PROGV SYMBOL-LIST VALUE-LIST
      ...
      (DOLIST (ACTION ACTION-LIST)
        ...   (EVAL `(locally (declare (special ,@symbol-list)) ,ACTION))

But on the basis of what you've told me so far I'd try the PROCLAIM at symbol 
creation time approach first.