issue SYNTACTIC-ENVIRONMENT-ACCESS, version 5

[sandra%defun@cs.utah.edu (Sandra J Loosemore)]

OK, here's a new, improved writeup on this issue that incorporates all
the changes we seem to have agreed on.  If this doesn't say what you 
thought it would, let me know.

Forum:		Compiler
Issue:          SYNTACTIC-ENVIRONMENT-ACCESS
References:     CLtL Chapter 8: Macros
                Issue MACRO-FUNCTION-ENVIRONMENT
                Issue GET-SETF-METHOD-ENVIRONMENT
                Issue COMPILE-FILE-ENVIRONMENT
Related Issues: Issue FUNCTION-NAME
		Issue PROCLAIM-LEXICAL
		Issue MACRO-ENVIRONMENT-EXTENT
		Issue DESTRUCTURING-BIND
Category:       ADDITION
Edit history:   Version 1, 2-Oct-88, Eric Benson
                Version 2, 17-Feb-89, Kim A. Barrett
		Version 3, 9-Mar-89, Kim A. Barrett (respond to comments)
		Version 4, 12-Mar-89, Sandra Loosemore (more revisions)
		Version 5, 20-Mar-89, Sandra Loosemore (only proposal SMALL)
Status:         **DRAFT**


Problem description:

 When macro forms are expanded, the expansion function is called with
 two arguments: the form to be expanded, and the environment in which
 the form was found.  The environment argument is of limited utility.
 The only use sanctioned currently is as an argument to MACROEXPAND or
 MACROEXPAND-1 or passed directly as an argument to another macro
 expansion function.  Recent cleanup issues propose to allow it as an
 argument to MACRO-FUNCTION and to GET-SETF-METHOD.

 It is very difficult to write a code walker that can correctly handle
 local macro and function definitions, due to insufficient access to
 the information contained in environments and the inability to
 augment environments with local definitions.


Proposal (SYNTACTIC-ENVIRONMENT-ACCESS:SMALL):

 The following functions provide information about syntactic
 environment objects.  In all of these functions the argument named ENV
 is an environment of the sort received by the &ENVIRONMENT argument to
 a macro or as the environment argument for EVALHOOK.  (It is not
 required that implementations provide a distinguished representation
 for such objects.)  Optional "env" arguments default to NIL, which
 represents the local null lexical environment (containing only global
 definitions and proclamations that are present in the runtime
 environment).  All of these functions should signal an error of type
 TYPE-ERROR if the value of an environment argument is not a syntactic
 environment.

 The accessors VARIABLE-INFORMATION, FUNCTION-INFORMATION, and 
 DECLARATION retrieve information about declarations that are in
 effect in the environment.  Since implementations are permitted to
 ignore declarations (except for SPECIAL declarations), these accessors
 are required only to return information about declarations that were
 explicitly added to the environment using AUGMENT-ENVIRONMENT.
 Implementations are also permitted to canonicalize declarations, so
 the information returned by the accessors may not be identical to
 the information that was passed to AUGMENT-ENVIRONMENT.


 VARIABLE-INFORMATION variable &optional env		[Function]

  This function returns information about the interpretation of the
  symbol VARIABLE when it appears as a variable within the lexical 
  environment ENV.  At least the following four values are returned;
  implementations are permitted to extend this function to return 
  additional values.

  The first value indicates the type of definition or binding which is
  apparent in ENV:

    NIL            There is no apparent definition or binding for variable.
    :SPECIAL       VARIABLE refers to a special variable, either declared 
                   or proclaimed. 
    :LEXICAL       VARIABLE refers to a lexical variable.
    :SYMBOL-MACRO  VARIABLE refers to a SYMBOL-MACROLET binding.
    :CONSTANT      VARIABLE refers to a named constant, defined by
                   DEFCONSTANT.

 [Note: If issue PROCLAIM-LEXICAL passes, then the :LEXICAL result
  will also refer to variables proclaimed lexical.]

  The second value indicates whether there is a local binding of the
  name.  If the name is locally bound, the second value is true.
  Otherwise, NIL is returned.

  The third value is the type specifier associated with the variable
  named by the symbol in the environment.  If no explicit association
  exists, either by PROCLAIM or DECLARE, then the result is the type
  specifier T.  It is permissible for implementations to return a type
  specifier that is equivalent to or a supertype of the one appearing 
  in the original declaration.
  
  The fourth value is true if the variable has been declared IGNORE.
  If no such declaration has been made or the information is not 
  available, NIL is returned.


 FUNCTION-INFORMATION function &optional env		[Function]

  This function returns information about the interpretation of the
  function name FUNCTION when it in a functional position within
  lexical environment ENV.  At least the following four values are
  returned; implementations are permitted to extend this function to
  return additional values.

  The first value indicates the type of definition or binding of
  the function name which is apparent in ENV:

    NIL            There is no apparent definition for FUNCTION.
    :FUNCTION      FUNCTION refers to a function.
    :MACRO         FUNCTION refers to a macro.
    :SPECIAL-FORM  FUNCTION refers to a special form.

  Some function names may refer to both a global macro and a global
  special form.  In such a case, the macro takes precedence, and
  :MACRO is returned as the first value.

  The second value specifies whether the definition is local or
  global.  If local, the second value is true, and it is false when
  the definition is global. 

  The third value is the type specifier associated with the function
  in the environment, or the symbol FUNCTION if there is no functional
  type declaration or proclamation associated with the function.  This
  value might not include all the apparent FTYPE declarations for
  FUNCTION.  It is permissible for implementations to return a type 
  specifier that is equivalent to or a supertype of the one that
  appeared in the original declaration.

  The fourth value is one of the symbols :INLINE, :NOTINLINE, or NIL,
  depending on whether the function has been declared INLINE or NOTINLINE.
  NIL is returned if there are no such declarations present or if the
  information cannot be determined.

  [Note: The use of "function name" rather than "symbol" as the
   description of the function argument is intended to be compatible
   with the various proposals to extend the syntax of function
   specifiers.  If no such change actually occurs then this would only
   refer to symbols.]


 AUGMENT-ENVIRONMENT env &KEY variable
			      symbol-macro
                              function
                              macro
                              declare				[Function]

  This function returns a new environment containing the information
  present in ENV, augmented with the information provided by the keyword
  arguments.  The arguments are supplied as follows:

  :VARIABLE	A list of symbols which shall be visible as bound
		variables in the new environment.  Whether each
		binding is to be interpreted as special or lexical
		depends on SPECIAL declarations recorded in the
		environment or provided in the :DECLARE argument list.

  :SYMBOL-MACRO A list of symbol macro definitions, specified as a
                list of (name definition) lists (that is, in the same
                format as the CADR of a SYMBOL-MACROLET special form).
		The new environment will have local symbol-macro bindings
		of each symbol to the corresponding expansion, so that
		MACROEXPAND will be able to expand them properly.

  :FUNCTION	A list of function names which shall be visible as local
		function bindings in the new environment.

  :MACRO	A list of local macro definitions, specified as a
                list of (name definition) lists.  Each definition must
                be a function of two arguments (a form and an environment).
                The new environment will have local macro bindings of each
		name to the corresponding expander function, which
		will be returned by MACRO-FUNCTION and used by
		MACROEXPAND.

  :DECLARE	A list of decl-specs.  Information about these 
                declarations can be retrieved from the resulting
                environment using the VARIABLE-INFORMATION,
                FUNCTION-INFORMATION, and DECLARATION accessors.

  An error is signalled if any of the symbols naming macros in the
  :SYMBOL-MACRO alist are also included in the :VARIABLE list.
  An error is signalled if any of the names specified as keys in the
  :MACRO alist are also included in the :FUNCTION list.  The consequences
  of destructively modifying the list structure of any of the arguments
  to this function are undefined.

  The extent of the returned environment is the same as the extent of
  the argument environment.  The result may share structure with the
  argument environment, but the argument environment is not modified.

  While an environment argument from EVALHOOK is permitted to be used
  as the environment argument for this function, the reverse is not
  true.  If an attempt is made to use the result of AUGMENT-ENVIRONMENT 
  as the environment argument for EVALHOOK, the consequences are
  undefined.  The environment returned by AUGMENT-ENVIRONMENT may only
  be used for syntactic analysis, ie. the functions specified by this
  proposal and functions such as MACROEXPAND.


 PARSE-MACRO name lambda-list body &optional env	[Function]

  This function is used to process a macro definition in the same way
  as DEFMACRO and MACROLET.  It returns a lambda-expression that accepts
  two arguments (a form and an environment).  The "name", "lambda-list",
  and "body" arguments correspond to the parts of a DEFMACRO or MACROLET
  definition.

  The "lambda-list" argument may include &ENVIRONMENT and &WHOLE.
  The "name" argument is used to enclose the "body" in an implicit
  BLOCK, and may also be used for implementation-dependent purposes
  (such as including the name of the macro in error messages if the
  form does not match the lambda-list).


 DECLARATION decl-spec &optional env     		[Function]

  This function returns a list of declaration-specifiers whose CAR
  is the symbol DECL-SPEC that are in force in the environment ENV,
  sorted so that the innermost declaration is first on the list.

  It is required that this function recognize OPTIMIZE and DECLARATION
  declarations.  The results are unspecified for any other built-in
  declaration-specifier.

  If an implementation has been extended to recognize additional
  declaration specifiers in DECLARE or PROCLAIM, it is required that
  either the DECLARATION function should also recognize those
  declarations, or that the implementation provide an accessor that is
  specialized for that declaration specifier.  Note that if an
  extended declaration specifier may be "bound", this function should
  return only those declarations that apply to the lexically visible
  binding.


Rationale:

  This proposal defines a minimal set of accessors and a constructor
  for environments.

  The PARSE-MACRO function is provided so that users don't have to
  write their own code to destructure macro arguments.  Most
  implementations probably already have a similar internal function.

  Making TYPE and FTYPE declarations optional continues to allow
  implementations the freedom to simply ignore all such declarations.
  The same holds true for other declarations accessed with the
  DECLARATION function.


Examples:

#1:  This example illustrates the first value returned by the function
     VARIABLE-INFORMATION.

  (DEFMACRO KIND-OF-VARIABLE (VAR &ENVIRONMENT ENV)
    `',(VARIABLE-INFORMATION VAR ENV))

  (DEFVAR A)

  (DEFUN TEST ()
    (LET (B)
      (LET (C)
        (DECLARE (SPECIAL C))
        (SYMBOL-MACROLET ((D ANYTHING))
          (LIST (KIND-OF-VARIABLE A)
                (KIND-OF-VARIABLE B)
                (KIND-OF-VARIABLE C)
                (KIND-OF-VARIABLE D)
                (KIND-OF-VARIABLE E))))))

  (TEST) -> (:SPECIAL :LEXICAL :SPECIAL :SYMBOL-MACRO NIL)
      

#2:  This example illustrates the first value returned by the function
     FUNCTION-INFORMATION.

  (DEFMACRO KIND-OF-FUNCTION (FUNCTION-NAME &ENVIRONMENT ENV)
    `',(FUNCTION-INFORMATION FUNCTION-NAME ENV))

  (DEFUN A ())

  (DEFMACRO B ())

  (DEFUN TEST ()
    (FLET ((C ()))
      (MACROLET ((D ()))
        (MULTIPLE-VALUE-CALL #'LIST
              (KIND-OF-FUNCTION A)
              (KIND-OF-FUNCTION B)
              (KIND-OF-FUNCTION QUOTE)
              (KIND-OF-FUNCTION C)
              (KIND-OF-FUNCTION D)
              (KIND-OF-FUNCTION E)))))

  (TEST) -> (:FUNCTION      NIL
             :MACRO         NIL
             :SPECIAL-FORM  NIL
             :FUNCTION      T
             :MACRO         T
             NIL            NIL)


#3:  This example shows how a code-walker might walk a MACROLET special
     form.

(defun walk-macrolet (form env)
    (let ((macros  (make-macro-definitions (cadr form) env)))
	(multiple-value-bind (body decls) (parse-body (cddr form))
	    (walk-implicit-progn
	        body
		(augment-environment env :macro macros :declare decls))
             )))


(defun make-macro-definitions (defs env)
    (let ((results  nil))
	(dolist (d defs)
	    (push (list (car d)
			(coerce (parse-macro (car d) (cadr d) (cddr d) env)
				'function))
		  results))
	results))



Cost to Implementors:

 Most implementations already record some of this information in some
 form.  Providing these functions should not be too difficult, but it
 is a more than trivial amount of work.

Cost to Users:

 This change is upward compatible with user code.

Current practice:

 No implementation provides all of this interface currently.  Portable
 Common Loops defines a subset of this functionality for its code
 walker and implements it on a number of diffent versions of Common
 Lisp.

Discussion:

 The first version of this proposal expressly did not deal with the
 objects which are used as environments by EVALHOOK.  This version is
 extended to support them in the belief that such environments share a
 lot of functionality with the syntactic environments needed by a
 compiler.  While the two types of environments might have very
 different implementations, there are many operations which are
 reasonable to perform on either type, including all of the accessor
 functions described by this proposal.

 AUGMENT-ENVIRONMENT currently requires signaling an error when
 symbol-macro names match variable names in the same call.  This could
 be reduced to "should signal".  By requiring the error signaling, this
 proposal is compatable with Proposal SYMBOL-MACROLET-DECLARE:ALLOW,
 which says

   "... signals an error if a SPECIAL declaration names one of the symbols
   being defined as a symbol-macrolet."

 Maintaining compatability with the SYMBOL-MACROLET-DECLARE proposal
 allows fairly trivial implementations of the SYMBOL-MACROLET
 special-form in terms of the AUGMENT-ENVIRONMENT function.

 Moon notes:

  Symbolics Genera includes an undocumented internal macro, used
  quite a bit in the implementation of the interpreter and code
  analyzers, that could have been called WITH-AUGMENTED-ENVIRONMENT,
  taking keywords like AUGMENT-ENVIRONMENT and also body forms,
  and producing an environment with dynamic extent bound to a
  variable within the body forms.  Would it be useful to have this
  too, or instead of AUGMENT-ENVIRONMENT?  I'm unsure.

 Some people have indicated they think that the :MACRO argument (and
 the :SYMBOL-MACRO argument too?) to AUGMENT-ENVIRONMENT should be an
 a-list of the form (name . definition).
-------