Issue: FUNCTION-TYPE (Version 8)

[Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>]

Issue:        FUNCTION-TYPE
References:   functions (p32), types (p33), FUNCTIONP (p76),
              SYMBOL-FUNCTION (p90), APPLY (p107), COERCE (pp51-52)
Category:     COMPATIBLE CHANGE
Edit History: 26-Feb-87, Version 1 by Gabriel
              15-Mar-87, Version 2 by Cleanup Committee
              10-May-87, Version 3 by Fahlman
              29-May-87, Version 4 by Masinter (incorporate comments)
              15-Jun-87, Version 5 by Fahlman (include two options)
              23-Oct-87, Version 6 by Masinter (only STRICT-REDEFINITION)
              09-Nov-87, Version 7 by Masinter (minor cleanup)
	      14-Nov-87, Version 8 by Pitman (major restructuring)

Problem Description:

 The definition of the term ``function'' in CLtL includes all symbols and
 many lists in addition to `true' functions.

 Also, page 47 of CLtL states that the FUNCTION type specifier can only
 be used for declaration and not for discrimination. Some of the original
 Common Lisp designers maintain that this restriction on the use of the
 FUNCTION specifier was meant to apply only to long-form FUNCTION
 specifiers, but since this intent was not explicitly stated, the status
 of FUNCTION as a type is blurred. 

 A consequence of the p47 confusion is that (FUNCTIONP x) cannot portably
 be relied upon to be equivalent to (TYPEP x 'FUNCTION).

Proposal FUNCTION-TYPE:CONSERVATIVE

 1. Introduce a new type PROCEDURE that can be used both for declaration
    and discrimination.

    1a. The types CONS, SYMBOL, ARRAY, NUMBER, CHARACTER, and PROCEDURE
        are pairwise disjoint.  In particular, a list may not be used
 	to implement any PROCEDURE subtype.

    1b. Define that the type COMPILED-FUNCTION is a subtype of PROCEDURE.
        Implementations are free to define other subtypes of PROCEDURE.

    1c. Introduce a new function, PROCEDUREP, such that
	(PROCEDUREP x) == (TYPEP x 'PROCEDURE).

 2. Define that a ``function'' may be a procedure, a list whose car is
    the symbol LAMBDA, or any symbol (whether fbound or not).

    2a. Clarify that the FUNCTION type behaves as if it had been
        defined by:

         (DEFUN LAMBDA-P (X) (AND (NOT (ATOM X)) (EQ (CAR X) 'LAMBDA)))

         (DEFTYPE FUNCTION ()
	   `(OR SYMBOL (SATISFIES LAMBDA-P) PROCEDURE))

    2b. Clarify that (FUNCTIONP x) == (TYPEP x 'FUNCTION).
        This change is compatible.

    2c. Clarify that the list form of the FUNCTION type specifier may
        still only be used for declaration.

    2d. Clarify that the symbol form of the FUNCTION type specifier may
        be used for type discrimination.

    2e. Clarify that FUNCALL and APPLY continue to accept functions
        as arguments. However, some implementations may produce better
	code for expressions such as (FUNCALL (THE PROCEDURE ...) ...)
	or (APPLY (THE PROCEDURE ...) ...).

 3. Clarify that the result of a FUNCTION special form must be a procedure.

    3a. This implies that some (FUNCTION name) may be implicitly interpreted
	as (THE PROCEDURE (FUNCTION name)). As such, the potential
	optimizations mentioned in 2e are also possible for
	(FUNCALL (FUNCTION ...) ...) and (APPLY (FUNCTION ...) ...).

 4. Clarify that it is an error to use the special form FUNCTION on a
    symbol that does not denote a function in the lexical environment in
    which the special form appears. Specifically, it is an error to use the
    FUNCTION special form on a symbol that denotes a macro or special form.

    4a. Some implementations may choose not to signal this error for
        performance reasons, but implementations are forbidden from
        defining the failure to signal an error as a `useful' behavior.

 5. Clarify that it is permissible for FBOUNDP to return true for a macro
    or special form, and that it is permissible to call SYMBOL-FUNCTION
    on any symbol for which FBOUNDP returns true.

    5a. The value returned by SYMBOL-FUNCTION when FBOUNDP returns true
        but the symbol denotes a macro or special form is not well-defined,
        but SYMBOL-FUNCTION will not signal an error. 

    5b. Assuming that symbol is fbound,
	(PROCEDUREP (SYMBOL-FUNCTION symbol))
	implies
	(AND (NOT (MACRO-FUNCTION symbol))
	     (NOT (SPECIAL-FORM-P symbol))).

    5c. The effect of
        (SETF (SYMBOL-FUNCTION symbol) non-procedure)
	is not defined. Implementations are permitted to signal an error,
	but they are also permitted to define useful (non-portable)
	interpretations.

    5d. The motivation for this distinction between FUNCTION and 
	SYMBOL-FUNCTION is that FUNCTION is intended for day-to-day
	use within programs while SYMBOL-FUNCTION is a data structure
	accessor used primarily for meta-level applications and not
	recommended for general use. It is provided primarily to
	complete the set of accessors on symbols.

	Implementations are permitted, but not required, to store
	information about a global macro-function or special form
	in the function cell. This definition of SYMBOL-FUNCTION
	is intended to leave enough freedom for such implementations
	to conveniently implement FUNCTION, SPECIAL-FORM-P, and
	MACRO-FUNCTION using SYMBOL-FUNCTION as the underlying
	subprimitive.

 6. COERCE is extended to allow objects to be coerced to type procedure.

    6a. (COERCE symbol 'PROCEDURE) extracts the symbol-function of the
        given symbol, signalling an error if SYMBOL is not fbound or if
	the contents of the symbol-function cell is not a procedure.

    6b. (COERCE lambda-expression 'PROCEDURE) is equivalent to
        (EVAL `(FUNCTION ,lambda-expression)).

 7. Clarify *MACROEXPAND-HOOK* is permitted to contain any kind of function.
    The function is coerced to a procedure before being called as the
    expansion interface hook by MACROEXPAND-1.

Rationale:

 The fuzzy definition of ``function'' has descended from older dialects of
 Lisp, such as Maclisp. Many places in existing code make assumptions about
 the current meaning, making any change painful.

 It is very important both for documentation clarity and for program type
 discrimination (such as CLOS) to have a clear term which denotes a 
 ``true function.''

 This proposal manages a compatible definition with most existing uses of
 the term ``function'' while providing a graceful upgrade path to the term
 ``procedure'' for use in situations that call for a higher degree of clarity.

Current Practice:

 In some implementations, (TYPEP x 'FUNCTION) signals an error.
 In some implementations, (TYPEP x 'FUNCTION) is the same as (FUNCTIONP x).
 In some implementations, (TYPEP x 'FUNCTION) is the same as what this
  new proposal calls (TYPEP x 'PROCEDURE).

 Implementations vary on what my go into the function cell, depending on
 how much error checking they want to have to do at function call time, and
 depending on whether they store other kinds of information (such as special
 form information) in the function cell.

 Few current Common Lisp implementations are likely to have exactly the
 semantics described in this proposal, but most are likely to be very close.

Adoption Cost:

 Bringing type predicates (FUNCTIONP, etc.) and higher order functions
 (APPLY, etc.) into compliance should require little or no effort in most
 implementations.

 Compiled functions are true functions in almost all current
 implementations, but in some implementations interpreted functions and
 closures stored in the function cell of a symbol are represented as lists.
 Under this proposal, such lists would have to be changed to be procedures
 (implemented either as structures or to some special internal data type).
 The behavior of COMPILE, STEP, TRACE, and possibly ED would have to be 
 modified to deal with functions that are not lists (but from which the
 list form can be easily reconstructed if necessary).

Benefits:

 The term ``function'' would be given a useful meaning that was relatively
 compatible with existing usage.

 A new term ``procedure'' would be available for descriptional clarity.

 The type hierarchy would be simplified.

 The new PROCEDURE datatype would be useful for type discrimination in CLOS.

 This proposal brings Common Lisp into closer alignment with Scheme and
 the work of the EuLisp committee. Scheme, for example, also has the concept
 of a ``procedure'' which is compatible with this proposal.

 This proposal provides useful constraints which will be of aid to systems
 doing automatic program analysis for the purpose of ``selective linking.''
 Such tools may in turn make it possible to reduce the total size of a
 delivered application program because only those Common Lisp functions
 that are actually called need to be included.

Conversion cost:

 The conversion cost associated with this proposal is very low because the
 model of FUNCTIONP which it takes is largely consistent with existing 
 practice.

 The new features introduced by this proposal, particularly the PROCEDURE
 data type, can be converted to fairly lazily.

 Because CLtL's language was somewhat fuzzy about what might go into the
 function cell of a symbol, some code that explicitly deposited symbols
 or lists in a symbol's function cell might have to change. Such code was 
 already not portable, however, since some implementations signal an error
 when this is done.

Aesthetics:

 Adding the term ``procedure'' for what has previously been loosely
 termed a ``true function'' is an aesthetic improvement because it gives
 a name to a concept which had no formally defined name.

Discussion:

 This proposal has been discussed at great length; this section attempts
 only to summarize the important points.

 There is general agreement that the definition of the FUNCTION data type
 must be clarified or revised. The cleanup of the type hierarchy is important
 to the CLOS group.

 There are additional issues which were formally attached to this proposal
 and are now separated and should be brought up independently as another
 issue. Key among them is the issue of whether implicit type coercion should
 be done by functions like APPLY, FUNCALL, MAPCAR, and so on. Or, in the
 terminology of this proposal, should those functions allow functions or
 just procedures as arguments?

 The description of COMPILE must be changed, since it is no longer
 meaningful to speak of a symbol with a definition that "is a
 lambda-expression".  We believe this is a subject for a separate
 proposal, as the behavior of COMPILE needs additional clarification.

 Pitman supports this restructured proposal.