Issue: DECLARATION-SCOPE (Version 2)

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

    Date: 14 Jan 88 19:45 PST
    From: Masinter.pa@Xerox.COM

    In order for this proposal to go out in the mailing to X3J13 committee members,
    we will need a version of the proposal that reflects, either in the cost/benefit
    analysis, or in the discussion, the topics mentioned in the mail so far. 

In the hope of helping out, I re-read the mail on the topic.  The
suggested changes were:

-- use a less judgemental term than `normal'.  Suggestions were `bound'
and `binding-related'.  `Bound' doesn't have quite the right connotations,
but I prefer it over `normal'.

-- change the scope of `free' declarations to be something different from
what CLtL p.155 defines to be the scope of `pervasive' declarations.  There
was no coherent proposal, instead the discussion was diverted by a variety
of red herrings.

-- extend Common Lisp to allow TYPE declarations to be used `free'.
The proposal already mentions that X3J13 is discussing this.  It's a
complex issue, and is concerned with semantics rather than with scoping,
so I believe it should be handled independently.

-- extend Common Lisp to allow IGNORE declarations to be used `free'.
However, no meaningful interpretation of this was proposed.

-- include a table of all existing declarations, showing whether they are
`normal' or `free'.

    If you want to separate out the issues of type declarations into a separate
    proposal, then this proposal is misnamed, it probably should be
    DECLARE-SPECIAL-SCOPE and you need another one that is DECLARE-TYPE-SCOPE. 

Free type declarations are not a scoping issue.

    Personally, I think we are close enough to attempt a comprehensive set of rules
    for DECLARATION-SCOPE and that you should go for it; there's not that much time.

Here is a revision that addresses the only two suggested changes above
where it was actually possible to take any action.  I'd like to see this
sent out, since I think this issue is important.  I didn't check with Hornig
before rewriting his proposal, so I hope I have not done any damage to it.


Issue:         DECLARATION-SCOPE

References:    Section 9.1 (pp. 153-157).
	       Cleanup issue FLET-DECLARATIONS.

Category:      CHANGE

Edit history:  V1: Hornig@Symbolics.COM -- 5 January 1988
	       Version 2, Moon, 2-Feb-1988 (edits based on discussion)

Problem description:

The description of the scope of declarations made with DECLARE is both
unclear (although unambiguous) and arguably a mistake.  The issue is
whether the scope of some or all of the declarations includes code
appearing in the non-body part of the special form containing the
declaration.


Proposal DECLARATION-SCOPING:LIKE-VARIABLE:

For the purposes of this proposal, we divide all declarations introduced
with DECLARE into two classes.  When a declaration of a variable appears
before the body of a special form or lambda-expression that binds that
variable, the declaration is called `bound'.  All other declarations
are called `free'.  This division replaces the division into pervasive,
nonpervasive, and SPECIAL declarations appearing in CLtL.

The scope of a `bound' declaration is exactly the scope of the
associated lexical variable or function.  If the declaration is
associated with a special variable, the scope is the scope the variable
would have had if it had not been special.

`Free' declarations are scoped as if they appeared in a new LOCALLY form
which surrounded the entire special form at the beginning of whose body
the declaration appears.  This is the same as what CLtL p.155 defines to
be the scope of `pervasive' declarations.

The following is a complete listing of the types of declarations and
their class (`bound' or `free'):

SPECIAL declarations may be either `bound', affecting both a binding and
references, or `free', affecting only references, depending on whether
the declaration is attached to a variable binding, as described above.

TYPE declarations may only be `bound' (see next section).

FTYPE and FUNCTION declarations may only be `free' (see next section).

INLINE declarations may only be `free' (see next section).

NOTINLINE declarations may only be `free' (see next section).

IGNORE declarations may only be `bound'.

OPTIMIZE declarations may only be `free'.

The `free' or `bound' scoping of implementation-dependent declaration
specifiers is implementation-dependent.


Interactions with other proposals:

There has been some discussion in X3J13 of permitting `free' TYPE
declarations.  This is a semantic issue, not a scoping issue, and should
be treated independently.

If Common Lisp is extended to permit declarations in FLET and LABELS
forms, by acceptance of cleanup proposal FLET-DECLARATIONS:ALLOW,
then declarations of functions (FTYPE, FUNCTION, INLINE, and
NOTINLINE) which appear before the body of a FLET or LABELS form which
defines that function are `bound'.  Such declarations in other contexts
remain `free'.

Common Lisp is ambiguous about whether a variable may be bound several
times by a single form.  It has been proposed that multiple bindings be
permitted for LET*, DO*, PROG* forms and for &AUX variables in lambda
expressions.  If multiple bindings are permitted, `bound' declarations
are treated as if there were a separate `bound' declaration for each of
the bindings.


Examples:

;;; Some examples of `free' and `bound' declarations.

(let ((a 1))
  (declare (optimize speed))		;this is a `free' declaration
  (let ((b 2))
    (declare (type integer b))		;this is a `bound' declaration
    (declare (special a))		;this is a `free' declaration
    ()))

;;; This example applies if you believe that FLET may have declarations.
(flet ((foo (x) (1+ x)))
  (declare (notinline foo))		;this is a `bound' declaration
  (declare (notinline 1+))		;this is a `free' declaration
  ())

;;; The following code is from Pavel.
;;; It produces 7 in existing implementations.
;;; If the proposal is adopted, it will produce 8.
(let ((foo 6))			;a special binding of FOO
  (declare (special foo))	;`bound' declaration
  (let ((foo 7))		;a lexical binding of FOO
    (let ((foo (1+ foo)))	;is the second FOO special or not?
      (declare (special foo))	;`bound' declaration
      foo)))

;;; Treatment of LET* under the proposal if multiple bindings of the same name are allowed.
;;; This form produces the value 9.
(let ((foo 6))			;a special binding of FOO
  (declare (special foo))	;`bound' declaration
  (let ((foo 7))		;a lexical binding of FOO
    (let* ((foo (1+ foo))	;special binding, lexical reference
           (foo (1+ foo)))	;special binding, special reference
      (declare (special foo))	;`bound' declaration, applies to both bindings 
      foo))		        ;special reference


Rationale:

`Bound' declarations are made to control a particular binding of a
variable and should be scoped the same way as that binding.  This is as
true of `bound' declarations which were pervasive under the old rules
as it is of those that were not.


Current practice:

The `bound'/`free' division based on context replaces CLtL's static
pervasive/nonpervasive/SPECIAL division.  Most implementations implement
the rules in CLtL.  Symbolics currently implements rules based on
Zetalisp which are different from both this proposal and Common Lisp.
Symbolics plans to change to Common Lisp rules in the future.


Cost to Implementors:

The cost of implementing this change should be moderate.  The change
will be localized to a handful of places in the compiler and interpreter
which apply declarations to variables.  The other cost would be in
providing tools for users to find programs whose meaning will change.


Cost to Users:

The proposal changes only the treatment of `bound' declarations.  This
change will break very few existing production programs.

It is possible to mechanically examine a program to determine whether
its behavior would change under the new rules.  This permits an
implementation to provide a transition tool to ease conversion to the
new definition.


Cost of non-adoption:

The ability of a `bound' declaration to affect code outside the scope
of the variable which it appears to declare has led to endless confusion
and discussion at Symbolics, on the Common-Lisp mailing list, and
elsewhere.  It will continue to do so unless it is smoothed over somehow.


Benefits:

The costs of non-adoption will be avoided.


Aesthetics:

The distinction between `bound' and `free' declarations introduced by
this proposal is a natural one.


Discussion:

A proposal to forbid `free' declarations except in LOCALLY forms and a
proposal to have `free' declarations affect only the body were discarded
as being too incompatible.

The mapping from the existing pervasive/nonpervasive/SPECIAL division of
declarations and the one proposed here is complex.  In general,
nonpervasive declarations are `bound' and pervasive declarations are
`free'.  SPECIAL declarations are either `bound' or `free' based on
their context, and are no longer treated as a special case.

Some historical support for having `free' and `bound' declarations:

Date: Tue, 20 Dec 83 15:50 EST
 From: "David A. Moon" <Moon%SCRC-TENEX@MIT-MC.ARPA>
Subject: Declarations
To: Common-Lisp@SU-AI.ARPA
...
There are two disjoint classes of declaration: those that are attached
to a particular variable binding, and those that are not.  Note that I
am not discussing proclamations here; they have their own scoping rules
which are different from the rules for declarations.

The scoping rule for the first kind of declaration is that it applies to
precisely the text that is in the lexical scope of the variable binding
with which it is associated.  Such declarations are shadowed by a
variable binding for the same name inside their scope.  Since the
lexical scoping rules are very well and precisely defined, we already
understand everything about this kind of declaration.

The scoping rule for the second kind of declaration is that it is
pervasive.  The declaration is attached to a lambda-expression or to a
form (one of the special forms listed on page 125).  The declaration
applies to all text inside that expression or form; there are no special
cases such as init-forms of LET or DO.  Such declarations are shadowed
by a conflicting declaration inside their scope.

Possible names for the two kinds of declaration are "lexical" and
"pervasive" or "variable" and "non-variable."
...