Issue: ADJUST-ARRAY-NOT-ADJUSTABLE (Version 6)

[masinter.pa@Xerox.COM]

I'm sorry it has taken me so long to get back to this. 

Unfortunately, not all of the mail was cc'd to cl-cleanup, which resulted
in a number of crossed signals.

I have a fairly lengthy conversation initiated by the enclosed message
which transpired between Jan 27 and Jan 30, plus Guy Steele's attempt to
fix the same problem, and the discussion of that, and then a fairly
strongly-worded rejoinder from JonL on the issue.


The status is that Version 5 (i.e., Version 4, with an amendedment) passed
at the January 1989 X3J13 meeting.   David has proposed a Version 6, which
I enclose. I'm uncertain whether this version is really unacceptable; it
looks OK to me, and, I think, if I re-read all the mail from Dussud, maybe
it is OK with him too. (Still.)

At the risk of making you repeat yourself, if you have specific
recommendations you think would make Version 6 better than it is now, could
you please restate them?

It will help if we can keep the tone of the discussion low-key. Please keep
perjoratives and characterizations to a minimum. 

Thanks,

Larry
     ----- Begin Forwarded Messages -----

Return-Path: <Moon@ALDERAAN.SCRC.Symbolics.COM>
Received: from ALDERAAN.SCRC.Symbolics.COM ([128.81.41.109]) by Xerox.COM ;
25 JAN 89 16:32:05 PST
Received: from EUPHRATES.SCRC.Symbolics.COM by ALDERAAN.SCRC.Symbolics.COM
via CHAOS with CHAOS-MAIL id 262758; Mon 23-Jan-89 16:27:19 EST
Date: Mon, 23 Jan 89 16:27 EST
 From: David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>
Subject: Issue: ADJUST-ARRAY-NOT-ADJUSTABLE (Version 6)
To: RPG@lucid.com, JonL@lucid.com, Dussud@lucid.com, Masinter.pa,
Moon@STONY-BROOK.SCRC.Symbolics.COM, KMP@STONY-BROOK.SCRC.Symbolics.COM
Message-ID: <19890123212739.0.MOON@EUPHRATES.SCRC.Symbolics.COM>
Line-fold: No

There were some wording problems with the amended version of
ADJUST-ARRAY-NOT-ADJUSTABLE that was passed at the X3J13 meeting
last week.  It ended up not saying what I think was intended.
Two problems:

(1) The amendment made the following function have undefined behavior
in some perfectly meaningful cases, such as the call shown:

  (defun double (a)
     (if (adjustable-array-p a)
         (adjust-array a (* (length a) 2))
         (let ((new (make-array (* (length a) 2))))
           (replace new a :end1 (length a))
           new)))

  (double (make-array 30))

(2) The meaning of the SIMPLE-ARRAY type remains unclear.  In particular,
it is not clear whether the following function is valid Common Lisp or
can violate the type declaration:

  (defun foo ()
    (let ((a (make-array 100)))
      (declare (simple-array a))
      ...))

The unamended proposal addressed issue 1, but neither it nor the
amendment addressed issue 2.

I propose to replace the amended proposal, which X3J13 has already voted
in, with the following.  Any comments?  Should this be sent out in the
upcoming letter ballot?  Should I remove the discussion, since the proposal
has in effect been rewritten twice since that discussion occurred?


Issue:        ADJUST-ARRAY-NOT-ADJUSTABLE
References:   ADJUST-ARRAY (p297), ADJUSTABLE-ARRAY-P (p293),
              MAKE-ARRAY (pp286-289), simple arrays (p28, 289)
Category:     CLARIFICATION/CHANGE
Edit history: 22-Apr-87, Version 1 by Pitman
              15-Nov-88, Versions 2a,2b,2c by Pitman
              02-Dec-88, Version 3 by Pitman
              11-Jan-89, Version 4 by Pitman
              16-Jan-89, Version 5, by Gabriel.  Amended at the meeting to
shorten.
              23-Jan-89, Version 6, by Moon.  Shorten without the bug
introduced
                        by the amendment, add clarification of SIMPLE-ARRAY
type.

Problem Description:

  The description of the :ADJUSTABLE option to MAKE-ARRAY on p288
  says that ``the argument, if specified and not NIL, indicates that
  it must be possible to alter the array's size dynamically after
  it is created. This argument defaults to NIL.''

  The description of the :ADJUSTABLE option does not say what 
  MAKE-ARRAY will do if the argument is unsupplied or explicitly NIL.

  The description of ADJUSTABLE-ARRAY-P on p293 says that it is
  true ``if the argument (which must be an array) is adjustable, and
  otherwise false.'' However, the description of MAKE-ARRAY makes
  it clear that this is not necessarily the same as asking if
  the array was created with :ADJUSTABLE T. If ADJUSTABLE-ARRAY-P
  returns NIL, you know that :ADJUSTABLE NIL was supplied (or no
  :ADJUSTABLE option was supplied), but if ADJUSTABLE-ARRAY-P returns
  T, then there is no information about whether :ADJUSTABLE was used.

  The description of ADJUST-ARRAY on pp297-298 says that it is
  ``not permitted to call ADJUST-ARRAY on an array that was not
  created with the :ADJUSTABLE option.'' This is inconsistent with
  ADJUSTABLE-ARRAY-P.
  
  A problem which comes up in practice is that some programmers
  expect runtime error checking if they have done
  (MAKE-ARRAY ... :ADJUSTABLE NIL) and they later try to adjust
  the array using ADJUST-ARRAY.

  The definition of the SIMPLE-ARRAY type and its subtypes needs
  clarification of its relationship to adjustability.


Proposal (ADJUST-ARRAY-NOT-ADJUSTABLE:CLARIFY):

  1. ADJUST-ARRAY signals an error if ADJUSTABLE-ARRAY-P of its
  first argument is false.  ADJUST-ARRAY does not signal an "array
  not adjustable" error if ADJUSTABLE-ARRAY-P of its first argument
  is true.

  2. ADJUSTABLE-ARRAY-P is true of all arrays created with a non-NIL
  :ADJUSTABLE option to MAKE-ARRAY.  Whether ADJUSTABLE-ARRAY-P is
  true of some other arrays is unspecified.

  3. If MAKE-ARRAY is called with the :ADJUSTABLE, :FILL-POINTER, and
  :DISPLACED-TO arguments each either unspecified or NIL, the resulting
  array is a simple array.  (This just repeats what CLtL says on page 289,
  it's here to aid in understanding the next point.)

  4. If MAKE-ARRAY is called with one or more of the :ADJUSTABLE,
  :FILL-POINTER, or :DISPLACED-TO arguments non-NIL, whether the resulting
  array is simple is unspecified.


Rationale:

  This effectively makes the status quo explicit.  This preserves the
  raison d'etre of simple arrays, which is to provide a portable interface
  to implementation-dependent specialized arrays that trade decreased
  functionality for faster access.

  Specifying the points left unspecified (requiring all simple arrays to be
  non-adjustable and all adjustable arrays to be non-simple) would require
  large changes to some implementations and would be of little benefit to
  users, merely making one kind of nonconforming program fail in all
  implementations instead of failing only in some implementations.  Users
  need to know that certain arrays are simple, so they can put in
  declarations and get higher performance, but users have no need to be
  able to create arrays that are definitely non-simple (for lower
  performance) or definitely non-adjustable (to cause errors).


The following are logical consequences of this proposal:

  Whether an array can be both simple and adjustable is unspecified.

  There is no specified way to create an array for which ADJUSTABLE-ARRAY-P
  definitely returns NIL.

  There is no specified way to create an array that is non-simple.

  This legitimizes ADJUSTABLE-ARRAY-P as an appropriate predicate to
  determine whether ADJUST-ARRAY will reliably succeed.


Examples:

  1. The following program is conforming.  It is unspecified which branch
  of the IF it follows.
  
    (defun double (a)
       (if (adjustable-array-p a)
           (adjust-array a (* (length a) 2))
           (let ((new (make-array (* (length a) 2))))
             (replace new a :end1 (length a))
             new)))
  
    (double (make-array 30))

  2. The following program is conforming.  In no implementation is the
  type declaration violated.

    (let ((a (make-array 100)))
      (declare (simple-array a))
      (frob a))


Current Practice:

  Probably everyone is compatible with this proposal. 

  Symbolics Genera makes :ADJUSTABLE NIL arrays adjustable in most cases,
  and ignores adjustability in deciding whether an array is simple,
  and is compatible with this proposal.

  Lucid, IIM, and Symbolics Cloe make :ADJUSTABLE NIL arrays non-adjustable
  in all cases, and make all arrays non-simple unless the Common Lisp
  language requires them to be simple, and are compatible with this
proposal.

Cost to Implementors:

  It's in principle possible that some implementation would have to change,
  but in practice there are no known implementations that would have to
change.

Cost to Users:

  None. This is a fully compatible change from the user's standpoint.

Benefits:

  Users would know what to expect.

Non-Benefits:

  Users who expect adjusting arrays created with :ADJUSTABLE NIL to signal
  an error would not get the desired error checking.

Aesthetics:

  Most people believe the status quo is unaesthetic.  Having an aspect of
  the language explicitly unspecified is more aesthetic than having it
  implicitly unspecified on account of vague or inconsistent documentation.

Discussion:

  MACSYMA ran into portability problems due to the status quo.
  If the issue had been documented, that would have helped.
  Encouraging implementations that are able to at least make
  (MAKE-ARRAY ... :ADJUSTABLE NIL) create non-adjustable arrays
  where possible would help, too.

  We considered proposals to incompatibly change this primitive in a
  variety of ways, but the community was very split with strong proponents
  and opponents of each alternate proposal.

  The overriding concern driving this proposal is that Symbolics 
  has asserted that most of the other really interesting proposals would
  likely involve a sizable cost to implementors (and their installed bases)
  to implement what were judged by some as gratuitous changes from the
  status quo.

  Pitman wishes some of the other proposals were economically feasible to
  pursue but reluctantly agrees that maintaining (and clearly documenting)
  the status quo is probably the most reasonable avenue left to us.



     ----- End Forwarded Messages -----