Re: Issue: ADJUST-ARRAY-NOT-ADJUSTABLE (Version 4)

[Kent M Pitman <KMP@STONY-BROOK.SCRC.Symbolics.COM>]

A new proposal follows this initial commentary...

Current practice is ammended for Lucid and IIM. Please check for
accuracy.

The proposal is ammended per my responses to the following mail
 from Larry:

    Date: 6 Jan 89 11:37 PST
    From: masinter.pa@Xerox.COM

    I'd like to see the following changes to the proposal:

    a) I don't like saying that the effect of adjusting an array which "is not
    adjustable" is not defined. I don't see any reason not to say that it
    signals an error. 

It actually said this already (last paragraph), but it also had a
conflicting paragraph which I removed. Please check the result to see that
it is coherent.

    b) I like defining clearly that "is not adjustable" means
    "ADJUSTABLE-ARRAY-P returns false", and clarifying that MAKE-ARRAY may
    return an adjustable array even if the :ADJUSTABLE array is given NIL.

Right. I think I made this more clear.

    c) I would like to say that either all arrays are adjustable, or else only
    those that are specified with :ADJUSTABLE true, and that a conformal
    implementation will document which behavior it uses.

I think this complicates the user model because it suggests that there is
advantage to be had from knowing that all arrays are adjustable. I can't
imagine any serious way in which a truly portable program could take
advantage of this information even if we could guarantee it.

In fact, though, the Symbolics implementation makes -most- but not all
arrays adjustable. There are some odd situations involving displaced arrays
where I'm told the arrays are not adjustable. So we couldn't be conforming if
you did this. I therefore made no change.

    d) Rename the proposal name from "STATUS-QUO" to "MAY-BE-ADJUSTABLE" and
    change  "Document clearly" to "Specify". 

It wasn't clear what the noun was in "may be adjustable", so I avoided this
name and tried to pick a name that had no connotations at all: DONKEY.
(Surely there's no chance anyone will even think to connect the spirit of
this proposal or the character of its author with derogatory synonyms,
stubborn animals, or Massachusetts liberal democrats... :-) Anyway, hopefully
this will avoid wasted discussion on a supposedly semantics-free symbol.

    Is this OK?

Except for item C, I think I've done the best I could. Proposal follows...

-----
Issue:        ADJUST-ARRAY-NOT-ADJUSTABLE
References:   ADJUST-ARRAY (p297), ADJUSTABLE-ARRAY-P (p293),
	      MAKE-ARRAY (pp286-289)
Category:     CLARIFICATION/CHANGE
Edit history: 22-Apr-87, Version 1 by Pitman
	      15-Nov-88, Versions 2a,2b,2c by Pitman
	      02-Dec-88, Version 3 by Pitman
	      11-Jan-89, Version 4 by Pitman
Status:	      For Internal Discussion

Problem Description:

  The description of the :ADJUSTABLE option to MAKE-ARRAY on p288
  says that ``the argument, if specified and not NIL, indicates that
  it must be possible to alter the array's size dynamically after
  it is created. This argument defaults to NIL.''

  The description of the :ADJUSTABLE option does not say what 
  MAKE-ARRAY will do if the argument is unsupplied or explicitly NIL.

  Some of the original Common Lisp designers assert that the
  :ADJUSTABLE option exists in order to allow a user to select
  between getting adjustable and non-adjustable arrays.
  
  Others of the original Common Lisp designers assert that the
  :ADJUSTABLE option existed to permit implementations in which
  making all arrays adjustable was very expensive to make some
  arrays not adjustable.

  The former camp therefore believes that :ADJUSTABLE NIL means
  that the array MUST be non-adjustable. The latter camp believes
  that :ADJUSTABLE NIL means that the array MIGHT be non-adjustable.

  The description of ADJUSTABLE-ARRAY-P on p293 says that it is
  true ``if the argument (which must be an array) is adjustable, and
  otherwise false.'' However, the description of MAKE-ARRAY makes
  it clear that this is not necessarily the same as asking if
  the array was created with :ADJUSTABLE T. If ADJUSTABLE-ARRAY-P
  returns NIL, you know that :ADJUSTABLE NIL was supplied (or no
  :ADJUSTABLE option was supplied), but if ADJUSTABLE-ARRAY-P returns
  T, then there is no information about whether :ADJUSTABLE was used.

  The description of ADJUST-ARRAY on pp297-298 says that it is
  ``not permitted to call ADJUST-ARRAY on an array that was not
  created with the :ADJUSTABLE option.'' Although this sentence
  is slightly ambiguous (one might argue that :ADJUSTABLE NIL
  involves supplying the :ADJUSTABLE option), it is generally
  interpreted to mean that ``... with :ADJUSTABLE T.'' 

  One problem is that since ADJUSTABLE-ARRAY-P does not predicate
  whether the :ADJUSTABLE option was provided, then
  ADJUSTABLE-ARRAY-P is not an appropriate predicate in all
  implementations to determine whether an array is adjustable.
  Technically, :ADJUSTABLE NIL could create and adjustable array
  (one for which ADJUSTABLE-ARRAY-P returns true), and yet 
  ADJUST-ARRAY might refuse to adjust it (if it had recorded a
  separate bit saying whether :ADJUSTABLE T had been specified
  and used that bit for ADJUST-ARRAY). Fortunately, this problem
  has not been observed to occur in practice, but it is present
  in principle.
  
  A problem which comes up in practice is that some programmers
  expect runtime error checking if they have done
  (MAKE-ARRAY ... :ADJUSTABLE NIL) and they later try to adjust
  the array using ADJUST-ARRAY. That expectation is violated by
  legitimate implementations, since it is permissible for an 
  implementation to create an adjustable array even though it has
  not been asked for, and since calling adjust array on such an
  array "is an error" (and hence the behavior can be extended).

  This perceived lack of error checking may become a legitimate
  portability error to someone who has debugged his code in a 
  an implementation where :ADJUSTABLE NIL arrays might still be
  adjustable and then tried ported his code to an implementation
  which is more conservative in its interpretation.

Proposal (ADJUST-ARRAY-NOT-ADJUSTABLE:DONKEY):

  Define that omitting the :ADJUSTABLE option to MAKE-ARRAY or
  explicitly supplying :ADJUSTABLE NIL may not guarantee a
  non-adjustable array.

  Define that ADJUSTABLE-ARRAY-P -must- be true of an array created
  using :ADJUSTABLE T.

  Define that ADJUSTABLE-ARRAY-P -might- be true of an array created
  with no :ADJUSTABLE option or with :ADJUSTABLE NIL, but that it
  -might- return false.

  Clarify that the implication of the above is that saying that an
  array A "is adjustable" means that (ADJUSTABLE-ARRAY-P A) => true.
  Further clarify that the adjustability of an array has no necessary
  relation to any value was given (or not given) as the :ADJUSTABLE
  option in the call to MAKE-ARRAY which created the array A.

  Clarify that there is no portable way to create a non-adjustable
  array (that is, an array for which ADJUSTABLE-ARRAY-P is
  guaranteed to return false).

  Change the description of ADJUST-ARRAY to say that if an attempt
  is made to adjust an array which is not adjustable (that is, an
  array for which ADJUSTABLE-ARRAY-P would return false), an error
  will be signalled.

  Clarify that this legitimizes ADJUSTABLE-ARRAY-P as an appropriate
  predicate to determine whether ADJUST-ARRAY will reliably succeed.

Rationale:

  This effectively makes the status quo explicit.

  While changing this to a tighter interpretation would be desirable, some
  implementations have suggested that such a change might be very expensive
  or impossible given their existing data storage layouts.

  Although technically the changes to ADJUST-ARRAY are incompatible changes
  from the spec, it's not believed that there are any implementations which
  deviate, so we're still categorizing this as a clarification.

Current Practice:

  Probably everyone is compatible with this proposal. 

  Symbolics Genera makes :ADJUSTABLE NIL arrays adjustable in most cases,
  and is compatible with this proposal.

  Lucid, IIM, and Symbolics Cloe make :ADJUSTABLE NIL arrays non-adjustable
  in all cases, and are compatible with this proposal.

Cost to Implementors:

  It's in principle possible that some implementation would have to change,
  but in practice there are no known implementations that would have to change.

Cost to Users:

  None. This is a fully compatible change from the user's standpoint.

Benefits:

  Users would know what to expect.

Non-Benefits:

  Users who expect adjusting arrays created with :ADJUSTABLE NIL to signal
  an error would not get the desired error checking.

Aesthetics:

  Most people believe the status quo is unaesthetic.

Discussion:

  MACSYMA ran into portability problems due to the status quo.
  If the issue had been documented, that would have helped.
  Encouraging implementations that are able to at least make
  (MAKE-ARRAY ... :ADJUSTABLE NIL) create non-adjustable arrays
  where possible would help, too.

  We considered proposals to incompatibly change this primitive in a
  variety of ways, but the community was very split with strong proponents
  and opponents of each alternate proposal.

  The overriding concern driving this proposal is that Symbolics 
  has asserted that most of the other really interesting proposals would
  likely involve a sizable cost to implementors (and their installed bases)
  to implement what were judged by some as gratuitous changes from the
  status quo.

  Pitman wishes some of the other proposals were economically feasible to
  pursue but reluctantly agrees that maintaining (and clearly documenting)
  the status quo is probably the most reasonable avenue left to us.