New issue ADJUST-ARRAY-NOT-ADJUSTABLE-BROKEN, or Hokey-Pokey-ADJUST-ARRAY

[Jon L White <jonl@lucid.com>]

[The "Hokey" comes from RPG]

I've resisted commenting on this issue, until all the ballyhoo died down.
Looking at Dick's somewhat murky amendment, and Moon's attempt to simplify
it, I would characterize those as "good faith" attempts to reduce the
degree of discrepancy between the 3600 implementation and virtually
all other Common Lisps, by *** merely retracting some of the formerly
required error checking in ADJUST-ARRAY ***.   

Nevertheless, this "reduction in degree" does not address the underlying 
problem, which is the portabililty of the type SIMPLE-ARRAY.  Appallingly,
I see that many readers of this mailing list actually believe that the 
proposal is to retract the portable type SIMPLE-ARRAY, and leave it in as 
murky a state as FIXNUM formerly was.  This is the disaster which I have 
so vehemently opposed.  Unfortunately, your "new proposal" fosters this 
confusion, in the following explanatory phrases:

  In implementations of kind (1), a declaration that an array is simple
  (using the SIMPLE-ARRAY, SIMPLE-VECTOR, SIMPLE-STRING, or SIMPLE-BIT-VECTOR
  type specifier) may be taken literally: it is a guarantee that the array
  in question will be, among other things, not adjustable.  The compiler in
  such an implementation may rely on this declaration to produce good
  compiled code.

  In implementations of kinds (2) and (3), a declaration that an array
  is simple is understood a little bit differently: it constitutes a
  guarantee by the user that he made a good-faith effort to create a
  . . . 

Postscripted to this message is a copy of the message I sent to X3J13 
regarding the character proposal *** and in this message I argue very 
very strongly against imbuing CL with non-portable types [I'm including 
it with this note since not everyone reading cl-cleanup is on x3j13].  
Your phrase "In implementations of kinds (2) and (3), [type SIMPLE-ARRAY]
is understood a little bit differently" would be a giant step backwards 
in this regard.

Despite opaque, clever wordings -- and despite parliamentary legerdemain 
at the Hawaii meeting --  more and more people are beginning to see the 
light on this.  No matter what we do to ADJUST-ARRAY, there are only
two real alternatives:
   (1) 3600 Lisp (and TI Lisp?) remain technically incompatible on the
       definition of SIMPLE-ARRAY -- they could choose to defend this
       technicality to their customers on the grounds that it provides 
       a very worthwhile extension to CL, while at the same time alerting
       their users to the potential for porting problems.
   (2) we decide to "break" the type SIMPLE-ARRAY in some way (such as
       by just removing it from the language altogether);  of course, 
       the stock hardware implementations will do nothing of the sort 
       -- they will continue to use the type just as at present for the 
       enormous benefit it provides to "fast" compilation -- and the 1989 
       CL Standard will simply fail to describe this most critical and 
       widespread feature [i.e., the standardization process will have 
       broken down].

Incidentally, it's not obvious to everyone just why the definition of
SIMPLE-ARRAY found on CLtL p28 is so critical to open-coding on stock
hardware.  I don't want to go into a long lecture on implementational
techniques now, but at the Hawaii meeting I spoke with three other
compiler writers for stock hardware, and we all seem (independently)
to have come up with the same techniques.  We all stand firmly behind
the current dichotomy between SIMPLE-ARRAY and (non-simple) ARRAY, 
which excludes adjustability from "simpleness";  the matter is similar
to the use of "specialized storage" for various array element types.



-- JonL --



Date: Thu, 2 Feb 89 13:04:21 PST
 From: Jon L White <jonl>
In-Reply-To: David A. Moon's message of Tue, 31 Jan 89 16:53 EST
Subject: Really about TYPEP failures: Comments on the Character proposal dated January 1, 1989

re: I'm afraid I don't understand your comment.  What's inadequately 
    specified about SIMPLE-BASE-STRING?  

Sorry, I wasn't clear enough here.  The phrase used was "wishy-washy", 
and it didn't mean that the specification was unclear or something; 
rather, it called in question the enforcement of the specification.  As 
I understand it, it's entirely permissible for one implementation to
merge SIMPLE-BASE-STRING and SIMPLE-GENERAL-STRING into one type,
whereas another may make them type-disjoint.

This is a problem not specific to SIMPLE-BASE-STRING etc, but also applies
to any "wishy-washyness" about the disjointedness of types, where
there is good reason to believe that some implementations will truly
utilize that disjointnedness.  Recall my comment about the situation 
with the FLOAT datatype:

    Contrast this with the situation regarding the type FLOAT.  Although
    there are many aspects of non-portability regarding the _use_ of
    floating point numbers, there is no permitted variance in the definition
    of the type FLOAT.  It is never permissible, for example, for one
    implementation to implement the FLOAT datatype as lists of integers,
    and another to implement it as some low-level primitive datatype.
    Thus if a user's "declaration" (CHECK-TYPE X FLOAT) fails in one
    implementation, but works in another, it is not due to an inherent
    weakness in the specification of the type FLOAT.

Suppose for the moment that one implementation merges the FLOAT and CONS 
datatypes by implementing FLOATs as a list of three integers (such as the 
the values returned by integer-decode-float), but another implementation 
makes them disjoint as "primitive" types.  At first blush, one might want 
to dismiss this case as simply an "efficiency" concern for the second 
implementation.  But consider the problems for someone developing the 
following program on the first implementation and delivering it on the 
second:

     (defun foo (x) 
       (if (typep x 'float) 
           (sin x)
           (error "Must have a float")))

     (foo (list 1 0 1))                  ;; knowing that (float 1.0) is ...

This is a valid program in the first implementation, because the list
(1 0 1) is a valid FLOAT in that implementation.  But when moving it to 
the second implementation, you get a bug.  Now, implementing FLOATs as 
LISTs may look artificial, but this example exactly parallels one of the 
more annoying porting problems that some 3600 users have when going into
virtually any of the stock hardare implementations.  See footnote below.


Recall the recent cleanup proposal to "tighten up" the definition of
   FIXNUM so that its use will more frequently be portable.

Recall also the recent cleanup issue to tighten the FUNCTION type so
   that it is no longer ambiguous with SYMBOL.

Recall also the CLOS need to tighten up the disjointedness of the types
   found on CLtL p.31 (along with others too); 88-002R, p.1-17.

Recall the trouble we've had with the ambiguity in the alternatives 
   for SHORT-FLOAT, SINGLE-FLOAT, DOUBLE-FLOAT, and LONG-FLOAT.


Given this long history of misguided permissiveness, and our frequent
need to "tighten things up", don't you think it would a mistake to start 
out with the types SIMPLE-BASE-STRING and SIMPLE-GENERAL-STRING ambiguous?

. . . 

-- JonL --


Footnote:

Actually, the FLOATs-as-LISTs example is of more than passing interest,
because if you change the names appropriately, you will see exactly
the same problem that certain users have when porting code from the 3600
implementation to *any* stock hardware implementation that does serious
open-coding of AREF.  Note that the only function I've mentioned is
TYPEP -- not hokey-pokey-adjust-array or whatever; that's why all the
thousands of lines of discussion about adjust-array etc on cl-cleanup are 
not germane to the real problem.  Focusing on adjust-array can, at best, 
emasculate some of its mandated error checking; at worst, it can confuse 
some would-be heros into thinking that the type disjointedness in question
is merely a matter of semantics that can cleared up by clever wording.  
Make no mistake.  The disjointedness is essential to the optimization 
strategies of *all* the commercial stock hardware implementations
represented at the Hawaii meeting.