paper for IWoLES

[Masinter.pa@Xerox.COM]

I've committed to producing a paper for IWoLES (International Workshop on LISP
Evolution and Standardization) next month. I took my draft letter to Common Lisp
and a few other things and tried to turn it into a paper.

I need to send this off right away, but I thought, since it purports to speak
for the cleanup process as a whole, I should run it by you.  


\documentstyle{article}
\title{Common Lisp Cleanup}
\author{Larry Masinter}
\date{10 January 1988}
\evensidemargin=0cm
\oddsidemargin=0cm
\topmargin=0cm
\textheight=22.5cm
\textwidth=16.5cm
\columnwidth=\textwidth
\begin{document}
\maketitle

This paper describes some of the activities of the ``cleanup'' sub-committee of
the ANSI X3J13 group. It describes some fundamental assumptions of our work in
this sub-committee, the process we use to consider changes, and a sampler of
some of the changes we are considering.

\section{Assumptions.}

We believed the only reasonable way to obtain a standard for Lisp was to start
with a known, widely implemented description, namely ``Common Lisp the
Language'' by Guy L. Steele Jr. ({\em CLtL}) and to consider individually
various clarifications, changes, additions, enhancements, modifications.

Common Lisp ``works''. It is widely implemented. Common Lisp has a relatively
small kernel of semantics, facilities for extensions, and a large library of
data types and operations on them. It is a good starting point for a Lisp
standard.  We do not believe that it is necessary to start over to design a good
Lisp standard.  Even if one favors modifications to the fundamentals of the
language semantics (e.g., function and variable name scoping), most of the
library of data types and operations of Common Lisp will continue to form a good
basis for applications and it is valuable to clarify the operation of the
library.

There are three kinds of problems with CLtL we would like to resolve in creating
a standard.

First, CLtL had its roots in a user's level description of a programming system;
although it is wonderfully specific as user manuals go, there are numerous
places that allow more implementation flexibility than is desirable in a
standard.  In a standard, we would expect that ambiguities to be either
eliminated or else {em explicitly} allowed.

Secondly, CLtL was written and published without extensive prior use of some of
its features; a few have turned out to be unworkable or useless and not
implemented. We hope to change or remove those features.

Third, we've discovered that there are features that exist in nearly in every
implementation but with different names and calling conventions or (in a very
few instances) are logical extensions of the current specification. We would
like to augment the standard by providing access from the standard language to
those features.

\section {The process.}

The cleanup subcommittee considers those minor changes that are not being
addressed by any of the other single-topic subcommittees of X3J13 (objects,
windows and graphics, characters, iteration, compilation, validation, and a few
others.)

We consider proposals both internally generated by members of the cleanup
committee or are received from members of the community.

Almost all work of the committee is handled via electronic mail. We rarely meet
as a group in person. Most decisions are made by consensus. When we cannot agree
by consensus whether to endorse a proposal, we may present it to X3J13 for
voting with the various points of view represented in the writeup; we believe
our responsibility is to fully describe the issues, and allow the larger
community to vote.

We attempt to analyze the costs and benefits for each proposed modification.
Each proposal, considered individually, must make the language better--enough
better to be worth the cost of making a change.  To this end, we require that
each proposal be accompanied by an analysis of the various costs and benefits.
Of course, there are conflicts in the interpretation of costs and benefits.  

There are some general principles of good programming language design: programs
should be easy to read; programs should be easy to write; the language should
allow users to write efficient prgrams. These goals sometimes work against each
other: for example, adding features makes the job easier for the program writer,
but harder for the reader. These conflicts in desired make the design process
difficult, because it is not possible to optimize all of the design goals
simultaneously. 

The process of explicitly addressing costs and benefits has successfully
eliminated arguments based merely on personal preference. Most often, what
initially seems like a matter of taste in fact has roots in a more explicit
judgment of benefit over cost. By adopting an explicit format for those items,
we've managed to expose the underlying principles.

We only consider ``complete'' proposals. While we've discussed some open
problems, we generally avoid ``design by committee.'' Rather, after some initial
discussion, a single individual will produce a proposal.

\subsection {Elements of a proposal.}

These are the important elements of the ``cleanup'' proposals we develop:

\paragraph {Category.}
We attempt to distinguish between a clarification (proposal to resolve an
ambiguity or case of under-specified situation in CLtL), a change (proposal to
make an incompatible change to the language), and an addition (proposal to add
an upward compatible feature.) The category of a proposal will likely dictate
what a responsible implementor of Common Lisp would do prior to the acceptance
of a new draft standard; e.g., clarifications can be adopted immediately, while
incompatible changes might wait until there is an official standard or be
implemented in a separate "package", at which time incompatible changes can be
adopted all at once. 

\paragraph {Problem Description.}
Suprisingly, requiring an explicit description of the problem, indpendent of
some design for its solution, has been most helpful in avoiding unnecessary
changes to the language. Why does anything need to happen at all?  (``If it
ain't broke, don't fix it.'')

\paragraph {Proposal.}
We expect proposals to separate out the description of the proposal from the
arguments for it. Again, this simple measure helps lend a great deal of
objectivity to the process. While our proposals are expected to be precise and
accompanied by test cases and examples, we generally look to the editor and
editorial committee of X3J13 to actually produce the specification that reflects
the proposal.  

\paragraph {Current practice.}
What do current Common Lisp implementations do? Current implementations (at
least implementation that attempt to be faithful to CLtL) often give us good
indication of what implementors believed CLtL to mean. We put some faith that
they attempted a reasonable interpretation.  In cases where no implementation
exactly matches CLtL, this is a good indication that CLtL's description is
flawed.

\paragraph {Costs.}
Every change or resolution costs something to adopt: to implementors (What does
it take to change a non-conforming implementation to a conforming one? Does this
affect some, one, many functions? Is public-domain code available?), and to
current users (to convert existing user code or tools.) We hope to give a
measure of whether the changes required for each are small or large. In general,
we take cost to users more seriously than cost to implementors.

\paragraph {Benefits.}
Benefits are diverse: our goals are to improve the portability of the language,
its performance, the cleanliness of its semantics, and its aesthetics. Our major
concern is with allowing important applications of Common Lisp to work well
across many different implementations.

\section {Open issues.}

\renewcommand{\descriptionlabel}[1]{\hspace\labelsep $\bullet$ }

This section lists most of the proposals currently being considered, in an
extremely abbreviated form. Many of these issues have {\em not} been adopted and
many {\em will not be}; they are listed here to give an idea of the kinds of
issues the cleanup committee is currently addressing. New issues--especially
that has affected the portability of any current program--are welcome. 

{\em Reminder: these proposals have not been accepted; they are not part of any
standard. This is a list of issues considered, not of issues passed. While some
have been endorsed by X3J13, others are likely to be rejected.}

\paragraph {Clarifications.}
These are cases where CLtL does not specify behavior; usually these are also
cases where implementations actually differ and users have found code
non-portable.

\begin{description}

\item[append-dotted] \verb|(APPEND '(A . B) '(E F))| is not an error, returns
\verb|(A E F)|.
\item[colon-number] \verb|:123| is an error and not a keyword symbol.
\item[declaration-scope] Specify more precisely the scope of declarations.
\item[compiler-warning-break] \verb|*BREAK-ON-WARNINGS*| applies to compiler
warnings too.
\item[flet-implicit-block] The bodies of functions defined in \verb|FLET|,
\verb|MACROLET| and \verb|LABELS| have implicit \verb|BLOCK|s wrapped around
them in the same manner as \verb|DEFUN| and \verb|DEFMACRO| bodies.
\item[defvar-documentation] In \verb|(DEFVAR x y documentation)|, documentation
is not evaluated.
\item[defvar-initialization] In \verb|(DEFVAR x y)|, y is evaluated when
\verb|DEFVAR| is.
\item[pathname-stream] Pathnames can be obtained only from streams opened with
pathnames (or synonym streams.
\item[push-evaluation-order] \verb|(PUSH (A) (CAR (B)))| evaluates\verb|(A)|
before \verb|(B)|.
\item[remf-destruction-unspecified] What side effects are allowed/possible by
REMF, NREVERSE etc.?
\item[setf-method-for-symbols] Examples in CLtL of \verb|SETF|-method for
symbols and \verb|LDB| are inconsistent with left-to-right evaluation order.
Change them.
\item[shadow-already-present] What does SHADOW do if symbol is already there?
\item[sharpsign-plus-minus-package] Elements of \verb|*FEATURES*| are in the
keyword package.
\item[sharpsign-plus-minus-number] Numeric \verb|*FEATURES*| are not allowed.
\item[unwind-protect-non-local-exit] Exit from cleanup clause of
\verb|UNWIND-PROTECT| overrides any unwind in progress. 
\item[adjust-array-displacement] More precise rules for interaction of
\verb|ADJUST-ARRAY| and displaced arrays.
\item[do-symbols-duplicates] Can \verb|DO-SYMBOLS| visit a symbol more than
once?
\item[format-uparrow-scope] What is the scope of \verb|~â??| in \verb|FORMAT|
loops?
\item[function-argument-type-semantics] What are the semantics of argument types
in \verb|FUNCTION| declarations?
\item[lisp-symbol-redefinition] It is illegal (non-portable) to redefine or
shadow with \verb|FLET|, \verb|MACROLET|, \verb|LABELS| functions defined in
CLtL.
\item[princ-character] \verb|(PRINC #\C)| prints C.
\item[import-setf-symbol-package] \verb|IMPORT| has no effect on a symbol's home
package.
\item[format-op-c] \verb|(format t "~C" '#\C)| prints C.
\item[disassemble-side-effect] \verb|DISASSEMBLE| has no side-effects.
\end {description}

\paragraph {Additions.}
These are some new features we are considering adding to Common Lisp.
\begin {description}
\item[aref-1d] Add \verb|(ROW-MAJOR-AREF X N)| to reference n-th row-major
element of an array.
\item[format-comma-interval] Add option to \verb|FORMAT| to specify intervals
other than 3 between digit delimiter character.
\item[get-setf-method-environment] Add an environment argument to
\verb|GET-SETF-METHOD|.
\item[keyword-argument-name-package] Allow arbitrary symbols as keyword-argument
tokens.
\item[sequence-functions-exclude-arrays] Allow some sequence functions to work
on multi-dimensional arrays.
\item[assoc-rassoc-if-key] Add \verb|:KEY| argument to \verb|ASSOC-IF|.
\item[reduce-argument-extraction] Add \verb|:KEY| argument to \verb|REDUCE|.
\item[setf-function-vs-macro] Allow {\tt(DEFUN (SETF FN) (VALUE ARGUMENTS ...)
...)} to define what \verb|(SETF (FN ...) value)| means.
\item[pathname-subdirectory-list] Recommend that directory component of a
pathname be a list in a hierarchical file system.
\item[format-atsign-colon] \verb|@:| and \verb|:@| are the same in format
directives.
\item[stream-class-access] Add functions for finding out stream class, accessing
parts. 
\item[structure-definition-access] Add functions for getting at \verb|DEFSTRUCT|
accessor slots.
\item[trace-function-only] Enhance TRACE arguments.
\end {description}

\paragraph {Changes.}
These proposals are generally incompatible changes to the language, although in
some cases they are consistent with CLtL.
\begin {description}
\item Require \verb|STREAM|, \verb|PACKAGE|, \verb|PATHNAME|,  \verb|READTABLE|,
\verb|RANDOM-STATE| be disjoint types.
\item[pathname-symbol] Symbols do not automatically coerce to pathnames, e.g.,
\verb|(LOAD 'FILE)| is not allowed.
\item[declare-macros] \verb|(LET (A B) (MY-DECLARE) ...)|   Disallow macros
expanding into declarations.
\item[function-type] Change the \verb|FUNCTION| type to be distinct from any
other type, and change \verb|FUNCTIONP|  to accept only \verb|FUNCTION| objects
(not symbols or lists that begin with \verb|LAMBDA|.). Require the
\verb|FUNCTION| always returns a \verb|FUNCTION| object. Disallow automatic
coercion from symbols to functions by \verb|FUNCALL|, \verb|APPLY|, etc.
\item[defmacro-body-lexical-environment] Allow \verb|DEFMACRO| and
\verb|DEFTYPE| bodies to have a lexical environment.
\end {description}

\section {Open problems.}
There are a number of areas that are currently not being addressed, either by
the cleanup committee or by other current committee's of X3J13. We would like to
see some attempts at standardizing on those areas that are most concern to users
attempting to write portable code.

 One is that with the acceptance of a reasonable signal system, the hard work of
specifying the signals which may be raised by various error conditions remain.
We would like to make a part of the standard interface to functions the classes
of errors they can invoke under various circumstances.

No work is currently going on in the area of specifying reference to
multi-programming features, although many implementations of Common Lisp include
them.

There are several problems in the language specification for which we have no
good proposals. Most of these involve interactions of the many functions in the
Common Lisp library, e.g., how \verb|EQUAL| behaves on \verb|DEFSTRUCT| objects.
We believe these items need to be standardized, but have not found a reasonable
design.

\end{document}