Issue: DEFPACKAGE (version 5)

[David A. Moon <Moon@STONY-BROOK.SCRC.Symbolics.COM>]

I almost approve version 4, but there are a couple of problems with it
(enumerated below).  In the interest of saving time, here is a version 5
with the problems fixed and no other changes.  Naturally I approve it.

- fix trivial typos
- the complex forms of :IMPORT-FROM and :SHADOWING-IMPORT-FROM can
  be removed if the prohibition against repeated options is lifted.
  This seems like a worthwhile simplification.
- the change to :IMPORT-FROM and :SHADOWING-IMPORT-FROM to call
  FIND-SYMBOL instead of INTERN inadvertently made them useless,
  since there is no way to assure that non-exported, non-shadowed
  symbols exist.  Add a :INTERNAL option to take care of this.
- change current practice to note that this is an incompatible change
  for Symbolics
- specify what happens if the package already exists

Issue:         DEFPACKAGE

References:    CLtL section 11.7.
               Issue: IN-PACKAGE-FUNCTIONALITY

Category:      ADDITION

Edit history:  Version 1, 12-Mar-88, Moon
               Version 2, 23-Mar-88, Moon, changes based on discussion
               Version 3, 27-Sep-88, JonL 
                (remove :import, :shadowing-import; allow :export to work on
                 imported and inherited; update references to in-package, etc.)
               Version 4,  1-Oct-88, Masinter
               Version 5, 6-Oct-88, Moon

Problem description:

The package functions included in CLtL encourage a programming style
that tends to evoke the worst aspects of the package system.  The
problem is that if the definition of a package is scattered through
a program, as a number of individual forms, it is very easy to read
a symbol before the package setup needed to read that symbol correctly
has been accomplished.  Three examples: an inherited symbol that should
have been shadowed might be accessed; a single-colon prefix might be
used for a symbol that will later be exported, causing an error; a local
symbol might be accessed where a symbol that will later be imported or
inherited was intended.  These problems can be difficult to understand
or even to recognize, are difficult to recover from without completely
restarting the Lisp, and frustrating to programmers.

Proposal (DEFPACKAGE:ADDITION):
      
Add a DEFPACKAGE macro to the language.  In the description below,
'package-name' and 'symbol-name' can be a symbol or a string; if a symbol, 
only its name matters, not what package it is in.  If a string, capitalization
matters, normally uppercase is used.

The syntax of DEFPACKAGE is

  (DEFPACKAGE package-name {option}*)

where each option is a list of a keyword and arguments.  Nothing in a
DEFPACKAGE form is evaluated.

Standard options for DEFPACKAGE are listed below.   Options may appear
more than once (useful mainly for :IMPORT-FROM and :SHADOWING-IMPORT-FROM).
It is an error to specify :SIZE more than once.

(:NICKNAMES {package-name}*)
        Set the package's nicknames to the specified names.

(:USE {package-name}*)
        Inherit from the specified packages.

(:SHADOW {symbol-name}*)
        Create the specified symbols in the package being defined, and 
        place them on the shadowing symbols list.  

(:SHADOWING-IMPORT-FROM package-name {symbol-name}*)
        Find the specified symbols in the specified package and import
        them into the package being defined, and place them on the 
        shadowing symbols list.  In no case will 
        symbols be created in any package other than the one being defined; 
        a continuable error is signalled if no symbol is accessible in the
        package named package-name for one of the symbol-names.

(:IMPORT-FROM package-name {symbol-name}*)
        Find the specified symbols in the specified package and import
        them into the package being defined.  In no case will 
        symbols be created in a package other than the one being defined; 
        a continuable error is signalled if no symbol is accessible in the
        package named package-name for one of the symbol-names.

(:EXPORT {symbol-name}*)
        Find or create symbols with the specified names and export them.
        Note an interaction with the :USE option, since intern'ing may inherit 
        symbols rather than creating new ones; note also an interaction 
        with the :IMPORT-FROM and :SHADOWING-IMPORT-FROM options, since 
        intern'ing will merely access an already imported symbol. 

(:INTERNAL {symbol-name}*)
        Find or create symbols with the specified names.  This is useful
        if a :IMPORT-FROM or :SHADOWING-IMPORT-FROM option in a later
        DEFPACKAGE for another package expects to find these symbols,
        but the symbols are not to be exported.

(:SIZE integer)
        Declare the approximate number of symbols expected in the package.
        This is an efficiency hint only, so that the package's table will
        not have to be frequently re-expanded when new symbols are added
        to it (e.g., by reading in a large file "in" that package.)


Additional options might be allowed by an implementation;
implementations should signal an error if an option not recognized by
that implementation is present.

The collection of symbol-name arguments given to the options :SHADOW,
:INTERNAL, :IMPORT-FROM, and :SHADOWING-IMPORT-FROM must all be
disjoint; an error is signalled otherwise.  In a chronological sense,
the :EXPORT may be thought of as occurring last so that it can make
reference to inherited or imported symbols already created by the other
options.

DEFPACKAGE creates the package as specified, and returns it as its
value.  It has no other side effects; i.e., it does not do an
IN-PACKAGE.  If a package with the specified name already exists, the
existing package is modified, by adding to its attributes but not
removing any (for example EXPORT may be called, but UNEXPORT is never
called).


Examples:

;;; Play it super-safe, and use only strings as names; do not even
;;;  assume that the package it is read in to "uses" LISP; do *not* create
;;;  any symbols whatsoever in the package that it is read in to.

(LISP:DEFPACKAGE "MY-PACKAGE"
  (:NICKNAMES "MYPKG" "MY-PKG")
  (:USE "LISP")
  (:SHADOW "CAR" "CDR")
  (:SHADOWING-IMPORT-FROM "VENDOR-COMMON-LISP"  "CONS")
  (:IMPORT-FROM           "VENDOR-COMMON-LISP"  "GC")
  (:EXPORT "EQ" "CONS" "FROBOLA")
  )

;;; A similar call, using symbols rather than strings as names; expects
;;;  to be read in to a package that "uses" LISP, and *may* create
;;;  random internal symbols in that package (such as MY-PACKAGE etc).

(DEFPACKAGE MY-PACKAGE
  (:NICKNAMES MYPKG :MY-PKG)
  (:USE LISP)
  (:SHADOW CAR :CDR #:CONS)
  )

Rationale:

The availability of DEFPACKAGE encourages putting the
entire definition of a package in a single place.  It also encourages
putting all the package definitions of a program in a single file, which
can be loaded before loading or compiling anything that depends on those
packages.  This file can be read in the USER package, avoiding any
package bootstrapping issues.

In addition, DEFPACKAGE allows a programming environment to process
the whole package setup as a unit, providing better error-checking and
more assistance with package problems, by dint of global knowledge of
the package setup.

Current practice:

Symbolics Common Lisp has always had a DEFPACKAGE, and uses it in 
preference to individual calls to EXPORT, IMPORT, SHADOW, etc.  The SCL 
version of DEFPACKAGE has quite a few additional options, but none of them
appear to be necessary to propose for Common Lisp at this time.  This
proposal is incompatible with Symbolics DEFPACKAGE in some ways that
will probably not cause major problems.

Cost to Implementors:

Small; DEFPACKAGE can be implemented simply as a bunch of
calls to existing functions.

Cost to Users:

Small, this is upward compatible.

Cost of non-adoption:

Packages continue to be difficult to use correctly.

Benefits:

Guide users away from using packages in ways that get them into trouble.

Esthetics:

Neutral.

Discussion:

The "Put in seven extremely random user interface commands" mnemonic
described in CLtL p.191 could be removed, and the special compiler 
handling of these functions necessary to support that could be removed 
(except possibly for REQUIRE and PROCLAIM -- see the compiler Issue 
PROCLAIM-ETC-IN-COMPILE-FILE).  As this would be an incompatible change,
it is not part of this proposal.

The issue IN-PACKAGE-FUNCTIONALITY recommends that IN-PACKAGE  be 
incompatibly changed to recognize only existing packages, not to create 
them.  IN-PACKAGE would then not accept any keyword arguments.

The function MAKE-PACKAGE might also be extended to take all the keywords
that DEFPACKAGE does. This could be the subject of a separate cleanup.

The macroexpansion of DEFPACKAGE can usefully canonicalize
into the strings-as-name form, so that even though the source file
showed random symbols in the DEFPACKAGE form, the compiled file might
have only strings in it.

Frequently additional implementation-dependent options take the
form of a keyword standing by itself as an abbreviation for a list
(keyword T); this syntax should be properly reported as an unrecognized
option in implementations that do not support it.