Change to READ

[David A. Moon <Moon at MIT-AI>]

In the next Lisp machine system released, the way READ deals with delimiting
characters for those S-expressions that need one (symbols, numbers) will be
subtly changed.  If the delimiter is a "useless" or "whitespace" character,
it will be thrown away, rather than being UNTYI'ed back into the stream
so that it can be TYI'ed later.  This will happen regardless of whether the symbol
or number is seen at top level or is inside of an expression, and regardless of
whether input is being read from the keyboard or from a file.  This was chosen
after much consideration and dithering, because it is simpler and more
understandable than a scheme where the delimiter is sometimes retained and
sometimes discarded.

Certain programs, such as reader macros, may be broken by this.  For them,
a special variable which they may bind is provided.  If READ-PRESERVE-DELIMITERS
is bound non-NIL around a call to READ, all delimiters are preserved regardless
of whether they are whitespace or meaningful characters.  I believe there are
no programs other than LMS (which doesn't run on the Lisp machine currently)
which care about this.  However, if you have a problem with this change
READ-PRESERVE-DELIMITERS should make it easy to fix your code.

This change obviates most of the need for the function SI:READ-FOR-TOP-LEVEL,
however that function is still around for the purpose of ignoring unmatched
close parentheses and because there are many places that call it.

I would like to request a compatible change to Maclisp, to avoid divergence and
incompatibility on this point.