Extending the address space of MIT Cscheme (long reply)

[cph@kleph.AI.MIT.EDU (Chris Hanson)]

   Date: 2 May 88 02:37:53 GMT
   From: defun.utah.edu!shebs@cs.utah.edu  (Stanley T. Shebs)

   However, I suspect that either of
   these changes would be far too radical for CScheme, whose coding style and
   general design/engineering have given MIT a black eye in the Lisp/Scheme
   community...

   (Yes, it's a harsh assessment, but does accurately reflect both my own
   experiences in trying to understand CScheme, and a number of other people's
   feelings as well)

As you might expect, your statement ruffled my feathers.  I'd be
interested in your answers to some questions:

    * When you refer to CScheme's "coding style" are you referring to
    the portion written in C, the portion written in Scheme, or both?

    * How about some specific examples of design/engineering decisions
    that you consider flaws, and why?

    * Have you seen a lisp of comparable (or greater) functionality
    that is significantly better in either respect?  Please name it,
    and explain why.

    * When you refer to "experiences in trying to understand CScheme",
    how much of that can be attributed to lack of documentation?  Have
    you ever tried to understand another program of comparable
    complexity?

    For example, while I know a great deal about compilers, I would
    expect to spend a great deal of time and effort trying to
    understand a good one, such as GNU CC.  I'd probably get pretty
    pissed off at the crummy way that certain things were designed.
    But I wouldn't blame this on RMS being a poor programmer.  The
    real problem is that such a program is so complex that even the
    best programmer can't make it perfect.

A general statement like yours is easy to make and can convey false
impressions.  I'd rather not have my reputation (and that of my
colleagues) tarnished by vague accusations about the quality of our
work.  On the other hand, I welcome specific, well-reasoned criticism.

Here are my views on a few relevant topics:

* By and large, I'm very proud of the part of CScheme that is
implemented in Scheme.  I believe that it captures a great deal of
modularity and abstraction, contains many fine symmetries, and in
general is very robust.  It also pushes some of our language
technology to the limit: in particular it suffers from the lack of a
good module description facility.

* The part written in C is a very different story.  I think alot of
this can be attributed to C itself.  Some of it can be attributed to
the fact that most of us in the Scheme group at MIT hate C, and
therefore don't want to do the work that it would take to make it a
beautiful C program.  In fact, I'm not sure it would be possible to
make it beautiful, although clearly it could be much much better.

One thing that everyone here has agreed upon: we would all like to
rewrite that C program in Scheme or a nice Scheme-like language if
only the compiler technology were good enough to let us keep the
portability and performance.  Well, I think that will be true in
another year or two, and then maybe it will happen.

* We have virtually no documentation.  This is obviously a terrible
thing, and we are in fact generating some.  But the bottom line for
this is simply lack of time, plus the fact that none of us has much
text writing experience.

It is not widely known that CScheme, along with the Liar compiler, the
Edwin text editor, and a previous 68000-based implementation, have
largely been created by three people over about 5 years.  Two of us
had significant other commitments during that time which reduced the
amount of effort that could be devoted to the project.  Yet we
generated over a quarter million lines of hairy code in about 10 to 12
person-years.  It is only now that our project has expanded to the
point where we feel that time is available for documentation.

* Various representation decisions that we have made have been
criticised at various times.  In particular: high tags vs. low tags;
SCode vs. byte code; and more recently, register-based vs. stack-based
calling convention.  I can produce reasonable arguments for all of
these decisions.  One thing that I have noticed is that the
"community" seems to have preconceptions about some of these
alternatives being better than others.  In some cases we've tested
them and found that the "common knowledge" is misleading: often the
performance difference between two of the alternatives is very slight.

* In your message you essentially are criticising us for not being
able to change a fundamental representation decision.  I don't believe
this is a valid criticism.  I claim that NO implementation in
existence could easily make a large change at that level.  Small
changes, on the other hand, are a different matter.

In this particular case, there are a number of things we can easily
do, such as: reduce the size of the type field to reflect the fact
that we only use 6 type bits rather than 8; reduce the number of type
bits by making some of them manifest (we could probably go down to 3
or 4 without much work); or change the addressing from byte granular
to word (or even pair) granular.  But a BIBOP-style type
representation is a fundamental change that would require alot of
work.  I would be surprised if any system could claim to change
between such radically different representations easily.

In closing: please don't feel that I'm attacking you.  I am, however,
trying to defend myself, my colleagues, and our work, against your
rather vague condemnation.  I'm sure you have some good reasons for
feeling the way you do and I'd be happy to hear and discuss them.  But
please try to be a little more specific.  General arguments of this
nature do nothing to improve our software, much less the state of the
art.