Re: Benchmarking the Explorer vs. the Symbolics

[Mabry Tyson <TYSON@SRI-STRIPE.ARPA>]

Re your results in Genera 7.0....

It is rather obvious that the I/O to dynamic windows is extremely slow.  If you
want some fun, logout and try to access a file.  When it asks you to login,
do a Control-Meta-Suspend and then a Control-B.  The backtrace is about a
screen full, almost entirely due to the accept/presentation system.

In case you didn't know, PRIN1 in genera 7 is do a pretty-print by default.
So it makes a lot of difference how the output is being generated.  If indeed
the routines that are printing out are doing PRINTs or whatever, the added
overhead of pretty-printing is added in.  Of course, you get something back
for that extra overhead....more readable output.  (Yes, Guy Steele's book
describes the initial value of *PRINT-PRETTY* as implementation-dependent.)

By the way, you will find that output is faster if you move the mouse off of
the screen!  (Thanks to Dynamic Windows, I think.)

Indirectly, it sounds as though one of your "benchmarks" is loading a compiled
file.  The reason this is so slow in Genera 7 is that when you load the file
a set of cross-references are created in the who-calls database.  Benchmark
a M-X WHO CALLS in both systems and see which runs faster there!  If you don't
want that feature, you can turn it off.  See page 75 of Volume 0 of the
documentation.

It is difficult to have good benchmarks.  Code and usage tend to be
optimized for the machine and conditions you are used to.  (I just went
through some code that was originally written for Interlisp and it is
horribly inefficient for Zetalisp.  Conversely, code written for
Zetalisp would be horribly inefficient for Interlisp.)  I have not
decided whether I like the who-calls database being created at load
time.  It definitely slows you down up front but speeds you up if you
ever use M-X WHO CALLS.  I have not yet become accustomed to the extra
power offered by the Genera 7.0 release.

It is my hope and expectation that there are many efficiencies that are
being added by Symbolics that will be coming out in 7.1.  New code
always has problems and there was a significant change between release 6
and Genera 7.1.

The only true test of any system is how productive it makes you.
Benchmarks are like deciding who is a better person by how fast they
knit.  Obviously Symbolics thinks that the Common Lisp, Dynamic Windows,
new Flavors, Who Calls Database, and all the other features will make
their customers more productive.  In some cases, some customers will
probably disagree.  Certainly in the short run, before I get used to
them, I am not completely enamored of Dynamic Windows.  The concept is
interesting but the implementation gets in my way.

Not being that familiar with the current state of software on the
explorer (it has been quite a while since we last evaluated their
equipment and software), would it have been more fair(?) to have
compared an earlier Symbolics Release (say 5 or 6) world with the
Explorer world?  (Turn down the flames, that is more a question than a
statement... As I said, I don't know the current state of the software
on the explorer.)