[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

LispM's crummy I/O performance (was RE: LispM Market Share)

        [...misc deleted...]
           4. File read performance is crummy on the Lispms.  Since that's done
              often, it leaves the impression that the Lispms are slow machines
       This has got to be one of the most frustrating things about the
       Symbolics for me.  The symbolics may not be an inherently slow machine
       but compared to a sparcstation (or even my amiga :-) it gives the feel
       of being quite sluggish (disk i/o, keyboard response in certain
       situations, changing windows, etc. etc.).  Of course, I use a Connection
       Machine so I'm not used to waiting around unless I *really* have to ;-).
     Lets get quantitative about this.
     ?  Why is LISPM I/O is slow?
     ?  What is a good way to measure relative performance given hardware

I have a quantitative response to how slow LispM I/O is and how much of
an obstacle it can be.  I started working on a connection machine with
a LispM front end this summer.  I wrote a simple program to perform a
much needed task: to recognize and count all sequences of length <= n
in a stream of tokens.  My data set included a million tokens, which
was contained in a 5meg file.

Now, anyone who has touched UNIX knows that reading in a 5meg file and
even allocating memory for 1 million tokens (which are represented as
integers) takes on the order of a few minutes (at most).  The LispM
took almost over 5 hours to read in the data set!  This is on a 3650
under 7.2.  What's more, the same LispM took almost 48 hours to dump
the data (via NFS to a disk on a SUN4)!
The ironic part of this is the CM took only about 15 seconds to
actually process the data.

I solved the same problem using C without the connection machine.  In
fact, on a HP Series 800, the serial solution takes only a few hours,
and it would take under an hour if not for paging problems at runtime.

I think that 1meg/hr is not an acceptable I/O rate.

-- David Magerman (magerman@linc.cis.upenn.edu)
University of Pennsylvania LINC Laboratory