LispMachine File Reading Time.

["Robert W. Kerns" <RWK%fuji.ila.dialnet.symbolics.com@RELAY.CS.NET>]

Did you use :ELEMENT-TYPE 'STRING-CHAR streams?  If not,
you were in some sense comparing apples and oranges, and the
real Symbolics times will be noticably faster.

It is  very important when reporting benchmarks to report
all relevant issues like this.

    Date: Thu, 18 Jan 1990 8:37:02 PST
    From: Keith Price <price%iris.usc.edu@usc.edu>
    With all the talk of file reading times, here are the results of an internal
    benchmark for Lisp reading character data and storing in a structure.
    Further testing of the Symbolics version reveals that 90%+ of the time goes
    to the (read).  

Meaning?  I would guess that by this you mean that 90% of the time was spent
inside READ exclusive of time spent inside READ-CHAR or :TYI.  Is this what
you meant?

		    The comparison is with programs written in Lisp (i.e. the
    same program, no tuning for the system, no changes for each system).  The
    comparison is various versions of Suns (3s and 4s) in different
    configurations, Symbolics 3645 and XL400, TI Explorer II, and Lisp on a VAX
    8600.  

	    After several conversations with Symbolics people both here in LA and
    in MA, I am expecting great improvements in this program with Genera 8.0 (I
    was "promised" such operations were addressed), so these times reflect old
    data.   All the times are for local file accesses.   

I assume local means local LMFS for the LISPM case?

							 I should also state that
    the general benchmark results were strongly in favor of the Symbolics, even
    compared to the Sun 4.
    Input function runtimes.  Primarily Read the file, and put into the data
    structure.  The times depend on the disk, GC, and conversion times.
	     Sun3/160     Sun3/260	  Sun IV(S) SUN IV(U) Symbolics    TI-II  8600  
	    Real/Run     Real/Run	  Real/Run  Real/Run  Real(Paging)
    Small	20.8/20.4    17.0 /16.1	  9.58/9.10  8.2/8.0  17.1(1.1)    6.85   3.78
    Small	31.7/29.0    14.1 /13.4	 10.01/9.61 12.0/11.9 16.4(0.3)    6.93   3.74
    ---------------------------------------------------------------------
    Large	539.7/179.7  134.6/131.8 78.2/75.0  47.8/47.4 155.8(20.7) 57.89
    Large	350.6/117.5   85.6/ 77.8 39.1/37.6  35.9/35.6  75.7(10.5) 26.48
    _________________________________________________________________________

    (Times are in seconds, XL400 didnot fit on the list, but was not that much
    different from 3645 for this case.  All system overhead is included on
    Symbolics and not included for the Unix boxes.  You want real speed get a VAX
    8600, not a Sun4.)
	    Keith Price
	    price@usc.edu

If you're going to report times including Paging time on a Symbolics machine,
you had better compare memory sizes and disk organization.  Is the Symbolics
machine doing paging and file I/O from the same disk drive?  Is the LMFS
scattered around in several FEP files on one drive?  Is it fragmented?  What
kind of disk(s)?

Similar questions apply to the other configurations being tested.

Why no 8600 times for the large case?  What configuration?  What kind of disk(s)?
Disk controller(s)?

Did you include file opening & closing time, or just file processing time?

I don't mean to imply that your benchmark isn't useful, but it would be a lot
more useful if you were more careful about reporting your test conditions.
Too often, people do good benchmarking and then spoil it by reporting piecemeal
results.