[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: benchmarking



[b.t.w. I now understand the TAK time anomaly.  Your time is for 10 iterations,
whereas ours is scaled back to 1.]

Well, these numbers look more reasonable, and I actually wouldn't be
terribly unhappy if they were true.  I merged the MOD and non-MOD entries,
taking the lowest value for each implementation, and deleted the I/O
benchmarks, which although somewhat interesting measures of system
performance, don't indicate much about compiler technology.

       Name   AKCL cmulisp  akcl/cmu
      BOYER  1.533   2.150      0.71
     BROWSE  2.617   7.420      0.35
       CTAK  1.033   0.460      2.25
     DDERIV  0.933   1.740      0.54
      DERIV  0.650   1.480      0.44
     DESTRU  0.250   0.273      0.92
       DIV2  1.367   1.550      0.88
        FFT  0.183   0.300      0.61
     FRPOLY  5.917   9.680      0.61
     PUZZLE  1.117   0.650      1.72
       STAK  0.450   0.278      1.62
        TAK  0.300   0.620      0.48
       TAKL  0.183   0.370      0.49
       TAKR  0.067   0.168      0.40
   TRAVERSE  5.450   7.870      0.69
     TRIANG  9.283  13.030      0.71
 Geom mean:                     0.78

So, with appropriate compiler changes added, on a SPARC, using a benchmark
source without CMU declarations, AKCL does the unsafe Gabriel benchmarks
0.32 faster than beta CMU CL.

However, block compilation buys about 7%, and instruction scheduling bought
10-15% on the MIPS.  I don't know how much using register windows buys, but
I suspect that it explains much of CMU CL/SPARC's poor performance on
call-intensive Gabriels when compared to other implementations (and may
explain why AKCL does least well on CTAK.)

This will get us into the performance domain I suggested in my original
post.  

I am in the process of composing a message explaining why I am more
interested in making it 30% easier to get a Lisp program within 2x of C
performance than I am in making it possible for a Lisp program to get the
last 30% of the way to C performance.  It isn't the Lisp programs that run
30% slower than C that make people think Lisp is slow, it is the ones that
run 30x slower.

  Rob