[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: LISPM I/O performance

Received: from BLAISE.kahuna.decnet.lockheed.com by ALAN.kahuna.decnet.lockheed.com via INTERNET with SMTP id 18447; 26 Jan 90 11:49:29 PST
Date: Fri, 26 Jan 90 11:49 PST
From: Montgomery Kosma <kosma@ALAN.kahuna.decnet.lockheed.com>
Subject: Re: LISPM I/O performance
To: slug@ALAN.kahuna.decnet.lockheed.com
In-Reply-To: <9001260601.AA09618@uunet.uu.net>
Message-ID: <19900126194917.2.KOSMA@BLAISE.kahuna.decnet.lockheed.com>

    Date: Fri, 26-Jan-90 01:41:38-PST
    Date: Thu, 25 Jan 90 14:21:29 -0500
    From: kanderso@DINO.BBN.COM

      Here's the file I'm going to edit.  It was the biggest one I had readily
      available.
  
	    arppro13.zuu.1   53 235644(8)    !   11/10/89 12:54:56 (11/10/89)   Kosma
  
	   (scl:time (zwei:find-file #P"blaise:>kosma>dl>arppro13.zuu"))
      Evaluation of (ZWEI:FIND-FILE #P"BLAISE:>kosma>dl>arppro13.zuu") took 17.556589 seconds of elapsed time including:
  
    You might try wrapping (si:without-interrupts around your (time ...
    form.  Also, file opening time is probably a significant part of this
    test.  You might also try it on a file about twice the size.

okay, here goes:


---> Show Directory (files [default BLAISE:>*.*.newest]) BLAISE:>*.*.newest

BLAISE:>*.*.newest
  1796 free, 11924/13720 used (86%, 3 partitions) (LMFS records, 1 = 4544. 8-bit bytes)
  
      bigfoo.foo.1   105 471290(8)     !   01/26/90 11:38:17 (01/26/90)   Kosma
      File-Server.directory.1    1   DIRECTORY !   12/05/89 13:29:36 X=12/05/89 LISPM
      kosma.directory.1    1   DIRECTORY !   04/07/89 16:29:36 X=01/18/90 LISPM
      laszlo.directory.1    1   DIRECTORY !   11/22/89 19:45:35 X=12/04/89 Laszlo

108 blocks in 4 files

---> (si:without-interrupts (scl:time (zwei:find-file#P"BLAISE:>bigfoo.foo.newest")))

Evaluation of (ZWEI:FIND-FILE #P"BLAISE:>bigfoo.foo.newest") took 22.021669 seconds of elapsed time including:
  2.250 seconds processing sequence breaks,
  3.997 seconds in the storage system (including 1.379 seconds waiting for pages):
    1.727 seconds processing 530 page faults including 23 fetches,
    1.935 seconds in creating and destroying pages, and
    0.334 seconds in miscellaneous storage system tasks.
16,303 list, 930 structure words consed in WORKING-STORAGE-AREA.
55 list words consed in *HANDLER-DYNAMIC-AREA*.
18 list, 377 structure words consed in *PRESENTATION-AREA*.
366 list, 205,378 structure words consed in EDITOR-LINE-AREA.
43 list, 32 structure words consed in *ZMACS-BUFFER-AREA*.
NIL
#<ZWEI:FILE-BUFFER "bigfoo.foo > BLAISE:" 41201260>
---> 

not much improvement.  Still far too slow for me.  This is with a *local* file, 
without-interrupts.

      This works out to around 14K bytes/second, an abominable transfer rate,
      if you don't factor in overhead.  I don't know what zwei:com-read-file
      is doing internally, and I don't really care--it doesn't really matter.
      What DOES matter is that reading this file into ZMACS takes 16 to 17
      seconds, while reading the same file into emacs on my amiga takes 1 to 2
      seconds, and on our sparcstation loading it into gmacs, it's basically
      instantaneous (even I was surprized).

    I think it would be fairer to try ftp-find-file in Gnu EMACS.

why?  and by the way, I know it's not "really" a transfer rate, but nonetheless IT IS THE
RATE THAT THE USER SEES!  (that was in repsonse to another message which I don't have handy).
  
    I think that ZWEI:FIND-FILE and LOAD should be optimized as much as
    possible as these are probably the two most used I/O operations.  [See
    some ideas on this below.]


    I've suddenly found my self having to read in several larges files
    prepeatedly.  For example:

    -rw-rw-rw-  1 kanderso  5735077 Jan 23 00:57 rt.det.shrink

    The file consists of 49340 lines containing the ascii representation
    of 1085480 things as integers, single-floats, double-floats, and
    symbols.  I read these in and create CLOS instances for each.  [I/O
    may be slow, but i'm quite greatful i can do this.]

    My first version used READ.  My second version used SI:XR-READ-THING
    and internal read function.  This took 55 minutes to read in
    [:element-type 'string-char] .  I then wrote my own version of
    parse-integer [because the standard one conses when building
    double-floats and bignums], and functions to parse floats and symbols.
    Using these took 42 minutes about 30% less.

Too bad it couldn't get under *5* minutes :-/.

monty
kosma@alan.decnet.lockheed.com