[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: LISPM I/O performance
Here's the file I'm going to edit. It was the biggest one I had readily
arppro13.zuu.1 53 235644(8) ! 11/10/89 12:54:56 (11/10/89) Kosma
(scl:time (zwei:find-file #P"blaise:>kosma>dl>arppro13.zuu"))
Evaluation of (ZWEI:FIND-FILE #P"BLAISE:>kosma>dl>arppro13.zuu") took 17.556589 seconds of elapsed time including:
0.755 seconds processing sequence breaks,
2.358 seconds in the storage system (including 0.032 seconds waiting for pages):
0.365 seconds processing 1657 page faults including 1 fetches,
1.994 seconds in creating and destroying pages, and
0.000 seconds in miscellaneous storage system tasks.
9,083 list, 505 structure words consed in WORKING-STORAGE-AREA.
102,689 structure words consed in EDITOR-LINE-AREA.
45 list, 32 structure words consed in *ZMACS-BUFFER-AREA*.
#<ZWEI:FILE-BUFFER "arppro13.zuu >kosma>dl BLAISE:" 43600407>
You might try wrapping (si:without-interrupts around your (time ...
form. Also, file opening time is probably a significant part of this
test. You might also try it on a file about twice the size.
Then, I remembered that I just did that over the ethernet (TCP) so I
tried again, first copying the file to my local machine; there was about
a second difference overall:
Evaluation of (ZWEI:FIND-FILE #P"KARL:>foo.foo") took 16.414146 seconds
of elapsed time ...
This works out to around 14K bytes/second, an abominable transfer rate,
if you don't factor in overhead. I don't know what zwei:com-read-file
is doing internally, and I don't really care--it doesn't really matter.
What DOES matter is that reading this file into ZMACS takes 16 to 17
seconds, while reading the same file into emacs on my amiga takes 1 to 2
seconds, and on our sparcstation loading it into gmacs, it's basically
instantaneous (even I was surprized).
I think it would be fairer to try ftp-find-file in Gnu EMACS.
I think that ZWEI:FIND-FILE and LOAD should be optimized as much as
possible as these are probably the two most used I/O operations. [See
some ideas on this below.]
Anyway, I'll be very interested, after you make your program
either use a binary file, or if you feel you can't do that,
you use :STRING-LINE-IN and PARSE-INTEGER, or :READ-INPUT-BUFFER,
some care about buffer boundaries, and PARSE-INTEGER, and then
you come back and tell us: "It takes X seconds, and that's too
slow". That will be good and useful, and I hope you do it.
where does something like READ-LINE fit into all this? I've used that in
READ-LINE cons a new line each time it is called. For most
applications this is needless. :STRING-LINE-IN will fill a line that
you provide it, so no consing is done.
I've suddenly found my self having to read in several larges files
prepeatedly. For example:
-rw-rw-rw- 1 kanderso 5735077 Jan 23 00:57 rt.det.shrink
The file consists of 49340 lines containing the ascii representation
of 1085480 things as integers, single-floats, double-floats, and
symbols. I read these in and create CLOS instances for each. [I/O
may be slow, but i'm quite greatful i can do this.]
My first version used READ. My second version used SI:XR-READ-THING
and internal read function. This took 55 minutes to read in
[:element-type 'string-char] . I then wrote my own version of
parse-integer [because the standard one conses when building
double-floats and bignums], and functions to parse floats and symbols.
Using these took 42 minutes about 30% less.