[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
- To: firstname.lastname@example.org
- Subject: fast reading
- From: email@example.com (Alan Ruttenberg)
- Date: Thu, 29 Dec 1994 03:50:15 GMT
- Organization: MIT Media Laboratory Learning an Common Sense Group
- Sender: firstname.lastname@example.org
In article <email@example.com> firstname.lastname@example.org (Peter Szolovits) writes:
I need to read in some fairly large tables of data, in tab-delimited
format ... Fast!
A three ideas come to mind, they are - from slowest to fastest:
1) Use (set-macro-character #\tab #'(lambda(&rest ignore) :tab)) , and
use the usual read, or for something faster: stream-read.
2) Use the function (stream-reader stream), which returns a fast
function which can be used like read-byte, and do character oriented
parsing of the file yourself. Stream-reader is documented in the mcl
3) Use the mac-file-io example to read the whole file into the mac
heap and use your own parsing function using (%get-byte macptr offset)
to access each character as a number (or use %code-char to change it
into a character).
You would get the best code by using (declare (optimize (speed 3)
(safety 0))) , declaring your loop variable as fixnum and using
something like (the fixnum (%get-byte (the macptr pointer) (the fixnum
offset))) as the byte accessor.
Using a c-like %inc-ptr instead of the offset to move the pointer
would be less efficient (because of way pointers are implemented in
mcl), as would using the %code-char, since you can write the parsing
routines just as well using bytes and you avoid that much overhead.
If you are not constantly getting new data then you could use a really
slow routine to read the data (overnight) into an array and then
create a fasl file out of that, which will read in pretty fast. The
trick there is to compile a file which has one line in it
(setq the-answer #.the-array)
Compiling this file will create a fasl file with your data. After
loading the fasl file, the-answer will hold your data. I'm
assuming that your data is all immediates - strings and numbers.
The compiler doesn't write out some kinds of objects.
Hope this helps,