Re: saving a numeric table to a disk file

[hmadorf@eso.org]

Bill St Clair's solution posted earlier on this mail exploder 
works just fine. And despite the appearance of the CCL package in
his quick hack, the solution is completely portable (provided one
replaces the package name).

I tested it on a simple array of 10x1000 floats and it was 10 times 
faster than writing the array into a .lisp file and compiling this.

Also the amount of consing is minimal. Bill's solution allocates
about 5.8 kByte whereas the ASCII print conses up 3.6 MBytes!!! 

Below are the details of my timing experiments using MCL2.0b1p3 on a 
Mac SE/30 with 8MB memory.

Thanks again to Bill.

Hans-Martin Adorf
Space Telescope - European Coordinating Facility
European Southern Observatory
adorf@eso.com

;;;; -*- Mode: Common-Lisp; Package: CL-USER; -*-
;;;;
;;;; save-restore.lisp
;;;;
;;;; Fast save and restore an object.
;;;; 
;;;; Hans-Martin Adorf, ST-ECF, adorf@eso.org
;;;;
;;;; Using a method suggested by Bill St. Clair, Apple, bill@cambridge.apple.com
;;;;
;;;; Cooperates with a file "save-value.lisp" which contains
#|
; save-value.lisp

(in-package "CL-USER")

(setq *value* '#.*value*)
|#

(export '(save-value-to-file restore-value-from-file))

(defvar *value*)

(defparameter *save-value-source-file*
  ; Change to another directory if you like
  "lispm:db;save-value.lisp")

(defun save-value-to-file (value file)
  (let ((*value* value))
    (compile-file *save-value-source-file*
                  :output-file file)))

(defun restore-value-from-file (file)
  (let ((*value* nil))
    (load file)
    *value*))

#|
;; Test forms
;; Simple and slow method using an external ASCII representation:
(defvar *db*)
(progn 
  (setq *db* (make-array '(10 1000) 
                         :element-type '(simple-array float)
                         :initial-element 1.0))
  t)                                                        ;suppress output
(array-dimensions *db*)
(gc)
(time (with-open-file (output-stream "lispm:db;test-dat.lisp" 
                                     :direction :output
                                     :if-does-not-exist :create
                                     :if-exists :supersede)
        (print *db* output-stream)
        t))                                                 ;55.633 sec
(time (compile-file "lispm:db;test-dat"))                   ;29.267 sec
(time (load "lispm:test-db;db"))

;; Using the fast method suggested by Bill St. Clair:
(gc)
(time (save-value-to-file *db* "lispm:db;test-db"))         ; 5.550 sec sec
(setq *db* nil)
(time (progn 
        (setq *db* (restore-value-from-file "lispm:db;test-db"))
        t))                                                 ; 0.733 sec
|#

#|
Test on a 10x1000 array:

(WITH-OPEN-FILE (OUTPUT-STREAM "lispm:db;test-dat.lisp" 
                               :DIRECTION :OUTPUT 
                               :IF-DOES-NOT-EXIST :CREATE 
                               :IF-EXISTS :SUPERSEDE) 
  (PRINT *DB* OUTPUT-STREAM) NIL) took 3027 ticks (50.450 seconds) to run.
Of that, 267 ticks (4.450 seconds) were spent in The Cooperative
Multitasking Experience.
Of that, 201 ticks (3.350 seconds) was spent in GC.
 3613520 bytes of memory allocated.

(COMPILE-FILE "lispm:db;test-dat") took 1771 ticks (29.517 seconds) to run.
Of that, 161 ticks (2.683 seconds) were spent in The Cooperative
Multitasking Experience.
 224184 bytes of memory allocated.

(LOAD "lispm:db;db") took 48 ticks (0.800 seconds) to run.
Of that, 6 ticks (0.100 seconds) were spent in The Cooperative
Multitasking Experience.
 41816 bytes of memory allocated.

(SAVE-VALUE-TO-FILE *DB* "lispm:db;test-db") took 331 ticks (5.517 seconds) to run.
Of that, 25 ticks (0.417 seconds) were spent in The Cooperative
Multitasking Experience.
 5872 bytes of memory allocated.

(PROGN (RESTORE-VALUE-FROM-FILE "lispm:db;test-db") NIL) took 65 ticks
(1.083 seconds) to run.
Of that, 7 ticks (0.117 seconds) were spent in The Cooperative
Multitasking Experience.
 42192 bytes of memory allocated.
|#