AUTOLOADING [or "intercepting undefined function errors"]

[edsel!bhopal!jonl@labrea.stanford.edu (Jon L White)]

Jeff, you may remember that I often criticized the idea of "hanging" Lisp 
features off the error handler.  Although during my time at MIT I was 
responsible for the MacLisp autoloading kludge that seemed to work this way,
I plead:
   (1) the limitations of a machine with a 256K virtual address space, and
   (2) the general ignorance of the early 1970's.
In fact, in Interlisp, there were several competing kludges hanging off
the error handler, and one could easily become confused as to just
which kludge would have priority!

Speaking, now, as one who has been around the Lisp community a long
time (rather than as a spokesman for Lucid), might I suggest that the
total number of autoloadable names in an application may well permit
a more space-consumptive solution.  What do you think, for example, of
the following approach:

    (defun set-up-autoloadable (fn-name path &optional module-name)
      ;; Calling this fn establishes a stub definition for 'fn-name', which
      ;;  just loads in the file containing its definition, and restarts
      ;;  the original call to 'fn-name'.
      (let ((name-and-path-data `#(,fn-name ,path ,module-name)))
        (define-function fn-name
            #'(lambda (&rest args) (do-autoloading name-and-path-data args)))
        fn-name))

    (defun do-autoloading (name-and-path-data args)
      ;; Use CL's "require" when possible, so that we don't spuriously
      ;;  load in a fruitless file.  Since the "stub" definition is unique,
      ;;  it will be replaced by something else in a successful loading.
      (let ((fn-name       (svref name-and-path-data 0))
            (path          (svref name-and-path-data 1))
            (module-name   (svref name-and-path-data 2)))
        (let ((old-proc (symbol-function fn-name)))
          (if module-name
              (require module-name path)
              (load path))
          (when (eq old-proc (symbol-function fn-name))
            (error "~&AUTOLOAD failure: no definition for ~S after loading ~S"
                   fn-name path))
          (apply (symbol-function fn-name) args))))


When these two "enabling" functions are compiled in Lucid Common Lisp, then 
each call to 'set-up-autoloadable' will cons only about 10 Q's (a Q is a 
pointer-sized quantum of memory).  This would be in addition to the amount 
of storage taken for the most minimal "database" to hold a mapping from 
symbols to file pathnames.  A little arithmetic will show that such 
"databasing" could easily cost 20 Q's per entry (depending on the length of
the symbol's name and the length of the file's name).  

To make 1000 functions "autoloadable" by this non-intrusive approach, the 
cost in storage  might be 30K words rather than 20K words.  Not a lot extra 
to pay for forgoing a Lisp-Lobotomy.   And after a disksave is done, the 
whole kit-and-kaboodle will put into "static" area; so there should be
relatively little impact on anything other than virtual memory consumption.


-- JonL --


P.S. Lexical closures really work.