Layered Lisp Effort

[Grant Ricketts <grant@Franz.COM>]

Dear Rob: 

Last Spring I spoke with Mark Fox, who in turn put me in touch with
Scott Fahlman regarding our Layered Lisp effort.  Scott volunteered
you as a key contact for this activity.  I don't know how much
information Scott gave you, but the consortium is more organized now
and I thought I should check back with you to 1) make sure you have
some information, and 2) see if you have any initial feedback.  

Attached is a backgrounder for the effort (please excuse the
formatting).  So far, we have about 18 people/companies willing to
provide input and review output for this effort.  

Upon your review, would you let us know what you think? 

Sincerely, 

Grant Ricketts 
grant@franz.com

 ."this formats letter heading
."next two lines eliminate cutmark on laserwriter
.de CM
..
.nr PS 12
.nr VS 14
.ps 12
.vs 14
.nr HM .9i
.in +.2i
.hy14
."date on next line

.sp 
\fIBackground\fR
.sp
Common Lisp is widely viewed as a highly desirable vehicle for the 
development and testing of complex systems.  The programming environments 
that have grown around Lisp are unsurpassed for productivity. Yet the
prospect of using Common Lisp as a delivery language has strong detractors,
primarily because of the size of the system.  This overhead
effectively interferes with transferring technology from research
laboratories and universities into commercial production code.
Applications prototyped in Lisp are often rewritten in other languages
for delivery.  In fact, the cost of reimplementation is so high,
another language (e.g. C) may be chosen to develop the application
regardless of the decrease in productivity.
.sp
Lisp is also frequently used as an extension language for applications
primarily written in another language.  Elisp in Emacs and AutoLisp in
AutoCAD are two such examples.  Since Common Lisp semantics are
difficult to get correct, companies which write their own embedded Lisp
extension language end up writing yet another incompatible Lisp
dialect.  There is, of course, considerable duplicated effort as
everyone goes through the mechanics of reimplementing their own Lisp
interpreter. 
.sp
\fIProposition Goals\fR
.sp
We propose to restructure the way applications are built using Common
Lisp is such a way as to dramatically lower unnecessary overhead.  The
restructuring will allow automatic specification of exactly those
components of a Common Lisp that are necessary to run a particular
program: a minimum Common Lisp that is like a shrink-wrapped version
of the application combined with the needed functionality--but no
moore.  This phase of the project will be referred to as the
\fIMinimal Common Lisp Executable"\fR section. 
.sp
We also propose to allow Common Lisp to become easily embedded in any
C application, e.g. to be used as an extension language for the
application.  This phase of the project will be part of the
\fIEmbedded Common Lisp" section. 
.sp
Therefore, the two related goals outlined in this proposal are as follows:
.sp 
.in 3
\fI1) To allow Common Lisp to become a competitive language for
delivering applications written primarily in Common Lisp ("Minimal Common 
Lisp Executable"). 
.sp
2) To allow embedding arbitrary Common Lisp functionality in applications 
primarily written in another language ("Embedded Common Lisp"). \fR
.in 0
.in .2i
.sp
These goals are related in that the technology
proposed for achieving the "Minimal Common Lisp Executable" is directly
leveraged into achieving "Embedded Common Lisp". In particular, achieving
"Embedded Common Lisp", allows one to competitively use Common Lisp as an 
embedded extension language to any application.
.sp
Achieving the goals outlined in this proposal makes Common Lisp a competitive
language in commercial applications by removing a major deterrent effect
of using Common Lisp, namely the size overhead in using a Common Lisp system.
No one has yet achieved these goals. Nobody has yet been successful
in embedding arbitrary Common Lisp functionality in applications at reasonable
cost.  Another Common Lisp supplier is experimenting with tree-shaking 
technology to deliver applications without the full Common Lisp system, but 
this method has a large manual component, and as such, is human
time-intensive and prone to error. 
It is difficult to see how tree-shaking can be fully automated.
We believe that our goals can be achieved using automated tools and as
such considerably lower the barriers to widespread use.
.sp
\f3Deliverables\fR
.sp 
The following outlines key deliverables and milestones for this
project.  Specific tasks and work statement will be developed in
conjunction with consortium members.  The primary tasks of consortium
participants will be to provide input as well as review, test and
validate results of this effort.  Franz Inc. is prepared to absorb
most of the technical development burden.  The work effort is expected
to take 50 man-months spread over 3 years. 
.sp 
1. Feasibility study: this will consist of 3 deliverables:
.in 3
.sp
(a) Implementation design for Common Lisp so applications can dynamically
resolve functionality.
.sp
(b) Implementation design for Common Lisp so applications can statically
resolve functionality.
.sp
(c) Design for interface so Common Lisp can be usefully and flexibly embedded
in an application built in another language.  Specifically, one language will 
be chosen for a detailed design: this will be C.
.in 0
.in .2i
.sp
2. Prototype Implementation of the designs in 1(a) and 1(b).
.sp
3. Commercial implementation of the designs in 1(a) and 1(b).
.sp
4. Prototype implementation of the design in 1(c).
.sp
5. Commercial implementation of the design in 1(c).
.sp