MCL & Apple

[chewy@shell.portal.com (Paul Snively)]

I've been following this discussion with considerable interest, as I'm
sure so many others have.

I'm Paul Snively.  I was a MacDTS engineer at Apple for two and a half
years, during which I was the only MacDTS engineer supporting Common
Lisp on the Macintosh.  Of course, this also was a well-kept secret, as
one of the realities of Apple's purchase of Coral Software was that
many of Coral's existing customers were government and academic sites,
not Apple Partners, and therefore had not purchased the right to MacDTS
support.  Nevertheless, I was able to provide some help to those whom I
managed to come into contact with through channels other than
AppleLink.

It has always been my very great pleasure to see the real pushing of
the Macintosh programming envelope coming from the realm of the dynamic
languages, Lisp particularly, and Common Lisp especially.  I remember
the heady early days of Coral Common Lisp--a Common Lisp that actually
ran on a two megabyte Macintosh Plus.  I remember first acquiring
Portable CommonLOOPS (and hacking away at the sources as various CCL
optimizations broke between releases of CCL and releases of PCL).  I
remember meeting a Texan oil-industry consultant (Michael, are you out
there?) who showed me his graphical front-end to a huge Oracle
database--a map of Texas that he could zoom in on to show me tanks,
pipelines, flow amount, direction, and quantity, etc.  The whole
front-end had been written in CCL (MACL, by then) and PCL.

When I was at ICOM Simulations, writing the adventure game Deja Vu II,
we needed to compress data, including what text there was in the game. 
Our most brilliant colleague, Waldemar Horwat, had written a
common-substring elimination compression scheme in Common Lisp.  John
Lawler, who was then in the AI department of one of the Big Eight
accounting firms in Chicago, generously allowed me to use Allergo CL on
one of their sizable Sun workstations one evening after hours.  So I
trotted down to Chicago one night at about 8:00, hard disk with Deja Vu
II's text file and Waldemar's program in hand.

Deja Vu II was the largest MacVenture ICOM Simulations ever did.  The
Sun--and I cannot recall how much memory it had, how many other
processes were running or any of it--choked on Waldemar's program when
Deja Vu II's text was the data to operate on.  Ultimately, John and I
stripped _everything_ we could think to remove from a Mac II in his
office, and MACL ran Waldemar's program on the second try--on an eight
megabyte Macintosh II.

Granted, this is an example of how MCL allowed us to write a knock-off
utility--a knock-off utility that proved critical to the completion of
an award-winning adventure game, however.

(BTW, thanks again, John... I never realized how thoroughly deserted
the streets of Chicago's financial district are at midnight.  Let's go
back someday and have those buffalo wings we talked about at that place
on Division (two on the menu, three off).  And sorry about the
garbageman thing; I'd have loved to do it but we didn't have room.)

After ICOM, I wound up at Apple in MacDTS, where I came into frequent
contact with folks in Apple's Advanced Technology Group.  There was
(and probably still is) some extremely exciting work being done there
based on MCL.  For example, anyone who's seen ATG's presentations at
recent WWDCs has probably seen SK8, an intelligent multimedia authoring
tool with a distinctly AppleScript-like scripting language.  The
history of the SK8 project is, to me, rather interesting: when I first
encountered it, the project was called MacFrames (it was a frame-based
knowledge-representation system) and its uses were a little different. 
For example, yes, there was research being done into novel human
interface approaches, including constraint-based layout and the like. 
Another group at Apple, concerned with the QA process, was using
MacFrames to drive the Agent VU portion of the Virtual User testing
system by creating state-space models of popular applications and
`putting the apps through their paces' by literally having controlling
Macs enumerate all of the possible state-transitions that could be
generated by a user, and hence by Virtual User (of course, this
presumes that the state-space model was complete--admittedly an
extremely difficult task to complete, let alone prove correct).

Without a doubt, though, the most compelling project at ATG to me was
the Eager project.  An ATG researcher had taken HyperCard and hacked a
simple InterProcess Communication system into it (this obviously
predates System 7.0).  MacFrames also supported this rudimentary IPC,
so MacFrames and this slightly-customized HyperCard could talk to each
other.  The researcher then used MacFrames to create a taxonomy, as it
were, of all possible actions _and the HyperTalk to reproduce them_ in
HyperCard--again, a seemingly daunting task, but not at all impossible.
 He then added some sophisticated pattern-recognition features to the
model.  The result was a program that sat in the background under
MultiFinder, watched what the user did in HyperCard and, when it
recognized a pattern of repetitive user actions, would offer to a)
complete the actions for the user, b) provide a HyperTalk script to do
what the user was doing, or c) go away.

For example, let's say you had a HyperCard stack of netnews posts. ;-) 
The stack consists of one post to a card, with the subject field being
the first line of the text field that the card contains.  You wish to
create a summary card containing the enumeration of all of the subject
lines, so you go to the first card in the stack and create a new card,
giving it a text field.  On the first line of the text field, you type
"1) ", move to the next card, select the first line, copy it, go to the
first card, and paste the subject after the space.  Then you go down to
the next line, type "2) ", go to the third card, select the first line,
copy it, go to the first card, paste the subject, go to the next line,
type "3) ", go to the fourth card...

By this time, Eager would be jumping up and down (Eager literally had a
small window with a picture of a cartoon character sitting in a chair,
watching you work.  When he thought he recognized a pattern, he leaned
forward in his chair, looking intent.  The hacked HyperCard would
highlight use-anticipated human-interface elements--the menu that it
thought you were about to pull down would be highlighted in red, for
example, and then the menu item it was anticipating would be.  If you
verified its guesses by selecting the highlighted items, the cartoon
character would leap out of his chair and jump up and down).  If you
brought Eager to the foreground, you would get a dialog box asking
whether to finish for you, create a HyperTalk script to do what you
were doing, or go away.  If you had Eager finish for you, you got to
see the blur of activity as the program correctly finished making your
netnews stack summary for you--incrementing the summary line numbers,
selecting the first line of each card, copying it, and pasting it into
the appropriate line in the summary card.  If you had Eager generate a
HyperTalk script for you, you'd get a window with a script that you
could copy and paste into whichever HyperCard element was appropriate.

Personally, I think Eager should be taken as a proof-of-concept and
generalized to support the automation of AppleScript-savvy (recordable)
applications.  Instead of a hacked IPC mechanism and HyperCard, use the
recordability features to watch the user-actions of recordable apps, do
the pattern matching against the published AppleEvent suites and the
private events defined by the aete resource, and make an agent that
will either complete the work (force-feeding the app AppleEvents) or
write some AppleScript to do the job for you.  You'll need a good
knowledge-representation system to do this.  I suggest one of the
several that have been written in MCL.

Another poster has mentioned LOOM, which I had FTPed a while back and
begun playing with.  It's a very powerful knowledge representation tool
along the MacFrames line and then some.  The LOOM tutorial covers such
application domains as crisis planning--you need to figure out how to
get critical resources to a region that is difficult to get to and has
been made more so thanks to military action.  LOOM can help you do
this, but it can only help you do this on a Mac if you have MCL.

If I'm into computer music, chances are I'm going to FTP to
ccrma-ftp.stanford.edu and prowl around.  What do I find?  A
Smalltalk-80 directory that hasn't been updated since 1992 and a Lisp
directory that is continually developed, with systems like Common
Music, Common Music Notation, and Common Lisp Music.  Again, some of
these systems have been written with MCL in mind for the sake of the
interface to the expressive power of the underlying language.  And MIDI
control is not a task for lightweight Macs.  Imagine what a PowerPC
with a native MCL and a decent MIDI interface could do. Now imagine
Chip Davis, Chic Corea, and other famous Mac/MIDI enthusiasts using
CCRMA's tools on PowerPC Macs.

A common theme among the posts expressing their concern about the fate
of MCL seems to be that Apple is focusing exclusively on the
shrink-wrapped horizontal software developer that's trying to create
yet another word processor, spreadsheet, database, or video-production
system.  The vertical-market developer, the corporate developer, the
researcher in academia or industry all represent market segments that
Apple has tremendous difficulty identifying, much less addressing.  You
probably won't find the fruits of their labor sitting on the shelf at
Egghead at anywhere from $49.95 to $699.95.  Instead, you'll find
things like the oil-tracking system being consulted for to the tune of
thousands or tens of thousands of dollars or more--and probably selling
tens of thousands of dollars' worth of Macintoshes, as well.

Another common theme seems to be that as Apple says with one mouth that
dynamic languages are the solution to many development problems that
programmers face today, it says with another mouth that it doesn't have
the resources to bring the _only dynamic language it currently offers_
to what is obviously already becoming its flagship CPU family.  While
Dylan may prove to be a definite improvement specifically as a dynamic
delivery vehicle, for Apple to ignore the existing body of Common Lisp
code in general, and MCL code in particular, would seem to be a rather
grievous error in judgement, not only because of the resulting effects
on the Mac-buying Common Lisp community, but because of the apparently
increasingly-accurate assessment on the part of the developer community
that Apple simply cannot form and promote a consistent strategy with
respect to its development tools--or, for that matter, its developers.

There are innovators out there.  Some of them are using MCL.  You've
heard from MIT's Media Lab, Flavors Technology, and others.  Even if
they don't deliver with MCL, they prototype with it and deliver in C++
or some other more mainstream language, but ultimately, they sell Macs.
 With Apple's market share hovering at a seemingly impenetrable 10-13%,
it's time for Apple to stop driving developers away by trying to profit
from them and start investing in them.

Paul Snively
Former MacDTS Engineer and Dynamic Language Supporter
chewy@shell.portal.com
----------------------------------------------------------------------
Just because you're paranoid, it doesn't mean that no one is out to
get you!