Re: foreign function interface

[William.Lott@cs.cmu.edu]

wolf@prosun.first.gmd.de (Wolfgang Koehler) writes:
> Hello,
> 
> in one of the latest distributions of the files "cmu-user.tex" we read that
> there should be a possibility for passing procedure pointers to a C routine.
> (macro "def-c-procedure"). We are very interested in such a mechanism,
> because we want to include an existing C++ - object-system into the Lisp.
> 
> But it seems that in the release "15e-source.tar.Z" we got, this will not 
> work because in the source files "/code/c-call.lisp" these macros are
> ex-commented. (#|...|#)
> 
> It would be very kind of you to send us a mail what the current state of
> this topic is, and if it makes sense to wait for a subsequent version,
> where it is completely solved. 
> 
> Otherwise some hints how to do this are greatly appreciated, even if the
> calling mechanism is very restricted.
> (We have some good hackers here, they whopefully will solve the problems).
> 
> Many thanks in advance
> 
> Wolfgang Koehler
> GMD - First an der TU Berlin
> Hardenbergplatz 2
> 1000 Berlin 12 
> Tel. 030 25499-273
> 
> wolf@first.gmd.de

The cmu-user manual lied.  Basically, a long time ago, there was this
earlier version of CMUCL that only ran on IBM PC/RT's that supported
DEF-C-PROCEDURE.  That section of the user manual dates back to that
older system.  We have recently rewritten all of the aliens/foreign
function stuff and the associated documentation.  We will be releasing
this to the net sometime Real Soon Now.

If you really need to call lisp functions from C, it can be done, but
is non-trivial.  The reason why we haven't build a general tool for
this is because demand for it is kinda rare (you are only the second
person to ask for it in over 2 years), and because there is no easy
general way to solve all the problems.  But it can be done.

In order to call a Lisp function from C, you have to build a stack
frame on the Lisp stack and use the C routine ``call_into_lisp'' to
invoke the function.  Look in .../ldb/monitor.c for an example of how
to do this.

But the bigest problem is that the garbage collector doesn't know
anything about the C heap.  So if your C code holds onto Lisp pointers
(data or functions), they will not be fixed up the next time the
garbage collector runs.  So you have to either make sure the garbage
collector doesn't run while C has a handle on these pointers, or make
sure your pointers in C land get fixed up.

To keep the garbage collector from running while you are in C, use the
WITHOUT-GCING macro:

	(defun do-it ()
	  (system:without-gcing
	    (random-c-routine data #'call-me-with-result)))

	(defvar *results* nil)
	(defun call-me-with-result (result)
	  (push result *results*))

Obviously, this will only work if you are not allocating much storage.
Also, this will only work if the C code doesn't need to maintain any
state involving Lisp objects between calls.  If you need to maintain
state, then you have to re-supply C with all the lisp pointers after a
GC:

	(defun tell-C-about-Lisp-pointers ()
	  (remember-result-function #'record-result))
	(push #'tell-C-about-Lisp-pointers ext:*after-gc-hooks*)

And then have the C code know that whenever execution ventures into
Lisp, it might garbage collect, and hence needs to re-extract the Lisp
function pointer from wherever remember-result-function put it.

No matter what, you probably want to turn off interrupts for the
duration.  Otherwise, if you interrupted while in C and the interrupt
handler triggered a GC, there is no way you can fix any Lisp values
that were in registers at the time of the interrupt.

-William Lott
CMU Common Lisp Group