[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
foreign function calls
I have some questions about the foreign function interface. I'm not sure
of the best place to send this kind of question, so I'm cc-ing both
email@example.com and firstname.lastname@example.org.
We are trying to create an interface to a fortran package for constrained
optimization called NPSOL. NPSOL normally takes user supplied
(fortran) subroutines to compute the objective function to be minimized,
its gradient, etc. We want the user to be able to supply lisp functions
rather than fortran subroutines, which means we need to call back to lisp
from fortran (or from C called from fortran).
Before I invest a lot of time, I'd like to clear up a few points that
I find confusing in ch. 10 of the User Guide.
I'll appreciate any help or suggestions.
The recommended scenario seems to be something like this:
1) Lisp registers the data (several NxN arrays of double-floats,
where N may be as much as 1000) to be passed to NPSOL using
ff:register-value and registers the lisp objective and gradient
functions (eg. objfun-l) using ff:register-function.
2) Lisp calls a C wrapper for NPSOL (say npsol_c), passing the indexes of the
registered data and functions.
3) npsol_c saves the indexes in global variables.
It retrieves pointers to the lisp arrays using lisp_value.
It calls NPSOL with the pointers to lisp arrays and with
fixed C procedures as the objective and gradient subroutines
4) NPSOL, in the course of minimization calls the C procedures (eg. objfun_c),
with arguments that are pointers to lisp arrays.
5) objfun_c looks in the appropriate global variable to find the index
of objfun-l and calls it (using lisp_call) with arguments that
are pointers to lisp arrays that it got from NPSOL.
6) A garbage collection may occur. (Section 10.8 seems to imply that gc
cannot happen during the execution of foreign code, except during
the call back to the lisp function. Is this true---even with
multiprocessing? If it's not true, how can any foreign function
call work reliably?)
7) At this point, any pointers to lisp arrays in foreign code may be invalid,
so objfun_c should refetch the pointers to lisp arrays, using
lisp_value, before returning control to NPSOL.
The problem with this scenario is that there are pointers to lisp arrays
scattered throughout NPSOL that objfun_c can't possibly fix. I can imagine
several ways to deal with the problem:
a) Suspend all scavenging during the execution of foreign code.
As far as I can tell, there is no mechanism provided to do this.
b) Arrange to tenure all the lisp objects to be passed to foreign code
and prevent global gc by (setf excl:*global-gc-behavior* nil).
The problem with this is that there doesn't seem to be any way to
guarantee that the lisp objects are tenured. Sections 3.1.2 and
3.1.4 are contradictory on this point. Item (3) in 3.1.4 claims that
(excl:gc :tenure) "will cause ALL live objects to be tenured,"
but the description of excl:gc in 3.1.2 warns that
"You should not assume that all objects will be tenured..."
c) Have the wrapper npsol_c copy all the lisp arrays to foreign arrays,
except for those that are going to be passed back to lisp from
objfun_c. This is undesirable because, for large problems,
it means duplicating tens of megabytes of data. This raises a
question---can malloc be used in code loaded into lisp, or do I
have to predeclare large global arrays to be used to hold the copies?
System description: 1990-8-10 15:24:19
Machine: Sun4 id: 587210204
Lisp: Allegro CL
18.104.22.168 [Sun4] (0/0/0).
Features: (LOOP LOOP COMPOSER FAKE-ACTIVE-REGIONS PCL CLOS BIG-ENDIAN GSGC
ALLEGRO-V3.1 FRANZ-INC EXCL ALLEGRO COMMON-LISP CONFORMING-IEEE IEEE
FLAVORS SUNOS4.0 SUN SPARC SUN4 UNIX MULTIPROCESSING CLX XLIB
CLX-MIT-R4 CLX-CL-ERROR CW-X).
John Alan McDonald, Statistics, GN-22, U of W, Seattle, Wa. 98195
(206) 545 7438, 543 2517