For the purposes of N·World development, foreign functions are those functions coded in a language other than Lisp, such as Fortran, Pascal, or C. Although ANSI Common Lisp does not provide for defining foreign functions, most Lisp compilers contain some facility for loading (dynamic linking) compiled foreign code into Lisp images.
Handling Foreign Functions
There are four major steps to making your foreign functions callable in Lisp:
1. Compiling and linking shared object (.so) files
2. Loading .so files into Lisp
3. Mapping C argument and return value types to Lisp variable types
4. Defining the foreign function using NFF:DEFINE-FOREIGN-FUNCTION.
For SGI IRIX 5.3 and above, all foreign code is loaded as shared-object (.so) files. Foreign code must be compiled and linked as shared object files before they can be loaded into a running Lisp world. Functions contained within the .so files can then be made callable in Lisp by defining them as foreign functions.
Compiling and Linking Shared Object Files
To create a shared object (.so) file using the c source file foo.c:
1. Compile your code with the UNIX C compiler (cc).
/usr/foo-dir/cd /usr/foo-dir
cc -c foo.c
ld -o foo.so -shared foo.o
cc -c -O -KPIC foo.c
ld -o foo.so -shared -all foo.so
Including C Libraries
You can include system (C) libraries referenced by the code in the compile and load command, as in this example:
cc -c -O -KPIC foo.c -lm -lgl_s
ld -o foo.so -shared -all foo.so -lm -lgl_scc command:
-c suppresses the loading phase of the compilation and forces an object file to be produced even if only one program is compiled.
-o outfile produces an output object file with the specified filename
Consult your system documentation for additional details about compiling files on your system.
Loading Shared Object Files
Once you've created a shared-object file, you must load it into the current Lisp world.
To load the shared-object file foo.so:
1. Use the Lisp (load) function.
NFF:DEFINE-FOREIGN-FUNCTION.
In many cases, you need to access C data objects. WOW's NFF interface provides an easy way for you to define C data types in Lisp, and use these types to create C data objects in static (C-like) memory space or in dynamic (Lisp) memory space.
Defining C Types
You define C-types with
NFF:DEFINE-C-TYPE:
NFF:DEFINE-C-TYPE (NAME DEFINITION &KEY ALLOCATION
CONSTRUCTOR ACCESSORS)
<name> is the name of the type, then the reader is named <name>, and the the writer is (SETF <NAME>). The constructor is MAKE-<NAME>.
Typedefs
You can also use NFF to typedef C-types. A C-type defined with typedef does not have constructors or allocators, and they cannot be allocated. They are intended to serve as placeholders for C-types in code for the purpose of declaring variable or return types. You can use typedefs in C-type definitions. To create a C typedef:
(nff:define-c-typedef name definition)
Type Checking
WOW's NFF interface has compile time type checking, which prevents you from defining a C-type based on an undefined type.
Example
In this example, we'll create a simple C-type called my-c-type, and define it as the C-type unsigned char.
USER(101): (nff:define-c-type MY-C-TYPE :unsigned-char)
MY-C-TYPE
USER(102): (apropos `my-c-type :user)
my-c-type [function] (X)
my-c-type [function] (&KEY NUMBER IN-FOREIGN-SPACE)
(setf my-c-type)
Creating a C Object
Use the constructor function to instantiate an object of the new type:
USER(103): (make-my-c-type)
523960144
my-c-type. For convenience, we'll bind an object to a lisp variable:
USER(104): (setf my-value (make-my-c-type))
573721680
Getting the Value of a C Object from Lisp
Use the accessor function to obtain the value contained in the memory location referred to by the pointer:
USER(105): (my-c-type my-value)
0
Setting the Value of a C Object from Lisp
Use the writer to place a value in the object referred to by the pointer bound to my-value.
USER(106): (setf (my-c-type my-value) 255)
255
Data Types and Clipping
An overflow will occur If you pass a value to a C object which is greater than the maximum allowable value for that type. When a value overflows, the modulus of the value is passed. For example:
USER(109): (setf (my-c-type my-value) 256)
256
USER(110): (my-c-type my-value)
0
Defining a Type in Lisp Space
When you define a C-type in Lisp space, the resulting objects are allocated dynamically in Lisp memory. This means that they can be moved around by Lisp just like any other Lisp data object. The advantage of this approach is that it avoids many of the memory fragmentation issues associated with static memory allocation techniques like malloc. The disadvantage is that the physical location of the C object is not guaranteed to be the same every time for the C code which makes use of it. The key, therefore, is to pass objects allocated in Lisp memory space into your foreign functions each time you call them, rather than depending on your C code to "remember" where they are between calls.
USER(113):(setf my-value (make-my-c-type in-foreign-space nil))
#(560185050 301689879)
Defining More Complex C Structures
You can use the :structure keyword to create more complex C-types. Slot definition for the structure follow the keyword in the definition:
USER(119): (nff:define-c-type MY-STRUCTURE
(:structure
(mouse-x :long)
(mouse-y :long)
(mouse-click :int)
))
typedef struct my_structure
{ long mouse-x;
long mouse-y;
int mouse-click;
}
USER(119): (nff:define-c-type MY-STRUCTURE
(:structure
(mouse-x my-c-type)
(mouse-y my-c-type)
(mouse-click my-c-type)
))
Manipulating Slots in Structures
When you create a structure using the NFF interface, all of the standard accessors and constructors are defined as well. Accessors for individual slots are defined as well. For example, given the structure MY-STRUCTURE we defined in the previous example:
USER(120): (setf my-struct (make-my-structure))
573722000
<structure> is the name of the structure and <slot> the name of the slot, the name of the accessor is <structure>-<slot>.Thus, for the mouse-x slot:
USER(122): (my-structure-mouse-x my-struct)
0
(setf (<slot accessor> <object>):
USER(123): (setf (my-structure-mouse-x my-struct) 12345)
12345
mouse-x slot yields its new value:
USER(125): (my-structure-mouse-x my-struct)
12345
Defining Arrays
You can define C arrays using WOW's NFF interface. Each element of an array has a type. For example, to create a one-dimensional array with five elements, each of which is of type my-structure (which we defined in the previous example):
USER(127): (nff:define-c-type MY-ARRAY
(:array my-structure 5))
MY-ARRAY
my-array and bind it to a Lisp variable:
USER(129): (setf my-array-var (make-my-array))
573698832
USER(130): (my-array my-array-var 0)
573698832
USER(131): (my-structure-mouse-click (my-array my-array-var
0))0
USER(132): (setf (my-structure-mouse-click (my-array
my-array-var 0)) 1)1
C-Type Utilities
WOW's NFF interface includes several useful utilities you can use to manage your foreign types and foreign functions.
Listing C Types
The global variable NFF:*C-TYPES* is a list of all currently defined c-types. You can print this variable (or use a suitable FORMAT form) to print the list of currently defined C-types.
USER(40): (nff:map-c-types #'(lambda (c-type)
(when (typep c-type `nff:built-in-c-type)
(print c-type))))#<BUILT-IN-C-TYPE :UNION>
#<BUILT-IN-C-TYPE :STRUCTURE>
#<BUILT-IN-C-TYPE :POINTER>
#<BUILT-IN-C-TYPE :BIT>
#<BUILT-IN-C-TYPE :ARRAY>
#<BUILT-IN-C-TYPE :VOID>
#<BUILT-IN-C-TYPE :SIGNED-LONG>
(not (typep c-type `nff:built-in-c-type)) predicate to return a list of all user defined C-types.
Finally, you can find a specific C-type using
NFF:FIND-C-TYPE:
USER(41):(nff:define-c-type MY-TYPE :char)
MY-TYPE
USER(42):(nff:find-c-type `my-type)
#<C-TYPE MY-TYPE>
Examining C Types
You can use the describe function to examine C-types in more detail.
USER(43): (describe (nff:find-c-type `my-type))
#<C-TYPE MY-TYPE> is the c-type named MY-TYPE.
Its definition is :CHAR.
USER(44): (describe (nff:find-c-type `my-structure))
#<C-TYPE MY-STRUCTURE> is the c-type named MY-STRUCTURE.
Its definition is
(:STRUCTURE (MOUSE-X :LONG) (MOUSE-Y :LONG) (MOUSE-CLICK :INT)).
Lisp does not require explicit typing of variables, but C does. Each argument you pass to a foreign function, or value returned from one, must be typed. FF types are merely easy-to-use names for Lisp types, which unify naming conventions for argument, return-value, and array-types. FF types also provide for compile-time consistency (which is not provided by the underlying ACL foreign-function interface.)
Using FF Types
Some, such as :VOID, can only be used to specify return values. In Table 4.2, this information is listed as a set of one or more letters in parentheses after the Category listing in the rightmost column.
Defining New FF Types
Although the WOW NFF interface includes a comprehensive suite of C-types, you may want to define your own FF type. To do so, use NFF:DEFINE-FF-TYPE:
(nff:define-ff-type name definition)
(nff:define-ff-type MYBOOLEAN :unsigned-char)
FF Type Utilities
The same utilities exist for FF-types as for C-types, only with slightly different names.
nff:find-ff-types
nff:map-ff-types
nff:*FF-TYPES*
Managing Strings
C strings are null terminated, whereas Lisp strings are not. WOW's NFF interface includes two handy utilities for translating between foreign and Lisp strings. To translate a Lisp string to a foreign (null terminated) string:
USER(44):(nff:string-to-foreign-string "A test string")
268766640
USER(45): (nff:foreign-string-to-string *)
"A test string"
Use
Defining Foreign FunctionsNFF:DEFINE-FOREIGN-FUNCTION to define foreign functions:
(nff:define-foreign-function (NAME ARGLIST &KEY LANGUAGE
ENTRY-POINT ARG-CHECKING RETURN-TYPE CALL-DIRECT CALLBACK
PROTOTYPE DOCUMENTATION)
((arg1 :fixnum)(arg2 :character))
:fortran, :c, and :ansi-c.
Example:
Assume that you are working with the following C function, read_token:
int read_token(FILE* stream, char* buffer, int size)
{...
}
read-token.so. To define the foreign function:
(nff:define-foreign-function C-READ-TOKEN-INTERNAL
((c-stream :long)(buffer :char)(size :int))
:entry-point "read_token" :return-type :int)
read_token as the Lisp function C-READ-TOKEN-INTERNAL. For example, the file pointer stream becomes the Lisp variable c-stream, of type integer.
General Issues With Foreign Functions Watch out for Bignums & C Pointers
There are several important points to remember when specifying types for variables passed from Lisp to C:
int can accommodate values up to 32 bits in size (-231<=x<=231 - 1), but the Lisp type fixnum can accommodate variables no larger than 30 bits (-229<=x<=229-1). For ints which may be larger than 30 bits, use the type integer or bignum.
When to Recompile Your Lisp Code
If calldirect is used, any callers of the foreign function must be recompiled whenever there is a change in either the original C function definition, or the Lisp nff:define-foreign-function definition. If you specify :callback, you can make changes to your C source code without recompiling calling Lisp functions. Simply create and load an updated .so file which contains the C functions you modify. However, if changes to the C code affect the arguments of a foreign function, you MUST create an updated .so file AND recompile calling Lisp functions.
.SO Load Order
Although the Franz documentation indicates that load order does not matter, Nichimen Graphics has found that random, inexplicable breaks can occur if foreign functions are defined before the relevant .so files are loaded. Load .so files first, then define foreign functions.
.SO Files Cannot Reference External Definitions
Each .so file must be complete in itself; all routines must either be defined in that .so file or must be in a library compiled with the .so file.
N·World 2.1 relied strictly on the ACL foreign function interface. Table 4.3 shows the mapping between NFF and ACL arglist type arguments, while Table 4.4 shows the concordance between return types:
Converting N·World 2.1 Foreign Functions
Another fine product from Nichimen documentation!Copyright © 1996, Nichimen Graphics Corporation. All rights reserved.