1CL-CXX-JIT - Common Lisp C++ JIT for exposing C++ functions

This library provides an interface to C++ from lisp. It compiles C++ code, then loads it into lisp. It was inspired by RCRL and the older project CL-CXX.

1.1API

(from '("list of string headers") 'import "normal string is inserted as it is" '("function/function-pointer/mem-fun-pointer/lambda" . "name-used-in-lisp"))

1.2Examples

1.2.1SDL2 Example

(ql:quickload :cxx-jit)
(setf cxx-jit:*cxx-compiler-link-libs* "-lGL -lSDL2 -lSDL2main")

(cxx-jit:from '(
                "<SDL2/SDL.h>")
              'import
              '("[](){return SDL_Init(SDL_Init(SDL_INIT_VIDEO));}" . "init")
              '("SDL_CreateWindow" . "create-window")
              '("SDL_CreateRenderer" . "create-renderer")
              '("SDL_SetRenderDrawColor" . "set-color")
              '("SDL_DestroyWindow" . "destroy-window")
              '("SDL_RenderClear" . "clear-renderer")
              '("SDL_RenderPresent" . "renderer-render")
              '("SDL_Quit" . "sdl-quit"))

(init)
(setf wind (create-window "create-window" 0 0 600 700 0))
(setf rend (create-renderer wind -1 0))
(loop for x to (* 255 3)
      for r = (if (> x 255) 255 x)
      for g = (if (> x 255) (if (> x (* 2 255)) 255 (rem x 256)) 0)
      for b = (if (> x (* 2 255)) (rem x 256) 0)
      do
         (print x)
         (set-color rend r g b 255)
         (clear-renderer rend)
         (renderer-render rend)
         (sleep 0.01))

(destroy-window wind)
(sdl-quit)

1.2.2Basic Example

Start with (ql:quickload :cxx-jit) (in-package cxx-jit)

  (from '("<string>") 'import '("[](std::string x){return \"Hi, \"+x;}" . "hi"))
  (hi "there!")

  (from '("<cmath>") 'import '("static_cast<double(*)(double)>(std::sin)" . "cpp-sin"))
  (cpp-sin 0d0)
  (cpp-sin pi)
  (cpp-sin (/ pi 2))

  (from nil 'import "struct C{ auto hi(){return \"Hello, World\\n\";} auto bye(){return \"Bye\";} };" '("&C::bye" . "bye") '("&C::hi" . "hi") '("[](){static C x; return x;}" . "cc"))
  (cc)
  (hi *)
  (bye **)
    ;;; structure  definition could be written in a header file then be used as the following:
  (from '("c.hpp") 'import '("&C::bye" . "bye") '("&C::hi" . "hi") '("[](){static C x; return x;}" . "cc"))

1.2.3Eigen Library Example

Start with (ql:quickload :cxx-jit) (in-package cxx-jit)

  (from '("<Eigen/Core>" "<Eigen/Core>") 'import '("[](Eigen::CwiseNullaryOp<Eigen::internal::scalar_identity_op<double>,Eigen::Matrix<double, 3, 3>> x){
    std::stringstream s;
    s << x;
    return s.str();}" . "print-matrix"))

  (from '("<Eigen/Core>" "<Eigen/Core>") 'import '("static_cast<const Eigen::CwiseNullaryOp<Eigen::internal::scalar_identity_op<double>,Eigen::Matrix<double, 3, 3>> (*)()> (&Eigen::Matrix3d::Identity)" . "identity-matrix"))

  (print-matrix (identity-matrix))

1.3Prerequisites

• common lisp supporting CFFI
• working C++17 compiler
• the following variables should be set according to OS and compiler used

variable	default value
*cxx-compiler-executable-path*	"/usr/bin/g++"
*cxx-compiler-flags*	"-std=c++17 -Wall -Wextra -I/usr/include/eigen3"
*cxx-compiler-working-directory*	"/tmp/" #\/ '/' should be the last character
+cxx-compiler-lib-name+	(intern "plugin")
+cxx-compiler-wrap-cxx-path+	shouldn't be changed "path to wrap-cxx.cpp"
*cxx-compiler-internal-flags*	"-shared -fPIC -Wl,--no-undefined -Wl,--no-allow-shlib-undefined"
	for g++ and for clang++ "-shared -fPIC -Wl,-undefined,error -Wl,-flat_namespace"
*cxx-compiler-link-libs*	"-lm" these flags are added after "-o output" to link correctly
*cxx-type-name-to-cffi-type-symbol-alist*	alist to map c++ type name to cffi type example:"(push (cons "long unsigned int" :ulong) *)"

1.4Installation

Clone into home/common-lisp directory. Then (ql:quickload :cxx-jit-test)

1.5Supported Types

C++ type	Lisp cffi type
fundamental	same
string	:string
class	:pointer
std::is_function	:pointer
other	not implemented!

1.6Under The Hood

• function/lambda/member_function/function_pointer is wrapped into a dummy lambda class to have a unique template specialization.

         Import([&]() { return __VA_ARGS__; });
  

• Import function calls DecayThenResolve with function pointer as the template specialization so thunk pointer is omitted and we only return the direct function pointer which will be used from lisp side.
• InvocableTypeName returns a vector contains: [return type, class type for class function member, args]. It resolves C++ types as follows:
  • Fundamental types and pointers are passed directly
  • String is converted to char* with new[] operator, should be cleared with ClCxxDeleteObject(ptr, true)
  • Class/std::is_function is converted to void* with new[] operator, should be cleared with ClCxxDeleteObject(ptr, false)
  • rest report an issue for other cases
• Meta data for each function defined is passed through a lisp callback with this data:

  typedef struct {
    // could be void*
    void (*thunk_ptr)();
    bool method_p;
    const char **type;  // memory handled in C++
    std::uint8_t type_size;
  } MetaData;
  

1.7NOTE

Tested on:

• SBCL 2.0.1 on debian

1.8Todo List

1.8.1TODOAdd redirect stdout : freopen("/tmp/tmp.txt", "w", stdout); @apemangr

1.8.2TODOUse trivial-garbage with ClCxxDeleteObject

1.8.3TODOAdd non-polling from

1.8.4TODOTest functions

1.8.5TODOBenchmark

1.8.6TODOBetter class interface

1.9Copyright

Copyright (c) 2021 Islam Omar ([email protected])

1.10License

Licensed under the MIT License.