ext_ffi.html (10336B)
1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> 2 <html> 3 <head> 4 <title>FFI Library</title> 5 <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> 6 <meta name="Author" content="Mike Pall"> 7 <meta name="Copyright" content="Copyright (C) 2005-2016, Mike Pall"> 8 <meta name="Language" content="en"> 9 <link rel="stylesheet" type="text/css" href="bluequad.css" media="screen"> 10 <link rel="stylesheet" type="text/css" href="bluequad-print.css" media="print"> 11 </head> 12 <body> 13 <div id="site"> 14 <a href="http://luajit.org"><span>Lua<span id="logo">JIT</span></span></a> 15 </div> 16 <div id="head"> 17 <h1>FFI Library</h1> 18 </div> 19 <div id="nav"> 20 <ul><li> 21 <a href="luajit.html">LuaJIT</a> 22 <ul><li> 23 <a href="http://luajit.org/download.html">Download <span class="ext">»</span></a> 24 </li><li> 25 <a href="install.html">Installation</a> 26 </li><li> 27 <a href="running.html">Running</a> 28 </li></ul> 29 </li><li> 30 <a href="extensions.html">Extensions</a> 31 <ul><li> 32 <a class="current" href="ext_ffi.html">FFI Library</a> 33 <ul><li> 34 <a href="ext_ffi_tutorial.html">FFI Tutorial</a> 35 </li><li> 36 <a href="ext_ffi_api.html">ffi.* API</a> 37 </li><li> 38 <a href="ext_ffi_semantics.html">FFI Semantics</a> 39 </li></ul> 40 </li><li> 41 <a href="ext_jit.html">jit.* Library</a> 42 </li><li> 43 <a href="ext_c_api.html">Lua/C API</a> 44 </li><li> 45 <a href="ext_profiler.html">Profiler</a> 46 </li></ul> 47 </li><li> 48 <a href="status.html">Status</a> 49 <ul><li> 50 <a href="changes.html">Changes</a> 51 </li></ul> 52 </li><li> 53 <a href="faq.html">FAQ</a> 54 </li><li> 55 <a href="http://luajit.org/performance.html">Performance <span class="ext">»</span></a> 56 </li><li> 57 <a href="http://wiki.luajit.org/">Wiki <span class="ext">»</span></a> 58 </li><li> 59 <a href="http://luajit.org/list.html">Mailing List <span class="ext">»</span></a> 60 </li></ul> 61 </div> 62 <div id="main"> 63 <p> 64 65 The FFI library allows <b>calling external C functions</b> and 66 <b>using C data structures</b> from pure Lua code. 67 68 </p> 69 <p> 70 71 The FFI library largely obviates the need to write tedious manual 72 Lua/C bindings in C. No need to learn a separate binding language 73 — <b>it parses plain C declarations!</b> These can be 74 cut-n-pasted from C header files or reference manuals. It's up to 75 the task of binding large libraries without the need for dealing with 76 fragile binding generators. 77 78 </p> 79 <p> 80 The FFI library is tightly integrated into LuaJIT (it's not available 81 as a separate module). The code generated by the JIT-compiler for 82 accesses to C data structures from Lua code is on par with the 83 code a C compiler would generate. Calls to C functions can 84 be inlined in JIT-compiled code, unlike calls to functions bound via 85 the classic Lua/C API. 86 </p> 87 <p> 88 This page gives a short introduction to the usage of the FFI library. 89 <em>Please use the FFI sub-topics in the navigation bar to learn more.</em> 90 </p> 91 92 <h2 id="call">Motivating Example: Calling External C Functions</h2> 93 <p> 94 It's really easy to call an external C library function: 95 </p> 96 <pre class="code mark"> 97 <span class="codemark">① 98 ② 99 100 101 ③</span>local ffi = require("ffi") 102 ffi.cdef[[ 103 <span style="color:#00a000;">int printf(const char *fmt, ...);</span> 104 ]] 105 ffi.C.printf("Hello %s!", "world") 106 </pre> 107 <p> 108 So, let's pick that apart: 109 </p> 110 <p> 111 <span class="mark">①</span> Load the FFI library. 112 </p> 113 <p> 114 <span class="mark">②</span> Add a C declaration 115 for the function. The part inside the double-brackets (in green) is 116 just standard C syntax. 117 </p> 118 <p> 119 <span class="mark">③</span> Call the named 120 C function — Yes, it's that simple! 121 </p> 122 <p style="font-size: 8pt;"> 123 Actually, what goes on behind the scenes is far from simple: <span 124 style="color:#4040c0;">③</span> makes use of the standard 125 C library namespace <tt>ffi.C</tt>. Indexing this namespace with 126 a symbol name (<tt>"printf"</tt>) automatically binds it to the 127 standard C library. The result is a special kind of object which, 128 when called, runs the <tt>printf</tt> function. The arguments passed 129 to this function are automatically converted from Lua objects to the 130 corresponding C types. 131 </p> 132 <p> 133 Ok, so maybe the use of <tt>printf()</tt> wasn't such a spectacular 134 example. You could have done that with <tt>io.write()</tt> and 135 <tt>string.format()</tt>, too. But you get the idea ... 136 </p> 137 <p> 138 So here's something to pop up a message box on Windows: 139 </p> 140 <pre class="code"> 141 local ffi = require("ffi") 142 ffi.cdef[[ 143 <span style="color:#00a000;">int MessageBoxA(void *w, const char *txt, const char *cap, int type);</span> 144 ]] 145 ffi.C.MessageBoxA(nil, "Hello world!", "Test", 0) 146 </pre> 147 <p> 148 Bing! Again, that was far too easy, no? 149 </p> 150 <p style="font-size: 8pt;"> 151 Compare this with the effort required to bind that function using the 152 classic Lua/C API: create an extra C file, add a C function 153 that retrieves and checks the argument types passed from Lua and calls 154 the actual C function, add a list of module functions and their 155 names, add a <tt>luaopen_*</tt> function and register all module 156 functions, compile and link it into a shared library (DLL), move it to 157 the proper path, add Lua code that loads the module aaaand ... finally 158 call the binding function. Phew! 159 </p> 160 161 <h2 id="cdata">Motivating Example: Using C Data Structures</h2> 162 <p> 163 The FFI library allows you to create and access C data 164 structures. Of course the main use for this is for interfacing with 165 C functions. But they can be used stand-alone, too. 166 </p> 167 <p> 168 Lua is built upon high-level data types. They are flexible, extensible 169 and dynamic. That's why we all love Lua so much. Alas, this can be 170 inefficient for certain tasks, where you'd really want a low-level 171 data type. E.g. a large array of a fixed structure needs to be 172 implemented with a big table holding lots of tiny tables. This imposes 173 both a substantial memory overhead as well as a performance overhead. 174 </p> 175 <p> 176 Here's a sketch of a library that operates on color images plus a 177 simple benchmark. First, the plain Lua version: 178 </p> 179 <pre class="code"> 180 local floor = math.floor 181 182 local function image_ramp_green(n) 183 local img = {} 184 local f = 255/(n-1) 185 for i=1,n do 186 img[i] = { red = 0, green = floor((i-1)*f), blue = 0, alpha = 255 } 187 end 188 return img 189 end 190 191 local function image_to_grey(img, n) 192 for i=1,n do 193 local y = floor(0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue) 194 img[i].red = y; img[i].green = y; img[i].blue = y 195 end 196 end 197 198 local N = 400*400 199 local img = image_ramp_green(N) 200 for i=1,1000 do 201 image_to_grey(img, N) 202 end 203 </pre> 204 <p> 205 This creates a table with 160.000 pixels, each of which is a table 206 holding four number values in the range of 0-255. First an image with 207 a green ramp is created (1D for simplicity), then the image is 208 converted to greyscale 1000 times. Yes, that's silly, but I was in 209 need of a simple example ... 210 </p> 211 <p> 212 And here's the FFI version. The modified parts have been marked in 213 bold: 214 </p> 215 <pre class="code mark"> 216 <span class="codemark">① 217 218 219 220 221 222 ② 223 224 ③ 225 ④ 226 227 228 229 230 231 232 ③ 233 ⑤</span><b>local ffi = require("ffi") 234 ffi.cdef[[ 235 </b><span style="color:#00a000;">typedef struct { uint8_t red, green, blue, alpha; } rgba_pixel;</span><b> 236 ]]</b> 237 238 local function image_ramp_green(n) 239 <b>local img = ffi.new("rgba_pixel[?]", n)</b> 240 local f = 255/(n-1) 241 for i=<b>0,n-1</b> do 242 <b>img[i].green = i*f</b> 243 <b>img[i].alpha = 255</b> 244 end 245 return img 246 end 247 248 local function image_to_grey(img, n) 249 for i=<b>0,n-1</b> do 250 local y = <b>0.3*img[i].red + 0.59*img[i].green + 0.11*img[i].blue</b> 251 img[i].red = y; img[i].green = y; img[i].blue = y 252 end 253 end 254 255 local N = 400*400 256 local img = image_ramp_green(N) 257 for i=1,1000 do 258 image_to_grey(img, N) 259 end 260 </pre> 261 <p> 262 Ok, so that wasn't too difficult: 263 </p> 264 <p> 265 <span class="mark">①</span> First, load the FFI 266 library and declare the low-level data type. Here we choose a 267 <tt>struct</tt> which holds four byte fields, one for each component 268 of a 4x8 bit RGBA pixel. 269 </p> 270 <p> 271 <span class="mark">②</span> Creating the data 272 structure with <tt>ffi.new()</tt> is straightforward — the 273 <tt>'?'</tt> is a placeholder for the number of elements of a 274 variable-length array. 275 </p> 276 <p> 277 <span class="mark">③</span> C arrays are 278 zero-based, so the indexes have to run from <tt>0</tt> to 279 <tt>n-1</tt>. One might want to allocate one more element instead to 280 simplify converting legacy code. 281 </p> 282 <p> 283 <span class="mark">④</span> Since <tt>ffi.new()</tt> 284 zero-fills the array by default, we only need to set the green and the 285 alpha fields. 286 </p> 287 <p> 288 <span class="mark">⑤</span> The calls to 289 <tt>math.floor()</tt> can be omitted here, because floating-point 290 numbers are already truncated towards zero when converting them to an 291 integer. This happens implicitly when the number is stored in the 292 fields of each pixel. 293 </p> 294 <p> 295 Now let's have a look at the impact of the changes: first, memory 296 consumption for the image is down from 22 Megabytes to 297 640 Kilobytes (400*400*4 bytes). That's a factor of 35x less! So, 298 yes, tables do have a noticeable overhead. BTW: The original program 299 would consume 40 Megabytes in plain Lua (on x64). 300 </p> 301 <p> 302 Next, performance: the pure Lua version runs in 9.57 seconds (52.9 303 seconds with the Lua interpreter) and the FFI version runs in 0.48 304 seconds on my machine (YMMV). That's a factor of 20x faster (110x 305 faster than the Lua interpreter). 306 </p> 307 <p style="font-size: 8pt;"> 308 The avid reader may notice that converting the pure Lua version over 309 to use array indexes for the colors (<tt>[1]</tt> instead of 310 <tt>.red</tt>, <tt>[2]</tt> instead of <tt>.green</tt> etc.) ought to 311 be more compact and faster. This is certainly true (by a factor of 312 ~1.7x). Switching to a struct-of-arrays would help, too. 313 </p> 314 <p style="font-size: 8pt;"> 315 However the resulting code would be less idiomatic and rather 316 error-prone. And it still doesn't get even close to the performance of 317 the FFI version of the code. Also, high-level data structures cannot 318 be easily passed to other C functions, especially I/O functions, 319 without undue conversion penalties. 320 </p> 321 <br class="flush"> 322 </div> 323 <div id="foot"> 324 <hr class="hide"> 325 Copyright © 2005-2016 Mike Pall 326 <span class="noprint"> 327 · 328 <a href="contact.html">Contact</a> 329 </span> 330 </div> 331 </body> 332 </html>