Skip to content

Latest commit

 

History

History
381 lines (290 loc) · 15.1 KB

extensions.md

File metadata and controls

381 lines (290 loc) · 15.1 KB
Raw

Moonjit Extensions and API

Moonjit is fully upwards-compatible with Lua 5.1. It supports all standard Lua library functions and the full set of Lua/C API functions.

Moonjit is also fully ABI-compatible to Lua 5.1 at the linker/dynamic loader level. This means you can compile a C module against the standard Lua headers and load the same shared library from either Lua or moonjit.

Moonjit extends the standard Lua VM with new functionality and adds several extension modules. These extensions are intended to be fully compatible with moonjit in the v2.1.* series of releases and mostly compatible in v2.2.* and beyond.

Extensions Modules

Moonjit comes with several built-in extension modules:

bit.* — Bitwise operations

Moonjit supports all bitwise operations as defined by Lua BitOp:

bit.tobit  bit.tohex  bit.bnot    bit.band bit.bor  bit.bxor
bit.lshift bit.rshift bit.arshift bit.rol  bit.ror  bit.bswap

This module is a moonjit built-in — you don't need to download or install Lua BitOp. The Lua BitOp site has full documentation for all Lua BitOp API functions. The FFI adds support for 64 bit bitwise operations, using the same API functions.

Please make sure to require the module before using any of its functions:

local bit = require("bit")

An already installed Lua BitOp module is ignored by moonjit. This way you can use bit operations from both Lua and moonjit on a shared installation.

ffi.* — FFI library

The FFI library allows calling external C functions and the use of C data structures from pure Lua code.

jit.* — JIT compiler control

The functions in this module control the behavior of the JIT compiler engine.

C API extensions

moonjit adds some extra functions to the Lua/C API.

Enhanced Standard Library Functions

xpcall(f, err [,args...]) passes arguments

Unlike the standard implementation in Lua 5.1, xpcall() passes any arguments after the error function to the function which is called in a protected context.

loadfile() etc. handle UTF-8 source code

Non-ASCII characters are handled transparently by the Lua source code parser. This allows the use of UTF-8 characters in identifiers and strings. A UTF-8 BOM is skipped at the start of the source code.

tostring() etc. canonicalize NaN and ±Inf

All number-to-string conversions consistently convert non-finite numbers to the same strings on all platforms. NaN results in "nan", positive infinity results in "inf" and negative infinity results in "-inf".

tonumber() etc. use builtin string to number conversion

All string-to-number conversions consistently convert integer and floating-point inputs in decimal and hexadecimal on all platforms. strtod() is not used anymore, which avoids numerous problems with poor C library implementations. The builtin conversion function provides full precision according to the IEEE-754 standard, it works independently of the current locale and it supports hex floating-point numbers (e.g. 0x1.5p-3).

string.dump(f [,strip]) generates portable bytecode

An extra argument has been added to string.dump(). If set to true, stripped bytecode without debug information is generated. This speeds up later bytecode loading and reduces memory usage. See also the -b command line option.

The generated bytecode is portable and can be loaded on any architecture that moonjit supports, independent of word size or endianess. However the bytecode compatibility versions must match. Bytecode stays compatible for dot releases (x.y.0 → x.y.1), but may change with major or minor releases (2.0 → 2.1) or between any beta release. Foreign bytecode (e.g. from Lua 5.1) is incompatible and cannot be loaded.

Note: LJ_GC64 mode requires a different frame layout, which implies a different, incompatible bytecode format for ports that use this mode (e.g. ARM64 or MIPS64) or when explicitly enabled for x64. This may be rectified in the future.

table.new(narray, nhash) allocates a pre-sized table

An extra library function table.new() can be made available via require("table.new"). This creates a pre-sized table, just like the C API equivalent lua_createtable(). This is useful for big tables if the final table size is known and automatic table resizing is too expensive.

table.clear(tab) clears a table

An extra library function table.clear() can be made available via require("table.clear"). This clears all keys and values from a table, but preserves the allocated array/hash sizes. This is useful when a table, which is linked from multiple places, needs to be cleared and/or when recycling a table for use by the same context. This avoids managing backlinks, saves an allocation and the overhead of incremental array/hash part growth.

Please note this function is meant for very specific situations. In most cases it's better to replace the (usually single) link with a new table and let the GC do its work.

Enhanced PRNG for math.random()

moonjit uses a Tausworthe PRNG with period 2^223 to implement math.random() and math.randomseed(). The quality of the PRNG results is much superior compared to the standard Lua implementation which uses the platform-specific ANSI rand().

The PRNG generates the same sequences from the same seeds on all platforms and makes use of all bits in the seed argument. math.random() without arguments generates 52 pseudo-random bits for every call. The result is uniformly distributed between 0.0 and 1.0. It's correctly scaled up and rounded for math.random(n [,m]) to preserve uniformity.

io.* functions handle 64 bit file offsets

The file I/O functions in the standard io.* library handle 64 bit file offsets. In particular this means it's possible to open files larger than 2 Gigabytes and to reposition or obtain the current file position for offsets beyond 2 GB (fp:seek() method).

debug.* functions identify metamethods

debug.getinfo() and lua_getinfo() also return information about invoked metamethods. The namewhat field is set to "metamethod" and the name field has the name of the corresponding metamethod (e.g. "__index").

Fully Resumable VM

The moonjit VM is fully resumable. This means you can yield from a coroutine even across contexts, where this would not possible with the standard Lua 5.1 VM: e.g. you can yield across pcall() and xpcall(), across iterators and across metamethods.

Extensions from Lua 5.2

moonjit supports some language and library extensions from Lua 5.2. Features that are unlikely to break existing code are unconditionally enabled:

  • goto and ::labels::.
  • Hex escapes \x3F and \* escape in strings.
  • load(string|reader [, chunkname [,mode [,env]]]).
  • loadstring() is an alias for load().
  • loadfile(filename [,mode [,env]]).
  • math.log(x [,base]).
  • string.rep(s, n [,sep]).
  • string.format(): %q reversible. %s checks __tostring. %a and %A added.
  • String matching pattern %g added.
  • io.read("*L").
  • io.lines() and file:lines() process io.read() options.
  • os.exit(status|true|false [,close]).
  • package.searchpath(name, path [, sep [, rep]]).
  • package.loadlib(name, "*").
  • debug.getinfo() returns nparams and isvararg for option u.
  • debug.getlocal() accepts function instead of level.
  • debug.getlocal() and debug.setlocal() accept negative indexes for varargs.
  • debug.getupvalue() and debug.setupvalue() handle C functions.
  • debug.upvalueid() and debug.upvaluejoin().
  • Lua/C API extensions: lua_version() lua_upvalueid() lua_upvaluejoin() lua_loadx() lua_copy() lua_tonumberx() lua_tointegerx() lua_len() lua_rawlen() lua_absindex() lua_pushglobal() luaL_fileresult() luaL_execresult() luaL_loadfilex() luaL_loadbufferx() luaL_traceback() luaL_setfuncs() luaL_pushmodule() luaL_newlibtable() luaL_newlib() luaL_testudata() luaL_setmetatable() luaL_requiref() luaL_len() luaL_getsubtable() luaL_pushresultsize() luaL_tolstring()
  • Command line option -E.
  • Command line checks __tostring for errors.
  • String matching patterns may contain \0 as a regular character.

Other features are only enabled, if moonjit is built with -DLUAJIT_ENABLE_LUA52COMPAT:

  • goto is a keyword and not a valid variable name anymore.
  • break can be placed anywhere. Empty statements (;;) are allowed.
  • __lt, __le are invoked for mixed types.
  • __len for tables. rawlen() library function.
  • pairs() and ipairs() check for __pairs and __ipairs.
  • coroutine.running() returns two results.
  • table.pack() and table.unpack() (same as unpack()).
  • io.write() and file:write() return file handle instead of true.
  • os.execute() and pipe:close() return detailed exit status.
  • debug.setmetatable() returns object.
  • debug.getuservalue() and debug.setuservalue().
  • Remove math.mod(), string.gfind().
  • package.searchers.
  • module() returns the module table.

Note: this provides only partial compatibility with Lua 5.2 at the language and Lua library level. moonjit is API+ABI-compatible with Lua 5.1, which prevents implementing features that would otherwise break the Lua/C API and ABI (e.g. _ENV).

Extensions from Lua 5.3

Moonjit supports some extensions from Lua 5.3:

  • Unicode escape '\u{XX...}' embeds the UTF-8 encoding in string literals.
  • The argument table arg can be read (and modified) by LUA_INIT and -e chunks.
  • io.read() and file:read() accept formats with or without a leading *.
  • assert() accepts any type of error object.
  • string.pack(fmt, v1, v2, ···), string.packsize(fmt), string.unpack(fmt, s [, pos])
  • table.move(a1, f, e, t [,a2]).
  • coroutine.isyieldable().
  • math.maxinteger, max.mininteger, math.tointeger(x), math.type(x), math.ult(m, n) Note: moonjit uses the same numeric type model as Lua 5.1 which is incompatible with Lua 5.3. As a result, these functions work only in the range [-2^53, 2^53]. math.maxinteger and math.mininteger thus give the limits of this range.
  • utf8.char(...), utf8.charpattern, utf8.codepoints(s [, i [, j]]), utf8.codes(s), utf8.len(s [, i [, j]]), utf8.offset(s, n [, i])
  • Lua/C API extensions: lua_isyieldable(), luaopen_utf8()

Extensions from OpenResty luajit2

The following extensions were incorporated from luajit2, a LuaJIT fork maintained by the OpenResty project:

thread.exdata

syntax: exdata = th_exdata(data?)

This API allows for embedding user data into a thread (lua_State).

The retrieved exdata value on the Lua land is represented as a cdata object of the ctype void*.

As of this version, retrieving the exdata (i.e. th_exdata() without any argument) can be JIT compiled.

Usage:

local th_exdata = require "thread.exdata"

th_exdata(0xdeadbeefLL)  -- set the exdata of the current Lua thread
local exdata = th_exdata()  -- fetch the exdata of the current Lua thread

Also available are the following public C API functions for manipulating exdata on the C land:

void lua_setexdata(lua_State *L, void *exdata);
void *lua_getexdata(lua_State *L);

The exdata pointer is initialized to NULL when the main thread is created. Any child Lua thread will inherit its parent's exdata, but still can override it.

Note: This API will not be available if moonjit is compiled with -DLUAJIT_DISABLE_FFI.

Note bis: This API is used internally by the OpenResty core, and it is strongly discouraged to use it yourself in the context of OpenResty.

jit.prngstate

syntax: state = jit.prngstate(state?)

Returns (and optionally sets) the current PRNG state (a Lua number) currently used by the JIT compiler.

When the state argument is non-nil, it is expected to be a number, and will override the current PRNG state.

Usage:

local state = jit.prngstate()
local newstate = jit.prngstate(123456)

Note: This API has no effect if moonjit is compiled with -DLUAJIT_DISABLE_JIT, and will return 0.

-bl flag for jit.dump

The bytecode option l was updated to display the constant tables of each Lua prototype.

For example, luajit -bl a.lua' now produces bytecode dumps like below:

-- BYTECODE -- a.lua:0-48
KGC    0    "print"
KGC    1    "hi"
KGC    2    table
KGC    3    a.lua:17
KN    1    1000000
KN    2    1.390671161567e-309
...

bytecode option L to display lua source line numbers

The bytecode option L was added to display Lua sources line numbers.

For example, luajit -bL -e 'print(1)' now produces bytecode dumps like below:

-- BYTECODE -- "print(1)":0-1
0001     [1]    GGET     0   0      ; "print"
0002     [1]    KSHORT   1   1
0003     [1]    CALL     0   1   2
0004     [1]    RET0     0   1

The [N] column corresponds to the Lua source line number. For example, [1] means "the first source line".

Trace logging for debugging the JIT compiler

Internal memory-buffer-based trace entry/exit/start-recording event logging, mainly for debugging bugs in the JIT compiler. it requires -DLUA_USE_TRACE_LOGS when building moonjit.

C++ Exception Interoperability

moonjit has built-in support for interoperating with C++ exceptions. The available range of features depends on the target platform and the toolchain used to compile moonjit:

Platform Compiler Interoperability
POSIX/x64, DWARF2 unwinding GCC 4.3+ Full
Other platforms, DWARF2 unwinding GCC Limited
Windows/x64 MSVC or WinSDK Full
Windows/x86 Any No
Other platforms Other compilers No

Full interoperability means:

  • C++ exceptions can be caught on the Lua side with pcall(), lua_pcall() etc.
  • C++ exceptions will be converted to the generic Lua error "C++ exception", unless you use the C call wrapper feature.
  • It's safe to throw C++ exceptions across non-protected Lua frames on the C stack. The contents of the C++ exception object pass through unmodified.
  • Lua errors can be caught on the C++ side with catch(...). The corresponding Lua error message can be retrieved from the Lua stack.
  • Throwing Lua errors across C++ frames is safe. C++ destructors will be called.

Limited interoperability means:

  • C++ exceptions can be caught on the Lua side with pcall(), lua_pcall() etc.
  • C++ exceptions will be converted to the generic Lua error "C++ exception", unless you use the C call wrapper feature.
  • C++ exceptions will be caught by non-protected Lua frames and are rethrown as a generic Lua error. The C++ exception object will be destroyed.
  • Lua errors cannot be caught on the C++ side.
  • Throwing Lua errors across C++ frames will not call C++ destructors.

No interoperability means:

  • It's not safe to throw C++ exceptions across Lua frames.
  • C++ exceptions cannot be caught on the Lua side.
  • Lua errors cannot be caught on the C++ side.
  • Throwing Lua errors across C++ frames will not call C++ destructors.