LuaRegister
Github link

Goal

void register_functions(lua_State* lua) {
  LuaRegister::Register(lua, "begin" ImGui::Begin);
  LuaRegister::Register(lua, "end" ImGui::End);
  LuaRegister::Register(lua, "text", +[](const char* str) { return ImGui::Text("%s", str); })
  LuaRegister::Register(lua, "button" ImGui::Button);
  LuaRegister::Register(lua, "drag_float" ImGui::DragFloat2);
}

A function to register C++ functions and stateless lambdas. The function should only require a pointer to the function and should not require repetition of a function's signature.

Let's take a look at ImGui::DragFloat2 in imgui.h:

IMGUI_API bool DragFloat2(const char* label, float v[2], float v_speed = 1.0f, float v_min = 0.0f, float v_max = 0.0f, const char* format = "%.3f", ImGuiSliderFlags flags = 0);

To call this function from lua, we need to be able to:

return a primitive data type - easy, push it onto the stack
pass strings and primitive data types - easy, pop it from the stack
pass an array (pointer) - more difficult, because there is no way to know how big the array is and we don't want to dynamically allocate memory
allow modification of the given array - not at all possible, because a temporary is pushed
pass default parameters - not at all possible, these are "invisible" to the compiler

Bonus goal: perform as much work as possible at compile-time.

Required concepts and language features

Order of execution is very important here for reasons outlined below. The order of execution in C++ has been undefined for a long time but has recently been standardised in more cases.
C++20s requires is utilised, and C++17s if constexpr works really well to simplify template code and to avoid template specialisation.
Also from C++17, std::apply is required to make this approach work at all, and std::make_index_sequence allows us to reach the goal without having recursive.
remove_cvref is used, which is just a nice shortcut. templates.
There is lots of parameter packing templates and expansion.
Function pointers.
The comma operator! :)

Background

Registering a lot of functions with a scripting API is tedious and possibly error-prone. In this case, I specifically wanted to expose Dear ImGui to Lua, which is why that is the example I've used in the repo. One thing that registering ImGui does not showcase is that tables can be passed and returned, in other words, custom types can be passed back and forth to lua as well.

As an example of what registering functions to a scripting language might look like, here is a file from Overgrowth by Wolfire Games: https://github.com/WolfireGames/overgrowth/blob/main/Source/Scripting/angelscript/asfuncs.cpp

One of the worst things I know is when stringly typing code (https://blog.codinghorror.com/new-programming-jargon/) causes runtime crashes! Misspelling a string and then waiting for the code to compile only to have the game crash is such a huge drain of flow and motivation. Even worse when it is undefined behaviour.

Solution

Thanks to parameter packs, the final signature can be rather simple. All we need is a lua state, the function's name in lua, and a function pointer. To keep performance as high as possible, std::function is not used (see this StackOverflow post for some details). The only downside here is that we can't give lua stateful lambdas, but there's not much point in that right now anyway.

Digging into the code, the function to register a function to Lua is only 3 lines long and has a simple signature:

template<typename R, typename... Args>
void Register(lua_State* lua, const char* name, R (*f)(Args...))
{
    lua_pushlightuserdata(lua, (void*)f);
    lua_pushcclosure(lua, LuaWrapper<R, Args...>, 1);
    lua_setglobal(lua, name);
}

There is a minor inconvenience in that lua_pushlightuserdata takes a void*. Casting a function pointer to a void pointer is optionally supported since C++11, but POSIX and WinAPI both require casting void pointers to function pointers when opening shared libraries, and since I have no ambition of running this on anything else that'll work great. One minor note is that the cast is from R(*)(Args...) to void* and then back to R(*)(Args...), so there's no trickery going on.

A lot (most?) of modern computers are based on the Von Neumann architecture, this means that code and memory is available in the same space (RAM), unlike in the Harvard architecture. In other words, this code might not run on CPUs based on the Harvard architecture.

So, the given function pointer isn't actually registered to lua at all, but pushed as an upvalue for the actual function LuaWrapper. One nice upside of this implementation is that functions with the same signature don't generate multiple versions of Register and LuaWrapper.

All code from this point on will be based on what is generated when the following is compiled:

int doSomething(int, float*);

int main()
{
    lua_State* lua;
    LuaRegister::Register(lua, "do_something", doSomething);
    return 0;
}

LuaWrapper

Below is the source code (to the left, without comments) and the generated code (to the right, after some cleanup) of LuaWrapper. The generated code comes from cppinsights.io

template<typename R, typename... Args>
    requires(sizeof...(Args) > 0)
int LuaWrapper(lua_State* lua)
{
    int stackIndex = 1;
    std::tuple ownedArguments{GetParameter<std::remove_cvref_t<Args>>(lua, stackIndex)...};
    std::tuple functionArguments = ReferenceValues<Args...>(ownedArguments);

    int retCount = 0;
    auto f = (R(*)(Args...))lua_touserdata(lua, lua_upvalueindex(1));
    if constexpr(!std::is_same_v<R, void>)
    {
        R res = std::apply(f, functionArguments);
        LuaSetFunc<R>(lua, res);
        retCount++;
    }
    else
    {
        std::apply(f, functionArguments);
    }

    retCount += PushReturnValues<Args...>(lua, ownedArguments);
    return retCount;
}

// template<typename ReturnType, typename... Args>
template<>
int LuaWrapper<int, int, float*>(lua_State* lua)
{
    int stackIndex = 1;
    auto ownedArguments = std::tuple<int, std::array<float, 4>>{
            GetParameter<std::remove_cvref_t<int>>(lua, stackIndex),
            GetParameter<std::remove_cvref_t<float*>>(lua, stackIndex)
    };
    std::tuple<int, float*> functionArguments = ReferenceValues<int, float*>(ownedArguments);

    int retCount = 0;
    auto f = reinterpret_cast<int (*)(int, float*)>(lua_touserdata(lua, lua_upvalueindex(1)));
    // !std::is_same_v<int, void>)
    if constexpr(true)
    {
        int res = std::apply(f, functionArguments);
        LuaSetFunc<int>(lua, static_cast<long long>(res));
        retCount++;
    }

    retCount = retCount + PushReturnValues<int, float*>(lua, ownedArguments);
    return retCount;
}

Thanks to the C++20 requires keyword, we can have separate implementations for functions that return values and functions that don't, which is nice because we don't have to work around empty tuples.

Below is a version of the code with comments to outline what is going on. This will be broken down in the rest of this post.

template<typename R, typename... Args>
// There is an overloaded `LuaWrapper` for when no parameters are required by
// the function that we want to register
    requires(sizeof...(Args) > 0)
int LuaWrapper(lua_State* lua)
{
    /// 1. Get parameters from lua
    // The lua stack index to pull parameters from. This is modified by `GetParameter`
    int stackIndex = 1;
    std::tuple ownedArguments{GetParameter<std::remove_cvref_t<Args>>(lua, stackIndex)...};

    /// 2. Call function and return its value
    std::tuple functionArguments = ReferenceValues<Args...>(ownedArguments);
    // Keep track of the number of return values
    int retCount = 0;
    // Get function that was passed as an upvalue earlier. This is the function
    // that was passed to `Register`
    auto f = (R(*)(Args...))lua_touserdata(lua, lua_upvalueindex(1));
    // If the function return type is not void, make sure to return its value
    if constexpr(!std::is_same_v<R, void>)
    {
        // Call the function
        R res = std::apply(f, functionArguments);
        // Put the return value on the stack
        LuaSetFunc<R>(lua, res);
        retCount++;
    }
    else
    {
        std::apply(f, functionArguments);
    }

    /// 3. Return any additional parameters
    retCount += PushReturnValues<Args...>(lua, ownedArguments);
    return retCount;
}

1. Get parameters from lua

Immediately you're greeted by two obvious variables: ownedArguments, and functionArguments. As usual, the most difficult part of this project was to come up with names for these variables and the concepts they represent. The short version is that GetParameter fetches a parameter from the lua stack and takes ownership of it, therefore the owned parameters are stored in ownedArguments. functionArguments is another tuple which might contain owned values of its own, or pointers to values in functionArguments (i.e. they are not owned by functionArguments). This is important for later.

If an "owned" variable is something new to you, perhaps the concept of ownership is also something new, and you'll be happy to know that you've stumbled upon the opportunity to learn one of the most important parts of programming in C++! Though you'll have to look it up yourself - sorry. This post is long enough as it is. The short version is that the variable is copied and we are allowed to do whatever we want with it.

Note the "braced-init-list" that is used to create ownedArguments; that parameter pack expansion plus is where the magic happens!

//                         V---------------------braced-init-list----------------------V
std::tuple ownedArguments{GetParameter<std::remove_cvref_t<Args>>(lua, stackIndex)...};
//                                                                        Pack expansion ^

If the V and ^ don't match up in your browser, please send me an email!

std::remove_cvref_t is required because int, const int, const int& are all fetched in the same way from lua, and as such they should all be treated as int.

Magic (aka understanding the standard aka googling a bunch)

Well, there's not much magic going on other than the sequence rules introduced in C++11. Having a peek at https://en.cppreference.com/w/cpp/language/eval_order, the very first sentence spells problems:

Order of evaluation of any part of any expression, including order of evaluation of function arguments is unspecified

The problem is that GetParameter needs, for reasons explained soon, to be called in the same order as the parameters of the function we want to register. If we read on, and we know what we're looking for since the wording here isn't very easy to follow, we find the following statement and some text explaining that braced-init-lists are full-expressions:

Each value computation and side effect of a full-expression is sequenced before each value computation and side effect of the next full-expression.

We also find the following text on https://en.cppreference.com/w/cpp/language/list_initialization:

Every initializer clause is sequenced before any initializer clause that follows it in the braced-init-list. This is in contrast with the arguments of a function call expression, which are unsequenced (until C++17) indeterminately sequenced (since C++17).

The short version is: the order of evaluation was (and sometimes still is) undefined, but it is no longer undefined in braced-init-lists.

Why is order important? Well, have a look at the GetParameter call(s). It takes a stackIndex! This stack index is the lua stack index of the parameter that GetParameter should get. Sounds easy enough, but not all parameters actually occupy a single stack index, so the parameter is a reference:

GetParameter(lua_State* lua, int& stackIndex)

In other words, the calls to GetParameter must come in order because the stackIndex parameter is not guaranteed to increment with every call, so it can't simply be a GetParameter(lua, stackIndex + X).

A very important note is that GetParameter does not have to return the same type that we give it! If we call GetParameter<int> we do get an int back, but if we call GetParameter<float*>, we do not get a float back. This is because a float* parameter might just be a single value that we want to modify, or it is an array - we just don't know (yay C!). At the time of writing, GetParameter<float*> actually returns a std::array<float, 4>. ImGui has functions like DragFloat4, but not DragFloat5, so it works fine for now.

2. Call function and return its value

We have fetched all the arguments from lua and we're almost ready to call the function. But first...

As you may know, C and C++ requires us to match parameter types and argument types. As you may know if you read the previous very important note, we have a tuple ownedArguments that does not necessarily match the functions parameter types (remember the float* turning into an std::array). This is where the tuple functionArguments comes in:

std::tuple functionArguments = ReferenceValues<Args...>(ownedArguments);

This tuple's types does match the function we'd like to call. Each member of functionArguments will either point to or contain a copy of the same member in ownedArguments. The implementation is pretty simple and I have added comments for clarity. Note that ReferenceValues is just a nice wrapper around calling ReferenceOneValue for each member of a tuple:

template<typename TargetType, typename T>
auto ReferenceOneValue(T& val)
{
    if constexpr(std::is_same_v<TargetType, lua_State*>)
        // Don't do anything with a lua_State*
        return val;
    else if constexpr(std::is_pointer_v<TargetType> && !std::is_same_v<const char*, TargetType>)
        // Function wants a pointer, so give it a pointer. I just noticed this very unsafe cast...
        // Unless it is a const char*, which is always treated as a string
        return (TargetType)&val;
    else
        // TargetType is not a pointer, so T will just be a value of type TargetType
        return val;
}

// Overload for std::array to return `val.data()` instead of casting the
// `std::array` directly to a pointer type
template<typename TargetType, typename T, std::size_t Size>
auto ReferenceOneValue(std::array<T, Size>& val)
{
    if constexpr(std::is_pointer_v<TargetType> && !std::is_same_v<const char*, TargetType>)
        return val.data();
    else
        // Compile time error if the above condition is not satisfied
        static_assert(always_false<T>);
}

And the function can be called using std::apply. As a reminder, this is the generated code:

// We want to register a function with the signature `int(int, float*)`...
int LuaWrapper<int, int, float*>(lua_State* lua)
{
    ...
    auto ownedArguments = std::tuple<int, std::array<float, 4>>{
            GetParameter<std::remove_cvref_t<int>>(lua, stackIndex),
            GetParameter<std::remove_cvref_t<float*>>(lua, stackIndex)
    };
    // The `float*` points to `&std::get<1>(ownedArguments)`, the int is copied
    std::tuple<int, float*> functionArguments = ReferenceValues<int, float*>(ownedArguments);

    ...
    // `functionArguments` matches the parameters
    auto f = reinterpret_cast<int (*)(int, float*)>(...);
    ...
    {
        // Finally we can invoke the actual function that we registered!
        int res = std::apply(f, functionArguments);
        // And push the function's return value onto the stack
        LuaSetFunc<int>(lua, static_cast<long long>(res));
        retCount++;
    }
    ...
}

3. Return any additional parameters

Now that the function has been invoked and its return value is on the lua stack, we can turn our attention to the arguments that may have been modified by this function - pointers. These out-parameters are heavily utilised by ImGui, so it is imporant to return these to lua as well.

As mentioned in the beginning of this post, we can't (at least I can't, please send in your suggestions :) ) actually modify the lua variables, so they will be pushed to the stack as well. Lua allows multiple return values so this is all par for the course.

Remember, all pointers in the functionArguments tuple points to a member of the ownedArguments tuple. If the function modified any of its pointer arguments, ownedArguments is actually the tuple that is modified, which is why it is passed toPushReturnValues;

retCount += PushReturnValues<Args...>(lua, ownedArguments);

retCount = retCount + PushReturnValues<int, float*>(lua, ownedArguments);

PushReturnValues is, just like ReferenceValues, a nice wrapper around PushOneReturnValue. The interesting detail here is that we want to call PushOneReturnValue for each member of ownedArguments. For that we need a compile-time for-loop or something equivalent.

Compile-time counters with C++14

The only purpose of PushReturnValues is to wrap the call to PushReturnValuesImpl because I think the call to std::make_index_sequence is long and difficult to parse, and I don't want to have it in LuaWrapper. The generated code isn't particularly difficult to understand though. If anything, std::make_integer_sequence and std::index_sequence might take a while to wrap your head around...

Note that FuncTypes and TupleTypes show up here. FuncTypes are the parameter types of the function that we want to register with lua (i.e. ownedArguments), and TupleTypes are the types that are returned by ReferenceValues (i.e. functionArguments).

template<typename... FuncTypes, typename... TupleTypes>
int PushReturnValues(lua_State* luaState, std::tuple<TupleTypes...>& tuple)
{
    int returnValueCount = 0;
    PushReturnValuesImpl<FuncTypes...>(
        luaState,
        tuple,
        std::make_index_sequence<std::tuple_size_v<std::tuple<TupleTypes...>>>{},
        returnValueCount);
    return returnValueCount;
}

template<>
int PushReturnValues<int, float*, int, std::array<float, 4>>(
    lua_State* luaState,
    std::tuple<int, std::array<float, 4>>& tuple)
{
    int returnValueCount = 0;
    PushReturnValuesImpl<int, float*>(
        luaState,
        tuple,
        std::integer_sequence<unsigned long, 0, 1>{},
        returnValueCount);
    return returnValueCount;
}

Same as GetParameter taking a stack index, this function passes a counter to keep track of how many values were actually returned to lua - remember, only pointers need to be returned since only those may have been modified. PushReturnValuesImpl only performs a number of calls to PushOneReturnValue, one call for each index in the given tuple. In the generated code it is (somewhat) clear that each call (potentially) pushes one single value from the tuple:

template<typename... Types, typename... TupleTypes, std::size_t... I>
void PushReturnValuesImpl(
    lua_State* luaState,
    std::tuple<TupleTypes...>& tuple,
    std::index_sequence<I...>,
    int& returnValueCount)
{
    // Note: comma operator is used here! That doesn't happen often!
    (..., PushOneReturnValue<Types>(luaState, std::get<I>(tuple), I + 1, returnValueCount));
}

template<>
void PushReturnValuesImpl<int, float*, int, std::array<float, 4>, 0, 1>(
    lua_State* luaState,
    std::tuple<int, std::array<float, 4>>& tuple,
    std::integer_sequence<unsigned long, 0, 1>,
    int& returnValueCount)
{
    PushOneReturnValue<int>(
        luaState,
        std::get<0UL>(tuple), // Taking value from the tuple
        static_cast<int>(0UL + 1), // Lua stack index
        returnValueCount),
    PushOneReturnValue<float*>(
        luaState,
        std::get<1UL>(tuple), // Taking value from the tuple
        static_cast<int>(1UL + 1), // Lua stack index
        returnValueCount);
}

And then, finally, PushOneReturnValue checks if the given type should be returned (if it is a pointer). The generated code expands to two functions here, since there are two arguments.

template<typename ReturnType, typename T>
void PushOneReturnValue(lua_State* lua, T& val, const int stackIndex, int& returnValueCount)
{
    // Not a pointer or constant, then the value can't be changed by the
    // function
    if constexpr(!std::is_pointer_v<ReturnType> || is_const_pointer_v<ReturnType>)
        return;
    else
    {
        using U = std::remove_pointer_t<std::remove_reference_t<ReturnType>>;
        // Lua decides if an array should be returned
        if(lua_istable(lua, stackIndex))
        {
            int len = luaL_len(lua, stackIndex);
            lua_createtable(lua, len, 0);
            for(int i = 0; i < len; ++i)
            {
                lua_pushnumber(lua, i + 1);
                LuaSetFunc<U>(lua, val.at(i));
                lua_settable(lua, -3);
            }
        }
        else
            LuaSetFunc<U>(lua, val.at(0));

        returnValueCount += 1;
    }
}

// Return type is int, therefore it will not return anything
// template<typename ReturnType, typename T>
template<>
void PushOneReturnValue<int, int>(
    lua_State* lua,
    int& val,
    const int stackIndex,
    int& returnValueCount)
{
    // !std::is_pointer_v<ReturnType> || is_const_pointer_v<ReturnType>
    if constexpr(true)
    {
        return;
    }
}

// Return type is float*, therefore it should return at least 1 float
// template<typename ReturnType, typename ValType>
template<>
void PushOneReturnValue<float*, std::array<float, 4>>(
    lua_State* lua,
    std::array<float, 4>& val,
    const int stackIndex,
    int& returnValueCount)
{
    //(!std::is_pointer_v<float*> || is_const_pointer_v<float*>)
    if constexpr(false) { }
    else
    {
        if(lua_istable(lua, stackIndex))
        {
            int len = luaL_len(lua, stackIndex);
            lua_createtable(lua, len, 0);
            for(int i = 0; i < len; ++i)
            {
                lua_pushnumber(lua, static_cast<double>(i + 1));
                LuaSetFunc<float>(
                    lua,
                    static_cast<double>(val.at(static_cast<unsigned long>(i))));
                lua_settable(lua, -3);
            }
        }
        else
        {
            LuaSetFunc<float>(lua, static_cast<double>(val.at(0)));
        }

        returnValueCount = returnValueCount + 1;
    }
}

And finally, after the return values have been pushed onto the lua stack the function is done.

Limitations

This is just a barely-functional implementation. There is no error-checking. The error checking probably wouldn't introduce anything new, so it probably won't need any more information to understand.

Overall the generated code should be fairly efficient, and it is an absolute breeze to register functions.

Future work

General error checking. It is possible to pass the wrong types from lua.
Array bounds checking. ImGui::DragFloat2 wants a pointer that points to two floats, ImGui::DragFloat3 wants three.
C++ support. In other words, passing std::array instead of naked pointers.
Supporting per-function default parameters.

Anything else?

Yes!

Variadic arguments are supported. To avoid dynamic allocation, a max size has to be set at compile-time.

It is possible to manually parse arguments by registering a function that takes a LuaRegister::Placeholder, which will just contain the stack index of a parameter, letting you manually parse it.

It is possible to return tables by registering a function that returns a LuaRegister::Placeholder. Manually create the table using lua_createtable, fill it with data, and leave it on the top of the stack.

Registering functions with custom structs or classes is possibly by creating specialisations of LuaRegister::LuaSetFunc, LuaRegister::LuaGetFunc, and LuaRegister::GetDefault. Note: Placing them in the namespace is imporant.

LuaRegister Github link