Zig Meta Programming

Zig Meta Programming

I will skip the intro defining meta-programming (see my previous meta-programming post). Unlike Lisp (and Nim) Zig doesn't explicitly have compile-time code generation. However, zig has something unique, which, while type-based is very different from C++/D's template programming. zig calls this comptime. I would describe it as very aggressive constant folding in the compiler.

If zig can evaluate something at compile time it will be evaluated. The resulting programming style blends the compile time code and the run time code together, in a way which, for me at least, at first hid the true power of zig comptime.

Generating SQL Prepare Statements

This example is not as difficult as my previous meta-programming example. I have made progress on a version of that for zig but so far it's restricted to comptime values i.e. values known completely at compile-time (constants).

Instead the motivating example this time is generating sql prepare statements for a long list of sql. Prepare? Briefly, database api's separate the pre-processing/parsing of sql-language statements and their use later, much like regex libraries allow one to pre-compile regex patterns rather than parsing/interpreting them every time they are matched.

I aim do the following:

• define a variable to hold each prepared (pre-compiled) sql statement.
• call the sql api prepare() function to fill that variable (this will happen in a separate init() function).
• free/deinit the variable at completion (probably program exit).

This can be quite tedious for a long list of sql statements and I usually define a macro to help me. Here is the desired interface.

const statements = .{
// variable-name, sql-statement
.{"add_it", "insert into t1 (x,y,z) values (?,?,?)"},
.{"list_special", "select id,a,b,c from t1 where a>?"},
.{"del_all", "delete from t1"},
.{"del_special", "delete from t1 where a>?"},
...
};

The macro should generate a set of variables and two functions, say prepare_all and finalize_all.

The Zig Macro

Generating top level code doesn't seem to be possible with zig comptime, so instead of generating a set of global variables, I generate a struct-type, with a field where I would have had a global variable. Then I manually instance that struct (a single line which is unchanged when the statement list is changed).

I also use 2 separate function templates for prepare_all and finalize_all. They are not completely generated by the macro. So it's more like 3 macros. I have put the main macro in a separate file to hide it from the user and I could probably move the 2 function templates into that separate file too (with some awkwardness).

So the final code I use (apart from the statement list above) looks like this:

const AllStatements = macros.varcontainer_struct(sl.Stmt, statements);
var st: AllStatements = undefined;

fn prepare_all() !void {
  inline for(statements) |x| {
    @field(st, x[0]) = try sl.prepare(x[1]);
  }
}

fn finalize_all() !void {
  inline for(statements) |x| {
    try sl.finalize(@field(st, x[0]));
  }
}

The inline for above gives a glimpse into comptime possibilities. The sl prefix is for my sqlite3 module and isn't really relevant.

Finally the scary part. This is the code which generates a struct with one field per statement each of type sqlite3.Stmt, those fields which are used above in prepare_all/finalize_all.

pub fn varcontainer_struct(comptime FieldType: type, comptime pairs: anytype) type {
    const ze: std.builtin.Type.StructField = undefined;
    var fields = [_]std.builtin.Type.StructField{ze}**pairs.len;
    inline for(pairs) |x,i| {
      fields[i] = .{ .name = x[0], .type = FieldType, .default_value = null, .is_comptime = false, .alignment = 0 };
    }
    var Decl = std.builtin.Type{
      .Struct = .{
        .layout = .Auto,
        .fields = &fields,
        .decls = &[_]std.builtin.Type.Declaration{},
        .is_tuple = false,
      },
    };
    return @Type(Decl);
}

Yes, indeed... An overview of what happens here may help allay your frights.

• zig treats types as first class values and we can return a type (the struct type) from a comptime function.
• zig functions may also take types as parameters (must be marked comptime). I use the parameter FieldType as the struct field type (all fields have the same type in this example).
• The pairs parameter is the statements list from above (also comptime known). We will create a struct field for each pair in the list, using only the first element, x[0], as the field's name.
• The final aim is to produce a struct type and zig's builtin @Type() takes a description of a type (as a Type struct) and returns a type. This is like reflection in reverse, at compile time.
• So far, so normal. Now we come to inline for. Think of this as a for loop which unrolls itself completely into a list of statements. This is where the code generation comes in. Not everything can be generated like this. For example it would have been easier to directly generate the array constant and assign it to .fields. That's not possible. I must manually create a pre-sized array, then generate a list of assign statements which fill it, and finally use that constant, fields above, to initialize Type.fields slot.
• The interesting, macro part is the inline for. There is a lot of verbiage required to get what we want (a list of named, global variables). However, at least it is possible in zig, and the horror can mostly be hidden from the normal use-site code and, importantly, be re-used.

It was a fascinating, intellectual puzzle, going from my pre-conceived notion of generating a list of global variables and two functions to the final, working, zig implementation, which was quite different but achieves basically what I wanted.

Finally, the generated variables (struct fields) are used as follows:

  if(try sl.query_one(st.list_one, .{id})) |row| {...