If you are a modern programmer of Windows apps, there are numerous frameworks for you, hundreds of SDKs, scripted wrappers, IDEs to hide behind, and just layers upon layers of goodness to keep you safe and sane. So, when it comes to using some of the core Windows APIs directly, you can be forgiven for not even knowing they exist, let alone how to use them from your favorite environment.

I’ve done a ton of exploration on the intricacies of the various Linux interfaces, Spelunking Linux goes over everything from auxv to procfs, and quite a few in between. But, what about Windows? Well, I’ve recently embarked on a new project lj2win32 (not to be confused with earlier LJIT2Win32). The general purpose of this project is to bring the goodness of TINN to the average LuaJIT developer. Whereas TINN is a massive project that strives to cover the entirety of the known world of common Windows interfaces, and provides a ready to go multi-tasking programming environment, lj2win32 is almost the opposite. It does not provide its own shell, rather it just provides the raw bindings necessary for the developer to create whatever they want. It’s intended to be a simple luarocks install, much in the way the ljsyscall works for creating a standard binding to UNIX kinds of systems without much fuss or intrusion.

In creating this project, I’ve tried to adhere to a couple of design principles to meet some objectives.

First objective is that it must ultimately be installable using luarocks. This means that I have to be conscious about the organization of the file structure. To wit, everything of consequence lives in a ‘win32’ directory. The package name might ultimately be ‘win32’. Everything is referenced from there.

Second objective, provide the barest minimum bindings. Don’t change names of things, don’t introduce non-windows semantics, don’t create several layers of class hierarchies to objectify the interfaces. Now, of course there are some very simple exceptions, but they should be fairly limited. The idea being, anyone should be able to take this as a bare minimum, and add their own layers atop it. It’s hard to resist objectifying these interfaces though, and everything from Microsoft’s ancient MFC, ATL, and every framework since, has thrown layers of object wrappers on the core Win32 interfaces. In this case, wrappers and other suggestions will show up in the ‘tests’ directory. That is fertile ground for all manner of fantastical object wrapperage.

Third objective, keep the dependencies minimal. If you do programming in C on Windows, you include a couple of well known headers files at the beginning of your program, and the whole world gets dragged in. Everything is pretty much in a global namespace, which can lead to some bad conflicts, but they’ve been worked out over time. In lj2win32, there are only a couple things in the global namespace, everything else is either in some table, or within the ffi.C facility. Additionally, the wrappings are clustered in a way that follows the Windows API Sets. API sets are a mechanism Windows has for pulling apart interdependencies in the various libraries that make up the OS. In short, it’s just a name (so happens to end in ‘.dll’) which is used by the loader to load in various functions. If you use these special names, instead of the traditional ‘kernel32’, ‘advapi32’, you might pull in a smaller set of stuff.

With all that, I thought I’d explore one particular bit of minutia as an example of how things could go.

The GetSystemMetrics() function call is a sort of dumping ground for a lot of UI system information. Here’s where you can find things like how big the screen is, how many monitors there are, how many pixels are used for the menu bars, and the like. Of course this is just a wrapper on items that probably come from the registry, or various devices and tidbits hidden away in other databases throughout the system, but it’s the convenient developer friendly interface.

The signature looks like this

int WINAPI GetSystemMetrics(
_In_ int nIndex
);

A simple enough call. And a simple enough binding:

ffi.cdef[[
int GetSystemMetrics(int nIndex);
]]

Of course, there is the ‘nIndex’, which in the Windows headers is a bunch of manifest constants, which in LuaJIT might be defined thus:

So, this meets the second objective of bare minimum binding. But, it’s not a very satisfying programming experience for the LuaJIT developer. How about just a little bit of sugar? Well, I don’t want to violate the same second objective of non-wrapperness, so I’ll create a separate thing in the tests directory. The systemmetrics.lua file contains a bit of an exploration in getting of system metrics.

All of this allows you to do a couple of interesting things. First, what if you wanted to print out all the system metrics. This same technique can be used to put all the metrics into a table to be used within your program.

OK, so what? Well, the systemmetrics.names is a dictionary matching a symbolic name to the value used to get a particular metric. And what’s this magic with the ‘sysmetrics[key]’ thing? Well, let’s take a look back at that hand waving from the systemmetrics.lua file.

So, what’s happening here with the setmetatable thing is, Lua has a way of setting some functions on a table which will dictate the behavior they will exhibit in certain situations. In this case, the ‘__index’ function, if it exists, will take care of the cases when you try to look something up, and it isn’t directly in the table. So, in our example, doing the ‘sysmetrics[key]’ thing is essentially saying, “Try to find a value with the string associated with ‘key’. If it’s not found, then do whatever is associated with the ‘__index’ value”. In this case, ‘__index’ is a function, so that function is called, and whatever that returns becomes the value associated with that key.

I know, it’s a mouth full, and metatables are one of the more challenging aspects of Lua to get your head around, but once you do, it’s a powerful concept.

How about another example which will be a more realistic and typical case.

local function testSome()
print(sysmetrics.SM_MAXIMUMTOUCHES)
end

In this case, the exact same mechanism is at play. In Lua, there are two ways to get a value out of a table. The first one we’ve already seen, where the ‘[]’ notation is used, as if the thing were an array. In the ‘testSome()’ case, the ‘.’ notation is being utilized. This is accessing the table as if it were a data structure, but it’s exactly the same as trying to access as an array, at least as far as the invocation of the ‘__index’ function is concerned. The ‘SM_MAXIMUMTOUCHES’ is taken as a string value, so it’s the same as doing: sysmetrics[‘SM_MAXIMUMTOUCHES’], and from the previous example, we know how that works out.

Now, there’s one more thing to note from this little escapade. The implementation of the helper function:

local function getSystemMetrics(what)
local entry = nil;
local idx = nil;
if type(what) == "string" then
entry = exports.names[what]
idx = entry.value;
else
idx = tonumber(what)
if not idx then
return nil;
end
entry = lookupByNumber(idx)
if not entry then return nil end
end
local value = ffi.C.GetSystemMetrics(idx)
if entry.converter then
value = entry.converter(value);
end
return value;
end

There’s all manner of nonsense in here. The ‘what’ can be either a string or something that can be converted to a number. This is useful because it allows you to pass in symbolic names like “SM_CXBLAHBLAHBLAH” or a number 123. That’s great depending on what you’re interacting with and how the values are held. You might have some UI for example where you just want to use the symbolic names and not deal with numbers.

The other thing of note is that ‘entry.converter’ bit at the end. If you look back at the names table, you’ll notice that some of the entries have a ‘converter’ field associated with them. this is an optional function that can be associated with the entries. If it exists, it is called, with the value from the system called passed to it. In most cases, what the system returns is a number (number of mouse buttons, size of screen, etc). In some cases, the value returned is ‘0’ for false, and ‘non-zero’ for true. Well, as a Lua developer, I’d rather just get a bool in those cases where it’s appropriate, and this helper function is in a position to provide that for me. This is great because it allows me to not have to check the documentation to figure it out.

What does this do exactly? Simply, it generates those constants in the ffi.C space, so that you can still do this:

ffi.C.GetSystemMetrics(ffi.C.SM_MAXIMUMTOUCHES)

So, there you have it. You can go with the raw traditional sort of ffi binding, or you can spice things up a bit and make things a bit more useful with a little bit of effort. I like doing the latter, because I can generate the more traditional binding from the table of names that I’ve created. That’s a useful thing for documentation purposes, and in general.

I have stuck to my objectives, and this little example just goes to prove how esoteric minute details can be turned into approachable things of beauty with a little bit of Lua code.

While spelunking Linux, trying to find an easier way to do this or that, I ran across this vDSO thing (virtual ELF Dynamic Shared Object). What?

Ok, it’s like this. I was implementing direct calling of syscalls on Linux, and reading up on how the C libraries do things. On Linux, when you wan to talk to the kernel, you typically go through syscalls, or ioctl calls, or netlinks. With syscall, it’s actually a fairly expensive process, switching from userspace to kernel space, issuing and responding to interrupts, etc. In some situations this could be a critical performance hit. So, to make things easier/faster, some of the system calls are implemented in this little ELF package (vDSO). This little elf package (a dynamic link library) is loaded into every application on Linux. Then, the C library can decide to make calls into that library, instead of syscalls, thus saving a lot of overhead and speeding things up. Not all systems have this capability, but many do.

Alright, so how does the C runtime know whether the capability is there or now, and where this little library is, and how to get at functions therewith? In steps our friend auxv. In the GNU C library, there is a lone call:

unsigned long getauxval(unsigned long);

What values can you get out of this thing? Well, the constants can be found in the elf.h file, and look like:

#define AT_PLATFORM 15
#define AT_PAGESZ 6

And about 30 others. How you use each of these depends on the type of the value that you are looking up. For instance, the AT_PLATFORM returns a pointer to a null terminated string. The AT_PAGESZ returns an integer which represents the memory page size of the machine you’re running on.

And so on and so forth, assuming you have the proper ‘libc’ luajit ffi binding, which gives you access to constants through the ‘libc.’ mechanism.

OK, fine, if I’m a C programmer, and I just want to port some code that’s already doing this sort of thing. By the way, the one value that we’re interested in is: AT_SYSINFO_EHDR. That contains a pointer to the beginning of our vDSO. Then you can call functions directly from there (there’s an API for that).

But, if I’m a lua programmer, I’ve come to expect more out of my environment, largely because I’m lazy and don’t like so much typing. Upon further examination, you can get this information yourself. If you’re hard core, you can look at the top of memory in your program, and map that location to a pointer you can fiddle with directly. Otherwise, you can actually get this information from a ‘file’ on Linux.

Turns out that you can get this info from ‘/proc/self/auxv’, if you’re running this command about your current process (which you most likely are). So, now what can I do with that? Well, the lua way would be the following:

The printAll() function uses the auxv iteration function, which in turns reads the key/value pairs directly from the /proc/self/auxv file. No need for the GNU C lib function at all. It goes further and turns the raw ‘unsigned long’ values into the appropriate data type based on what kind of data the key specified represents. So, you get lua string, and not just a pointer to a C string.

In the second example, getting singles, the output is simply this:

Platform: x86_64
Page Size: 4096

The code for this goes into a bit of trickery that’s possible with luajit. first of all, notice the use of ‘auxv_util.AT_PAGESZ’. There is nothing in the auxv_iter.lua file that supports this value directly. There is the table of names, and then there’s that ‘setmetatable’ at the end of things. Here’s where the trickery happens. Basically, this function is called whenever you put a ‘.’ after a table to try and access something, and that something isn’t in the table. You get a chance to make something up and return it. Ini this case, we first call ‘gencdefs()’ if that hasn’t already been called. This will generate ‘static const int XXX’ for all the values in the table of names, so that we can then do a lookup of the values in the ‘ffi.C.’ namespace, using the name. If we find a name, then we add it to the table, so next time the lookup will succeed, and we won’t end up calling the __index function.

At any rate, we now have the requisite value to lookup. Then we just roll through the iterator, and return when we’ve got the value we were looking for. The conversion to the appropriate lua type is automatic.

And there you have it! From relative obscurity, to complete usability, in one iterator. Being able to actually get function pointers in the vDSO is the next step. That will require another API wrapper, or worst case, and all encompassing ELF parser…

I have need to connect to some Azure services from Linux, using C++. There are a few C/C++ libraries around that will make life relatively easy in this regard, but I thought I’d go with an option that has me learn about something I don’t use that often, but will be fairly powerful and complete. I chose to use cURL, or libcurl.so to be more precise. Why? cURL has been around for ages, has continued to evolve, and makes it fairly easy to do anything from ftp down/upload to https connections. It has tons of options and knobs, including dealing with authentication, SSL, or just acting like an ordinary socket of you prefer that.

First I created a luajit binding to libcurl. This just follows my usual learning pattern. In order to conquer an API, you must first render it useful from Lua. I did my typical two part binding, first a fairly faithful low level binding, then added some luajit idioms atop. In this particular case, there’s not a big database to query, although there are quite a few options that get listed in a table:

This is an excerpt from the original curl.h header file. There is a listing of some 200 options which can be set on a curl connection (depending on the context of the connection). This CINIT is a macro that sets the value of an enum appropriately. Well, those macros don’t work in the luajit ffi.cdef call, so I needed a way to convert these. I could have just run the lot through the gcc pre-processor, and that would give me the values I needed, but I thought I’d take a different approach.

I wrote a bit of script to scan the curl.h file looking for the CINIT lines, and turn them into something interesting.

It’s actually easier than this. I threw in the loadstring just to ensure the output was valid. Yah, ok basics.

libcurl is an extremely convenient library. As such, it can be a challenge to use. Fortunately, it has an ‘easy’ interface as well. Here, I chose to wrap the easy interface in an object like wrapper to make it even easier. Here’s how you use it:

require("CRLEasyRequest")(url):perform();

That will retrieve the entirety of any given url, and dump the output to stdout. Well, that’s somewhat useful, and its only one line of code. This simple interface will become more sophisticated over time, including being the basis for REST calls, but for now, it suffices.

So, there you have it. libcurl seems to be a viable choice for web access from within lua. You could of corse just do it all with pure lua code, but, you probably wont be wrong to leverage libcurl.

Honestly, I don’t know what all the fuss is about. What is systemd? It’s that bit of code that gets things going on your Linux machine once the kernel has loaded itself. You know, dealing with bringing up services, communicating between services, running the udev and dbus stuff, etc.

--[[
Test using SDJournal as a cursor over the journal entries
In this case, we want to try the various cursor positioning
operations to ensure the work correctly.
--]]
package.path = package.path..";../src/?.lua"
local SDJournal = require("SDJournal")
local sysd = require("systemd_ffi")
local jnl = SDJournal()
-- move forward a few times
for i=1,10 do
jnl:next();
end
-- get the cursor label for this position
local label10 = jnl:positionLabel()
print("Label 10: ", label10)
-- seek to the beginning, print that label
jnl:seekHead();
jnl:next();
local label1 = jnl:positionLabel();
print("Label 1: ", label1);
-- seek to label 10 again
jnl:seekLabel(label10)
jnl:next();
local label3 = jnl:positionLabel();
print("Label 3: ", label3)
print("label 3 == label 10: ", label3 == label10)

In this case, a simple journal object which makes it relatively easy to browse through the systemd journals that are laying about. That’s handy. Combined with the luafun functions, browsing through journals suddenly becomes a lot easier, with the full power of lua to form very interesting queries, or other operations.

--[[
Test cursoring over journal, turning each entry
into a lua table to be used with luafun filters and whatnot
--]]
package.path = package.path..";../src/?.lua"
local SDJournal = require("SDJournal")
local sysd = require("systemd_ffi")
local fun = require("fun")()
-- Feed this routine a table with the names of the fields
-- you are interested in seeing in the final output table
local function selection(fields, aliases)
return function(entry)
local res = {}
for _, k in ipairs(fields) do
if entry[k] then
res[k] = entry[k];
end
end
return res;
end
end
local function printTable(entry)
print(entry)
each(print, entry)
end
local function convertCursorToTable(cursor)
return cursor:currentValue();
end
local function printJournalFields(selector, flags)
flags = flags or 0
local jnl1 = SDJournal();
if selector then
each(printTable, map(selector, map(convertCursorToTable, jnl1:entries())))
else
each(printTable, map(convertCursorToTable, jnl1:entries()))
end
end
-- print all fields, but filter the kind of journal being looked at
--printJournalFields(nil, sysd.SD_JOURNAL_CURRENT_USER)
--printJournalFields(nil, sysd.SD_JOURNAL_SYSTEM)
-- printing specific fields
--printJournalFields(selection({"_HOSTNAME", "SYSLOG_FACILITY"}));
printJournalFields(selection({"_EXE", "_CMDLINE"}));
-- to print all the fields available per entry
--printJournalFields();

In this case, we have a simple journal printer, which will take a list of fields, as well as a selection of the kinds of journals to look at. That’s quite useful as you can easily generate JSON or XML, or Lua tables on the output end, without much work. You can easily select which fields you want to display, and you could even change the names along the way. You have the full power of lua at your disposal to do whatever you want with the data.

In this case, the SDJournal object is pretty straight forward. It simply wraps the various ‘sd_xxx’ calls within the library to get its work done. What about some other cases? Does the systemd library need to be used for absolutely everything that it does? The answer is ‘no’, you can do a lot of the work yourself, because at the end of the day, the passive part of systemd is just a bunch of file system manipulation.

Here’s where it gets interesting in terms of decomposition.

Within the libsystemd library, there is the sd_get_machine_names() function:

Great, for those who already know this call, you can allocate a char * array, get the array of string values, and party on. But what about the lifetime of those strings, and if you’re doing it as an iterator, when do you ever free stuff, and isn’t this all wasteful?

So, looking at that code in the library, you might think, ‘wait a minute, I could just replicate that in Lua, and get it done without doing any ffi stuff at all!

OK, that looks simple. But what’s happening with that ‘files_in_directory()’ function? Well, that’s the meat and potatoes of this operation.

local function nil_iter()
return nil;
end
-- This is a 'generator' which will continue
-- the iteration over files
local function gen_files(dir, state)
local de = nil
while true do
de = libc.readdir(dir)
-- if we've run out of entries, then return nil
if de == nil then return nil end
-- check the entry to see if it's an actual file, and not
-- a directory or link
if dirutil.dirent_is_file(de) then
break;
end
end
local name = ffi.string(de.d_name);
return de, name
end
local function files_in_directory(path)
local dir = libc.opendir(path)
if not dir==nil then return nil_iter, nil, nil; end
-- make sure to do the finalizer
-- for garbage collection
ffi.gc(dir, libc.closedir);
return gen_files, dir, initial;
end

In this case, files_in_directory() takes a string path, like “/run/systemd/machines”, and just iterates over the directory, returning only the files found there. It’s convenient in that it will skip so called ‘hidden’ files, and things that are links. This simple technique/function can be the cornerstone of a lot of things that view files in Linux. The function leverages the libc opendir(), readdir(), functions, so there’s nothing new here, but it wraps it all up in this convenient iterator, which is nice.

systemd is about a whole lot more than just browsing directories, but that’s certain a big part of it. When you break it down like this, you begin to realize that you don’t actually need to use a ton of stuff from the library. In fact, it’s probably better and less resource intensive to just ‘go direct’ where it makes sense. In this case, it was just implementing a few choice routines to make file iteration work the same as it does in systemd. As this binding evolves, I’m sure there is other low lying fruit that I’ll be able to pluck to make it even more interesting, useful, and independent of the libsystemd library.

Like this:

It seems I’ve been writing binders, wrappers, frameworks, and the like for the entirety of my 35 years of programming. Why is that? Well, because the tools I use to program are often times not the same as those used to create various libraries, frameworks, and the like. Sometimes it’s just that there is no universal bond between all those disparate libraries that you want to use, so you end up writing some wrappers, to make things look and behave similar.

I’ve written quite a lot about LuaJIT bindings, and produced rather a lot of them myself. Most recently, I was writing a binding for the D-Bus library on Linux. D-Bus is a communications protocol and mechanism meant to support Interprocess Communications. As such, it’s about low latency, speed on read and write, and relative simplicity. You can do everything from pop-up and alert, to logging commands in a persistent log.

Here I want to show an approach that I’ve slowly grown and evolved over the past few years. This approach to writing bindings is written with a few constraints and aspirations in mind:

Provide a low level interface that is as true to the underlying framework as possible.

Porting typical C code to the wrapping should be straight forward and obvious

Provide an interface that supports and leverages Lua idioms and practices

Provide a wrapper that does not require a separate compilation step

That list of constraints is brief, but can cause enough trouble depending on how seriously you take each one of them. So, let’s get to it.

Here I will use the LJIT2dbus project, because it gives a chance to exhibit all of the constraints listed above. Here’s a bit of code:

Yes, again, more require() statements. But then, there are those couple of C functions that are defined. The other files have similar stuff in them. This is the basis of the constraint related to making the code familiar to a C programmer.

Let’s look at another file which might turn out to be more illustrative:

local dbus = require("dbus")
local err = dbus.DBusError();
dbus.dbus_error_init(err);
local bus = dbus.dbus_bus_get(dbus.DBUS_BUS_SESSION, err);
if (dbus.dbus_bus_name_has_owner(bus, SYSNOTE_NAME, err) == 0) then
io.stderr:write("Name has no owner on the bus!\n");
return EXIT_FAILURE;
end

First case is the creation of the ‘local err’ value.

local err = dbus.DBusError();
dbus.dbus_error_init(err);

The dbus.lua file does not have a function called ‘DBusError()’. All it has done is load a bunch of type and function declarations, to be used by the LuaJIT ffi mechanism. So, how do we get a function of that name? It doesn’t even exist in one of the required modules.

The trick here is in the ‘__index’ function of the dbus.lua table. The way the Lua language works, any time you make what looks like an access to the member of a table, if it can’t be found in the table, and if the __index function is implemented, it will get called, with the key passed in as a parameter.

In this case, the ‘__index’ function implements a series of lookups, trying to find the value that is associated with the specified key. First it tries looking in a table of constants. Then it tries looking in the actual library, for a function with the specified name. If it doesn’t find the value as a function, it will try to find it as a constant in the C namespace. This will find any enum, or static const int values that have been defined in ffi.cdef[[]] blocks. Finally, if it doesn’t find the key as one of the constants, it tries to figure out if maybe it’s a type (‘ffi.typeof’). This is in the case of DBusError, this one will succeed, and we’ll get back a type.

As it turns out, the types returned from the LuaJIT ffi can be used as constructors.
So, what we really have is this:

local errType = dbus.DBusError;
local err = errType();

The fact that you can just do it all on one line is a short hand.

This is a great convenience. Also, once the value is found, it is stuck into the dbus table itself, so that the next time it’s used, this long lookup won’t occur. The type is already known, and will just be retrieved from the dbus (exports) table directly.

Well, this works for the other kinds of types as well. For example, the ‘dbus_error_init()’ function is defined in a ffi.cdef[[]] block, and that’s about all. So, when re reference it through dbus.dbus_error_init(), we’re going to look it up in the library, and try to use the function found there. And again, once it’s found, it will be stuffed into the dbus (exports) table, for future reference.

This works out great, in that it’s fairly minimal amount of work to get all interesting things defined in ffi.cdef blocks, then just use this __index lookup trick to actually return the values.

I’ve come to this style because otherwise you end up doing a lot more work trying to make the constants, types, enums, and functions universally accessible from anywhere within your program, without resorting to global values. Of course you can make the __index lookup as complex as you like. You can lazily load in modules, for example.

That’s it for this round. An all in one interface that gives you access to constants, enums, and functions in a ffi.cdef wrapped library.

In this article: Isn’t the whole system just a database? – libdrm, I explored a little bit of the database nature of Linux by using libudev to enumerate and open libdrm devices. After that, I spent some time bringing up a USB module: LJIT2libusb. libusb is a useful cross platform library that makes it relatively easy to gain access to the usb functions on multiple platforms. It can enumerate devices, deal with hot plug notifications, open up, read, write, etc.

At its core, on Linux at least, libusb tries to leverage the uvdev capabilities of the target system, if those capabilities are there. This means that device enumeration and hot plugging actually use the libuvdev stuff. In fact, the code for enumerating those usb devices in libusb looks like this:

There’s more stuff of course, to turn that into data structures which are appropriate for use within the libusb view of the world. But, here’s the equivalent using LLUI and the previously developed UVDev stuff:

local function isUsbDevice(dev)
if dev.IsInitialized and dev:getProperty("subsystem") == "usb" and
dev:getProperty("devtype") == "usb_device" then
return true;
end
return false;
end
each(print, filter(isUsbDevice, ctxt:devices()))

It’s just illustrative, but it’s fairly simple to understand I think. The ‘ctxt:devices()’ is an iterator over all the devices in the system. The ‘filter’ function is part of the luafun functional programming routines available to Lua. the ‘isUsbDevice’ is a predicate function, which returns ‘true’ when the device in question matches what it believes makes a device a ‘usb’ device. In this case, its the subsystem and dev_type properties which are used.

Being able to easily query devices like this makes life a heck of a lot easier. No funky code polluting my pure application. Just these simple query predicates written in Lua, and I’m all set. So, instead of relying on libusb to enumerate my usb devices, I can just enumerate them directly using uvdev, which is what the library does anyway. Enumeration and hotplug handing is part of the library. The other part is the actual send and receiving of data. For that, the libusb library is still primarily important, as replacing that code will take some time.

Where else can this great query capability be applied? Well, libudev is just a nice wrapper atop sysfs, which is that virtual file system built into Linux for gaining access to device information and control of the same. There’s all sorts of stuff in there. So, let’s say you want to list all the block devices?

local function isBlockDevice(dev)
if dev.IsInitialized and dev:getProperty("subsystem") == "block" then
return true;
end
return false;
end

That will get all the devices which are in the subsystem “block”. That includes physical disks, virtual disks, partitions, and the like. If you’re after just the physical ones, then you might use something like this:

local function isPhysicalBlockDevice(dev)
if dev.IsInitialized and dev:getProperty("subsystem") == "block" and
dev:getProperty("devtype") == "disk" and
dev:getProperty("ID_BUS") ~= nil then
return true;
end
return false;
end

Here, a physical device is indicated by subsystem == ‘block’ and devtype == ‘disk’ and the ‘ID_BUS’ property exists, assuming any physical disk would show up on one of the system’s buses. This won’t catch a SD card though. For that, you’d use the first one, and then look for a property related to being an SD card. Same goes for ‘cd’ vs ramdisk, or whatever. You can make these queries as complex or simple as you want.

Once you have a device, you can simply open it using the “SysName” parameter, handed to an fopen() call.

I find this to be a great way to program. It makes the creation of utilities such as ‘lsblk’ relatively easy. You would just look for all the block devices and their partitions, and put them into a table. Then separately, you would have a display routine, which would consume the table and generate whatever output you want. I find this much better than the typical Linux tools which try to do advanced display using the terminal window. That’s great as far as it goes, but not so great if what you really want is a nice html page generated for some remote viewing.

At any rate, this whole libudev exploration is a great thing. You can list all devices easily, getting every bit of information you care to examine. Since it’s all scriptable, it’s fairly easy to taylor your queries on the fly, looking at, discovering, and the like. I discovered that the thumb print reader in my old laptop was made by Broadcom, and my webcam by 3M? It’s just so much fun.

Well there you have it. The more you spelunk, the more you know, and the more you can fiddle about.

I’ve been trawling through the Linux V4L2 group of libraries of late as part of LLUI. v4l2 is one of those sprawling libraries that does all things for all people in terms of video on Linux machines. It’s roughly equivalent to oh so many similar things from the past on the Windows side. This is one of the libraries you might utilize if you were to get into streaming from your webcam programmatically. Of course, you could just read from it directly with libusb, but then you lose out on all the nifty format conversions, and I miss this chance to write another pointless reminder for my later coding self.

So, what’s got me so bothered this time around? Well, lets say I’m just parusing my system, turning everything into a database as I go along. I’d like to get a hold of my webcam, and see what it’s capable of. There’s a call for that of course. Once you make the appropriate IOCtl call, you end up with a struct that looks like this:

The driver, card, and bus_info fields are pretty straight forward as they are simple ‘null terminated’ strings, so you have print them out if you like. It’s that ‘capabilities’ field that gives me fits. This is one of those combined bit flags sort of things. The value can be a combination of any of the numerous ‘capability’ flags, which are these:

For the embedded webcam in my laptop, the reported value is: 0x04000001;

Of course, when you’re doing something programmatically, and you just want to check whether a particular flag is set or not, you can just do:

canStream = band(V4L2_CAP_STREAMING, 0x04000001) ~= 0

Very common, and probably some of the most common code you’ll see anywhere. But what else? For various reasons, I want to create the string values for those bit fields, and use those values as keys to tables, or just to print, or to send somewhere, or display, or what have you.

I’ve seen enough ‘C’ code deal with this there is a common patter. First create the #define, or enum statement which encapsulates the values for all the flags. Then, to get the values as strings, create a completely separate string table, which does the mapping of the nice tight enum values and the string values. Then write a little lookup function which can go from the value to the string.

Well, here’s one of those things I love about Lua. In this case, the program IS the database. No need for those parallel representations. Here’s some code:

The function “getValueName()” at the top there simply does a reverse lookup in the table. That is, given a value, return the string that represents that value (is the string key for that value).

Next, the “enumbits()” function is an enumerator. It will iterate over the bit flags, returning a string name for all the ones that are set to ‘1’, and nothing for any of the other bits. Here’s an example:

Well that’s handy, particularly when you’re doing some debugging. Just a simple 20 line iterator, and you’re in business, printing flag fields like a boss! That is, if you’re in the lua environment, or any dynamic programming environment that supports iteration of a dictionary.

So, this note to future self is about pointing out the fact that even bitflags are nothing than a very compact form of database. Unpacking them into human readable, programmable form, requires just the right routine, and away you go, you never have to bother with dealing with this little item again. Great for debugging, great for sticking keys in tables, great for displaying on controls!