Wednesday, October 21, 2009
One Moon Daemon, coming up
My latest adventure into Lua has been to embed the language into a C application (an outgrowth of extending in C). I had this “proof-of-concept” network daemon I wrote that was simple enough to gut completely and replace the entire server logic with Lua, leaving the C code to handle the sockets.
But that wasn't hard enough. No. I wanted each connection handled by a Lua thread, and the ability to write “normal looking code” such as:
-- A simple echo server in Lua
function main(connection)
while true do
connection:write(connection:read("*a"))
end
end
I will say this, unlike Theo Schlossnagle, who doesn't care for Lua the language but likes embedding it, I came to the opposite conclusion: I like Lua the language but embedding it is … interesting. Perhaps because I dove headfirst into Lua coroutines to get this working.
The first oddness with embedding Lua is that you push parameters onto the Lua stack, but not all lua functions pop all parameters. Namely, anything using a table won't actually pop the table parameter off the stack, and the stack notation used in the manual is not the notation I'm used to (having come to stack notations in Forth). So I'm sure that a majority of the issues I had were due to improper stack manipulations on my part.
And about that stack. You can reference the stack in two ways, from the bottom using positive indices, or from the top using negative indices, with 0 as an invalid stack index.
| Bottom of Stack | Top of Stack | |||
|---|---|---|---|---|
| 1 -5 | 2 -4 | 3 -3 | 4 -2 | 5 -1 |
As a longtime C (and assembly) programmer, it just looks weird to use 1- based indices in C.
Another horrible issue—via the C API you can push integers onto the Lua stack, you can push
strings onto the Lua stack, you can push functions (either written in Lua or
C) onto the stack, heck, you can push everything but userdata
structures onto the Lua stack. Userdata structures are allocated by Lua
(via lua_newuserd
ata()), can contain anything you want, but there's no way to
actually push such a structure onto the Lua stack, so you always
have to keep it around on the Lua stack, or in a Lua variable or table, which
is a bit of a pain. I suppose it has to be that way in order for Lua to keep
track of it for garbage collection purposes, but still, a pain.
And then there's the coroutines …
Easy enough to create a coroutine:
lua_State *new; Foo *userdata; int rc; /*---------------------------- ; create a new thread ; main(userdata) ;----------------------------*/ new = lua_newthread(g_L); lua_getglobal(new,"main"); userdata = lua_newuserdata(new,sizeof(Foo)); /* fill in our userdata */ rc = lua_resume(new,1);
But once the thread finishes, then what?
lua_closevoid lua_close (lua_State *L);Destroys all objects in the given Lua state (calling the corresponding garbage-collection metamethods, if any) and frees all dynamic memory used by this state. On several platforms, you may not need to call this function, because all resources are naturally released when the host program ends. On the other hand, long-running programs, such as a daemon or a web server, might need to release states as soon as they are not needed, to avoid growing too large.
Okay, so I'm concerned about garbage collection—I don't want tons of
garbage piling up and then … the … … big … … … pause … … … … while … … … … …
garbage … … … … … … collection takes place. And seeing how
lua_close() mentions daemons, I figure I can call
lua_close when a thread terminates, since I'm writing a daemon.
But when I do, every open connection suddenly closes and any new connection
causes the daemon to crash.
Huh?
lua_newthreadlua_State *lua_newthread (lua_State *L);Creates a new thread, pushes it on the stack, and returns a pointer to a
lua_Statethat represents this new thread. The new state returned by this function shares with the original state all global objects (such as tables), but has an independent execution stack.There is no explicit function to close or to destroy a thread. Threads are subject to garbage collection, like any Lua object.
Reading between the lines, you don't really get a new state, you get a
copy of the existing state, but a new execution stack. Calling
lua_close() on this “new” state is just like calling
lua_close() on the already existing state.
Nope. What I wanted to do after each thread was to call lua_gc(), only now current connections are getting garbage-collected along with
the finished ones, which meant I wasn't using luaL_ref() and luaL_unref() properly, probably because I wasn't managing the Lua stack properly
…
Sigh.
It also took a bit of careful coding to get the Lua code to properly
block (lua_yield())
and resume (lua_resume())
when you have input and output asynchronously appearing. It was also a bit
tricky to have one Lua thread write to another Lua thread's socket without
blowing things up. But eventually, not only did I get it such that the echo
service above works (as written above) but the following simple chat script
as well:
if members == nil then
members = {}
end
local function login(socket)
socket:write("Handle you go by: ")
return socket:read()
end
local function wallaction(socket,who,everybody,me)
for connection in pairs(members) do
if connection ~= socket then
connection:write(string.format("%s %s\n",who,everybody))
else
if me ~= nil then connection:write(string.format("%s\n",me)) end
end
end
end
local function wall(socket,who,everybody,me)
return wallaction(socket,who .. ":",everybody,me)
end
function main(socket)
local name = login(socket)
members[socket] = name
wallaction(socket,name,"is in da room!","You are in the room.")
io.stdout:write(string.format("%s has joined the party!\n",name))
while true do
wall(socket,name,socket:read())
end
end
function fini(socket)
io.stdout:write(string.format("%s has left the party!\n",members[socket]))
wallaction(socket,members[socket],"has left the building!")
members[socket] = nil
end
Each Lua thread starts with main(). fini() is
called when a connection is dropped. I also made the daemon such that sending
it a SIGUSR1 will cause it to re-compile the script, so that
changes to the service can be made without having to restart the program as a
whole (which explains the odd way I define members—if we reload
the script, I don't want to lose the current members).
As written, the code isn't multi-threaded—at the C level, it's just one process round-robinning a bunch of Lua threads, so I can get away with the chat process above without worry as only one Lua thread is executing at any one time.
But it's now quite easy to write network daemons, so easy that I'll leave you with one more—an RFC compliant Quote of the Day server:
QUOTESFILE = "/home/spc/quotes/quotes.txt"
quotes = {}
do
local eoln = "\r\n"
local f = io.open(QUOTESFILE,"r")
local s = ""
for line in f:lines() do
if line == "" then
-- each quote is separated by a blank link
if #s < 512 then
table.insert(quotes,s)
end
s = ""
else
s = s .. line .. eoln
end
end
f:close()
end
math.randomseek(os.time())
function main(socket)
socket:write(quotes[math.random(#quotes)])
end
