Sunday, February 11, 2018
“I'm sorry. We can't take your order right now, the computer crashed and we have no idea how to proceed.”
Today is Game Day™. Every other week I get together with some friends to play D&D for a few hours. Sometimes it's hosted here at Chez Boca, other times one of my friend's house. Today it's not at Chez Boca, so as usual, I stop by the Scottish place for a quick lunch. And because I'm weird, I avoid the drive thru.
I noticed a few months ago that McDonald's had installed a few ordering kiosks. Aside from the clunkiness of the interface, it's not that bad. Which upon reflection is strange, because I hate the things at other restaurants. My experience using the Ziosk (not only at the Olive Garden but also Chili's) is not good. A few times the unit has been broken and it took a very confused staff longer to process the check than normal.
And heaven help you if you want to pay by cash.
Cash!
Ha!
So, why do I hate the Ziosks and yet, don't mind the kiosks at McDonald's? I've been thinking about that today, and the best I can come up with is that at a sit-down restaurant, I'm there to enjoy a meal with company and don't want to be distracted by a device not only demanding our attention, but taking up space on a table that can get quite crowded with plates and glasses. I think it also cheapens the waitrons in a way—no longer do you have a single person handling your dinner, but you have several people who cycle through, one person taking the order, one person serving the drinks, another bringing the food, there's not continuity in service. And there's this feeling of replaceability of not only the servers, but of the customers.
And who is the tip for, exactly?
It's hard to describe, and maybe it's just me.
There's also the issue of letting a device, which processes financial transactions into the hands of the customers! Customers which may attempt to hack the devices (although why do I trust the waitrons?). And if the Ziosk crashes … well … I've experienced the chaos it can cause with the staff.
But the kiosk at McDonald's though—there's no tipping. The staff you interact with is just there to take your order. There are other people who do the actual job of preparing your order. And even they don't prepare the entire order. In order to save time we customers are now mostly responsible for filling our own cups of whatever lovely beverage we chose to drink. So that's one job less they have to do. And the kiosks now remove one to two other jobs—taking the order and receiving payment (if you want to pay cash, you have the option to go to the register and pay). So pretty much all that's now left is food preparation.
And I'm fine with that for fast food. Maybe because it's somewhat impersonal to begin with.
All this came to a head today as I was grabing lunch on the way to the D&D game. All the kiosks at The Scottish place had crashed. And there was much confusion at the counter.
Ah, computer automation! Gotta love it.
Monday, February 12, 2018
Tough love, or stultifying ossification? I don't know
There is “Postel's Law” in protocol design that states:
2.10. Robustness Principle
TCP implementations will follow a general principle of robustness: be conservative in what you do, be liberal in what you accept from others.
TRANSMISSION CONTROL PROTOCOL—DARPA INTERNET PROGRAM—PROTOCOL SPECIFICATION
But then there's this:
4.5 Robustness
A well-designed protocol is robust.
Robustness and efficiency are often at odds. For example, although defaults are useful to reduce packet sizes and processing time, they tend to encourage implementation errors.
Counter-intuitively, Postel's robustness principle ("be conservative in what you send, liberal in what you accept") often leads to deployment problems. Why? When a new implementation is initially fielded, it is likely that it will encounter only a subset of existing implementations. If those implementations follow the robustness principle, then errors in the new implementation will likely go undetected. The new implementation then sees some, but not widespread deployment. This process repeats for several new implementations. Eventually, the not-quite-correct implementations run into other implementations that are less liberal than the initial set of implementations. The reader should be able to figure out what happens next.
Accordingly, explicit consistency checks in a protocol are very useful, even if they impose implementation overhead.
On the Design of Application Protocols
I bring this up because I recently came across this:
… The first byte of the clear data was a flags field which, he said, was promptly ossified by a middlebox vendor, leading to packets being dropped when a new flag was set.
That was a classic example of why changing network protocols is hard and what needs to be done to improve the situation. Middleboxes are the control points for the Internet as we know it now. The only defense against ossification of network protocols by middleboxes, he said at the conclusion of the talk, is encryption.
Via Hacker News, QUIC as a solution to protocol ossification [LWN.net]
And I thought, Oh crap! I do that!
More specifically, in SPCDNS
where I explicitly check for unused bits being
zero.
Am I ossifying the DNS protocol by doing this?
I did encounter this a few months ago. I was playing around with
monitoring the traffic on the home network and saw a bunch of MDNS requests flying around. I thought I
would decode them using SPCDNS
to see what exactly was being
queried when I hit a snag—some of the packets were invalid! Upon
investigation, it came down to that single undefined bit in the DNS packet being set. I quickly patched
the code (privately) to accept the packets but I didn't release the changes
since at the time, that last undefined bit was still undefined.
I didn't know how to deal with the situation then, and I still don't.
SPCDNS
can handle records it doesn't understand, but
that one undefined bit has me stymied. I could define the flag and
let a user of my code check for it, but when it does become defined, that
flag name will obviously have to change and that could be a breaking change.
But as of right now, any user code that wants to check for that bit has no
way of doing so (not that I've received any requests for it).
Then there's the fact that because my code was so fussy with verifying the packet, that bugs in other DNS implementations were found via my code. And at work, I put up a fight to remain picky in processing network packets for “Project: Sippy-Cup.” I mean, what's the point of a protocol specification if you don't follow it? I even referenced RFC-3177 (quoted above) as part of my argument for being picky. But I was overridden and had to carry on in the face of utter garbage (and yes, some of what we get is utter garbage). I did however keep track of each violation and we are in the process of talking with a vendor who is sending us garbage as I type.
This is tough.
At the time I wrote SPCDNS
, there was only one undefined bit
left in the DNS packet. Had I
written it back when RFC-1035 was written, there would have
been three undefined bits, and I ask myself how I would have handled that
over the years. I'm not even sure if I would have thought to include those
bits in what is decoded (I don't now).
This will require some thought.
Tortoise and hare racing
I never expected this to be a thing, but tortoise vs. hare, tortoise vs. hare and tortoise vs. hare. And each time, the result is the same.
No, really.
Wednesday, February 14, 2018
Again, it was quite crowded out tonight. What is it with February 14th and everybody eating out? I'm wondering if it's some sort of holiday
As we did last year, Bunny and I had a wonderful dinner at Roots Italian Kitchen, only this time, we were able to skip the whole non-Euclidean parking garage for the Biltmore that for some reason lurks here in downtown Boca Raton. Once you know where to look, it's easily skipped.
We walked inside and Yuriy (the owner of the restaurant with an incredible memory) greeted us warmly and thanked us for coming back and dining with his establishment on St. Valentine's Day.
We were shown to our table, which was decorated with rose petals. Every table was decorated with rose petals. It was an interesting touch I had never seen before.
The appetizer was okay.
Okay, who am I kidding? It was an incredible lobster-and-crabcake appetizer that was easily the best part of the entire meal. That's not to say the rest of the meal was bad—it wasn't. The Chicken Bella Risotto (grilled portobellos with chicken in a creamy risotto) was wonderful (we both had it) and dessert was worth the wait (Tiramisu for Bunny, and a brownie á la mode sans mode for me; sadly, the only misstep of the meal was the restaurant running out of vanilla ice cream, but still, that brownie was good).
But that lobster-and-crabcake appetizer … wow!
It was a good thing we parked a few blocks away (to avoid that non-Euclidean garage lurking in downtown Boca Raton)—we could use the walk after a meal like that.
Monday, February 19, 2018
Fun with banging moon rocks
Back in Debtember, I was notified of a potential problem with one of my Lua modules, speficically, the LuaRocks specfile of said module—it failed to compile properly on Mac OS-X.
I finally got around to it and fixed the issue, and while I was doing that, I figured I might as well fix some of the other Lua modules I have.
You can always tell when I do this because the version numbers accumulate in a rapid succession.
My CBOR Lua module went from version 1.2.4
to 1.2.11
in the span of an hour or so.
And it's all due to the picky little details.
As I mentioned a few months ago,
I pull the version directly from git
.
But I also have to define the version in the Makefile
in case someone downloads the code
instead of cloning the repository.
But there have been times when I updated the GitHub repository without updating the Makefile
,
or forgot to properly rename the LuaRocks specfile
(probably as a sanity check, the filename needs to include the version number which is also specified inside the file).
To handle those cases,
I added a script to git
to check for such things before pushing the update to GitHub.
So it should be easy, right?
Not quite.
One issue I forgot to check in the script was to ensure the LuaRocks specfile was syntactically correct. So there goes one version.
The Lua module is written with C99.
So I specified c99
as the compiler in the LuaRocks specfile.
Only the default options LuaRocks uses breaks on Mac OS-X with c99
as the compiler.
The why is simple—Mac OS-X uses clang
,
which supports different options than GCC
.
But LuaRocks defaults to using GCC
when compiling on Mac OS-X.
That's fine,
because there's a GCC
wrapper on Mac OS-X that translates the options for GCC
to clang
,
but only if the compiler you use is GCC
.
On Mac OS-X,
c99
defaults to clang
so of course it fails because clang
receives options for GCC
.
I can use the default compiler that LuaRocks defines on Mac OS-X and it compiles fine.
But not on Linux,
where GCC
defaults to “not C99.”
I finally figured out the magical incantation to fix that:
build = { platforms = { windows = { type = "builtin", modules = { ['org.conman.cbor_c'] = { sources = { 'cbor_c.c', 'dnf.c' }, defines = { 'VERSION="' .. source.tag .. '"' }, }, ['org.conman.cbor'] = 'cbor.lua', ['org.conman.cbor_s'] = 'cbor_s.lua', ['org.conman.cbormisc'] = 'cbormisc.lua', } } }, type = "make", build_variables = { CC = "$(CC)", CFLAGS = "$(CFLAGS) -DNDEBUG -I$(LUA_INCDIR)", LDFLAGS = "$(LIBFLAG)", LUA = "$(LUA)", }, install_variables = { LIBDIR = "$(LIBDIR)", LUADIR = "$(LUADIR)", LUA = "$(LUA)", }, platforms = { linux = { build_variables = { CC = "gcc -std=c99" } }, solaris = { build_varaibles = { CC = "c99" } }, } }
And it worked fine.
But it burned through some additional versions as I worked out the details.
And then I found an issue—if you look closely,
you'll notice that I have the patforms
stanza twice.
That means the second stanza replaces the first stanza and thus,
I lost the Microsoft Windows compiling instructions.
So yet another version goes by as I merge the two platforms
stanzas into a single block.
And then I forgot to update the version number in the LuaRocks specfile. It had the right name, but the wrong internal version.
So there's another version and an update the the checking script.
By now, you might be wondering why I don't bother testing this before updating the version number. And it's simple—I can't! The LuaRocks specfile contains this bit:
package = "org.conman.cbor" version = "1.2.11-1" source = { url = "git://github.com/spc476/CBOR.git", tag = version:match "[^-]+" }
To test it, LuaRocks will download the code from GitHub using the given tag value, which is based off the version number. But if I haven't checked in the code for that tag, LuaRocks will fail. So I have to check in the code with the proper tag before I can test the LuaRocks file.
Yes,
I could do the code updates and afterwards fix the LuaRocks specfile,
but the script that's run prior to pushing changes checks to see if the latest version is a “good version,”
like 1.2.10
.
It will reject a version like 1.2.10-1-g5b75b4f
(which is git
's way of saying “one commit past the previous tagged version”).
I added that check to ensure LuaRocks would get the proper version of everything—I would only push when I had everything ready for a new version.
Ah—unintended consequences.
We'll see if the next update goes smoother. One of these days I'll get this right.
Why one might want to load dynamic objects from memory instead of from disk
Years ago I started on a little project entitled the LEM Project. It was a project to make distribution of applications written in Lua easier (It's named after the Apollo Lunar Module—aka the Lunar Excursion Module; Lua is “moon” in Portuguese). I had a few goals:
- allow distribution of multiple versions of a Lua module;
- allow for different architectures for the same module written in C;
- run the entire application from the distribution file (nothing to install).
And I almost hit all the goals.
The first two were easy. I read over the ZIP file specification and saw that
- multiple files of the same name were entirely possible—nothing in the specification disallows it;
- additional metadata can be added to each file beyond what is defined in the specification.
So armed,
I created zip
file with some addtional metadata and …
[spc]lucy:~/projects/LEM>./zipf.lua sample.lem The Eagle Has Landed SunOS sparcv9 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.base64 SunOS sparcv9 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.crc SunOS sparcv9 LGPL3+ Lua 5.1 5.1 5.1 MODULES/org.conman.env SunOS sparcv9 LGPL3+ Lua 5.1 5.1 5.1 MODULES/org.conman.errno SunOS sparcv9 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.fsys SunOS sparcv9 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.hash SunOS sparcv9 LGPL3+ Lua 5.1 5.1 1.1 MODULES/org.conman.iconv SunOS sparcv9 LGPL3+ Lua 5.1 5.1 5.1 MODULES/org.conman.math SunOS sparcv9 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.net SunOS sparcv9 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.pollset SunOS sparcv9 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.process SunOS sparcv9 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.strcore SunOS sparcv9 LGPL3+ Lua 5.1 5.1 1.0.0 MODULES/org.conman.sys SunOS sparcv9 LGPL3+ Lua 5.1 5.1 5.1 MODULES/org.conman.syslog Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.base64 Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.crc Linux x86 LGPL3+ Lua 5.1 5.1 1.0.0 MODULES/org.conman.env Linux x86 LGPL3+ Lua 5.1 5.1 1.0.0 MODULES/org.conman.errno Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.fsys Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.fsys.magic Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.hash Linux x86 LGPL3+ Lua 5.1 5.1 1.1.1 MODULES/org.conman.iconv Linux x86 LGPL3+ Lua 5.1 5.1 5.1 MODULES/org.conman.math Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.net Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.net.ipacl Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.pollset Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.process Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.strcore Linux x86 LGPL3+ Lua 5.1 5.1 1.2.0 MODULES/org.conman.sys Linux x86 LGPL3+ Lua 5.1 5.1 1.0.2 MODULES/org.conman.syslog Linux x86 LGPL3+ Lua 5.1 5.1 MODULES/org.conman.tcc LGPL3+ Lua 5.1 5.1 MODULES/org.conman.cc LGPL3+ Lua 5.1 5.1 MODULES/org.conman.date LGPL3+ Lua 5.1 5.1 MODULES/org.conman.debug LGPL3+ Lua 5.1 5.1 MODULES/org.conman.dns.resolv LGPL3+ Lua 5.1 5.1 MODULES/org.conman.getopt LGPL3+ Lua 5.1 5.1 MODULES/org.conman.string LGPL3+ Lua 5.1 5.1 MODULES/org.conman.table LGPL3+ Lua 5.1 5.1 MODULES/org.conman.unix MIT Lua 5.1 5.1 0.10 MODULES/lpeg MIT Lua 5.1 5.1 0.10 MODULES/re Linux x86 MIT Lua 5.1 5.1 0.12 MODULES/lpeg Linux x86 MIT Lua 5.2 5.2 0.12 MODULES/lpeg MIT Lua 5.1 5.2 0.12 MODULES/re Linux x86 MIT/X11 Lua 5.1 5.1 0.4.work3 MODULES/zlib MIT/X11 Lua 5.1 5.1 2.0.2 MODULES/socket Linux x86 MIT/X11 Lua 5.1 5.1 2.0.2 MODULES/socket.core MIT/X11 Lua 5.1 5.1 2.0.2 MODULES/socket.ftp MIT/X11 Lua 5.1 5.1 2.0.2 MODULES/socket.http MIT/X11 Lua 5.1 5.1 2.0.2 MODULES/socket.smtp MIT/X11 Lua 5.1 5.1 2.0.2 MODULES/socket.tp MIT/X11 Lua 5.1 5.1 2.0.2 MODULES/socket.url MIT/X11 Lua 5.1 5.1 1.0.1 MODULES/ltn12 FILES/APPNOTE.TXT FILES/COPYING FILES/README FILES/Miscellaneous Things About Nothing [spc]lucy:~/projects/LEM>
You can see there are several copies of each module. The modules listed without a system (like Linux or SunOS) are written in Lua; the other ones are in C and both the system and architecture are listed. The license is included, along with the Lua verions the module will work for (it's “minimum version” and “maximum version”) as well as the version of the module (if known).
It's even a valid zip
file:
[spc]lucy:~/projects/LEM>unzip -l sample.lem Archive: sample.lem The Eagle Has Landed Length Date Time Name -------- ---- ---- ---- 25472 05-24-14 17:10 MODULES/org.conman.base64 13448 05-24-14 17:10 MODULES/org.conman.crc 12200 05-24-14 17:10 MODULES/org.conman.env 18688 05-24-14 17:10 MODULES/org.conman.errno 57032 05-24-14 17:10 MODULES/org.conman.fsys 24952 05-24-14 17:10 MODULES/org.conman.hash 17664 05-24-14 17:10 MODULES/org.conman.iconv 17648 05-24-14 17:10 MODULES/org.conman.math 77944 05-24-14 17:10 MODULES/org.conman.net 26296 05-24-14 17:10 MODULES/org.conman.pollset 88256 05-24-14 17:10 MODULES/org.conman.process 37848 05-24-14 17:10 MODULES/org.conman.strcore 15392 05-24-14 17:10 MODULES/org.conman.sys 24312 05-24-14 17:10 MODULES/org.conman.syslog 14175 06-12-14 22:04 MODULES/org.conman.base64 8214 06-12-14 22:04 MODULES/org.conman.crc 7193 06-12-14 22:04 MODULES/org.conman.env 10690 06-12-14 22:04 MODULES/org.conman.errno 31885 06-12-14 22:04 MODULES/org.conman.fsys 15567 06-12-14 22:04 MODULES/org.conman.fsys.magic 14067 06-12-14 22:04 MODULES/org.conman.hash 10197 06-12-14 22:04 MODULES/org.conman.iconv 10816 06-12-14 22:04 MODULES/org.conman.math 43651 06-12-14 22:04 MODULES/org.conman.net 25248 04-18-14 20:06 MODULES/org.conman.net.ipacl 15914 06-12-14 22:04 MODULES/org.conman.pollset 49607 06-12-14 22:04 MODULES/org.conman.process 15666 06-12-14 22:04 MODULES/org.conman.strcore 9763 06-12-14 22:04 MODULES/org.conman.sys 13303 06-12-14 22:04 MODULES/org.conman.syslog 21218 06-12-14 22:04 MODULES/org.conman.tcc 5617 06-12-14 22:04 MODULES/org.conman.cc 2500 06-12-14 22:04 MODULES/org.conman.date 3829 06-12-14 22:04 MODULES/org.conman.debug 2966 06-12-14 22:04 MODULES/org.conman.dns.resolv 3260 06-12-14 22:04 MODULES/org.conman.getopt 2464 06-12-14 22:04 MODULES/org.conman.string 5243 06-12-14 22:04 MODULES/org.conman.table 2732 06-12-14 22:04 MODULES/org.conman.unix 40081 05-25-14 15:17 MODULES/lpeg 6029 05-24-14 00:36 MODULES/re 40045 05-28-14 15:24 MODULES/lpeg 40045 05-28-14 15:24 MODULES/lpeg 6286 05-28-14 15:24 MODULES/re 19794 05-30-14 21:29 MODULES/zlib 4451 05-28-14 14:52 MODULES/socket 55449 05-28-14 14:52 MODULES/socket.core 9243 05-28-14 14:52 MODULES/socket.ftp 12330 05-28-14 14:52 MODULES/socket.http 8074 05-28-14 14:52 MODULES/socket.smtp 3612 05-28-14 14:52 MODULES/socket.tp 11036 05-28-14 14:52 MODULES/socket.url 8331 05-28-14 14:52 MODULES/ltn12 161412 05-09-14 01:24 FILES/APPNOTE.TXT The ZIP file format 7651 05-25-14 18:53 FILES/COPYING 9789 06-07-14 22:13 FILES/README Much Ado About Nothing 3946 05-10-99 23:00 FILES/Miscellaneous Things About Nothing If you this this has significance, think otherwise -------- ------- 1250541 57 files [spc]lucy:~/projects/LEM>
(Although if you try to unzip this file,
the unzip
program will ask you what you want to do with the duplicate files.)
The intent was that only those modules that match the Lua version, system (if a C-based Lua module) and architecture (again, C-base Lua module) match, load the module; otherwise, it would be ignored.
And it worked. For the most part.
I could load Lua modules written in Lua directly from the zip
file with no problem.
I even got LuaRocks to run completely from the zip
file
(with some hacks so it could load the configuration file).
But I could not do that for the C-based Lua modules.
For those,
you had to write them out to disk to load them up.
And because of that,
I lost interest in continuing the project.
Until now. I just came across MojoELF (via Project: 2ine which was linked via Hacker News)—a project to load ELF binaries directly from memory (and those C-based Lua modules I have are ELF binaries), which is exactly what I wanted all those years ago.
Wednesday, February 28, 2018
Notes on an overheard conversation while driving south along I-95
“When are they going to finish all this contruction? The road here is terrible!”
“It keeps the drivers on their toes.”
“The lanes are all uneven! It could force you into the next lane!”
“Yes, that's what makes it exciting.”
A potential way to have spaces in filenames and not break the Unix command line
A thread at
Hacker News about using makefiles for JavasScript
was about the
difficulty in using filenames with spaces.
And yes, it is difficult.
A Makefile
such as:
hello world:
Results in:
[spc]lucy:/tmp/foo>make make: Nothing to be done for `hello'. [spc]lucy:/tmp/foo>
Enclosing the filename with quotes (single or double, it doesn't matter) gives the same results. If I escape the space like:
hello\ world:
make
gives:
[spc]lucy:/tmp/foo>make cc hello world.c -o hello world cc: hello: No such file or directory cc: world.c: No such file or directory cc: world: No such file or directory cc: no input files make: *** [hello world] Error 1 [spc]lucy:/tmp/foo>
So yes,
using a filename with spaces is problematic with make
.
Part of that is the unique nature of the space character.
In ASCII,
it's grouped next to the information separator characters and thus,
could be treated as another type of separator character
(and on input, it usually is considered such).
It could also be considered a control character as a format effector such that it causes the character position to advance one place to the right
(and is thus used as such with output).
It's the prevasive use of space as a separator in Unix that causes the most issues,
such with make
,
and the command line in general.
But there is a solution …
[spc]lucy:/tmp/foo>ll total 12 -rw-r–r– 1 spc spc 14 Feb 28 18:13 Makefile -rw-r–r– 1 spc spc 76 Feb 28 18:13 hello world.c -rw-r–r– 1 spc spc 227 Feb 28 18:13 x.lua [spc]lucy:/tmp/foo>cat Makefile hello world: [spc]lucy:/tmp/foo>make cc hello world.c -o hello world [spc]lucy:/tmp/foo>./hello world Hello, world! [spc]lucy:/tmp/foo>
No, the output is not faked.
Yes,
the filename is hello world.c
.
The name,
however,
is not pure ASCII—it contains the Unicode character for a “non-breaking space”.
Cheating?
Perhaps.
But it is defined as a space (graphically),
and more importantly,
it's not considered an information separator by Unix utilties.
It also requires a filesystem that can support Unicode (or in my case, UTF-8)
and a command line that also supports Unicode (or again in my case, UTF-8).
And it was also not that easy to create the filename and Makefile
with the non-breaking space.
But other than those minor issues [Ha! —Editor], hey—spaces! In filenames!