The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Tuesday, January 01, 2013

Happy New Year

It's a war zone out there, what with the rockets' red glare, the explosions, the crazy neighbor attempting to scare off the old year and in the process, scaring in the new year as well as the neighbors. No way I'm going out there in these conditions. Even Murrow would give pause before entering the mæstrom of fire emanating from across the street. No fool, I (or Murrow).

So, from the relative safely of Chez Boca, I bid you Happy New Year.

At least this year we don't have to worry about the world ending in an Emmerichesque orgy of disaster porn or New Age woo-woo mystical nonsense. Thank <insert deity of your choice> for that!

Wednesday, January 02, 2013

Notes from the Bible Belt

For Xmas Christmas, Bunny gave me a subscription to the Transylvania Times, the newspaper of Brevard, NC (and not a taboid following the exploits of Vlad Ţepeş).

I like Brevard; it has that small town, “Mayberry” feeling. The downtown is easily walkable and has some excellent restaurants despite its small size. And every time I'm there I take the time to buy the local newspaper (published twice weekly) as it's more entertaining than any local newspaper here in Lower Sheol.

For instance, this letter to the Editor:

Who are they? Are you one of them?

Why are they taking God out of our country, out of our schools, off our highways and anywhere else they can?

They took prayer out of our schools. Now our children are being killed.

They stopped parents from correcting our children. They call it abuse.

Yes, I have to remind myself that North Carolina is part of the Bible Belt, so there's some level of craziness to contend with. And except for the highway bit (odd—I don't recall God ever being involved with the US road system) this type of letter is something one would expect from the Bible Belt.

Then we get to the real craziness—

They took Christ out of Christmas by putting Xmas and crossing him out.

On our highways, I see signs, “Deer Xing.” It's taking the cross out of crossing, the cross where Jesus shed his blood to forgive our sins.

The star that shone brightly in the sky, showing where Jesus was born, is not on the top of our Christmas trees. In its place is a ribbon or something else.

…

Um … what?

Wow. Just wow.

First off, the use of “Xmas” for “Christmas” isn't a recent phenomenon, nor is it part of a conspiracy to secularize Christmas. Xmas has a long history going back to 1755; further if you accept “Xp̄es mæsse” as a precedent (circa 1100). And using “X” as shorthand for “Christ” goes back to 1485.

And that's exactly what it was used for—shorthand. “They” took the “Christ” out of Christ a long time ago.

And that bit about deer crossings—funny, I don't recall Jesus dying on the cross for the sins of ungulates. As far as I can tell, the FHWA requires a picture of a deer for a “Deer Xing” sign, but North Carolina might use alternative signage, so blame North Carolina for Jesus' inability to save ungulates.

And then we have the next letter to the editor …

---

More notes from the Bible Belt

Not all people who live in the Bible Belt are Fundamentalist fruit baskets; no, sometimes they're clueless progressives who slept through civics class.

Why, in violation of the Constitution, has Rosman [Rosman being the only other town in Transylvania County, and is even smaller than Brevard. —Editor] allowed a local church to present a live Nativity scene on public property?

What local official is responsible for enforcing the law of the land in this situation?

The author might be thinking of the “separation of church and state,” but that phrase nevers appears in the Declaration of Independence nor in the Constitution of the United States of America (it comes from a letter written by Thomas Jefferson).

The only strictions placed upon the Federal Government is the First Amendment:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof …

and this bit in Article VI:

… no religious test shall ever be required as a qualification to any office or public trust under the United States.

It says nothing about banning religious displays. In fact, until the Fourteenth Amendment, it might have passed Constitutional muster for a state (for example, Utah had it been admitted as a state prior to 1868) to legislate an establishment of religion, under the Tenth Amendment (as the First Amendment only applies to Congress, not a state). The Fourteenth Amendment pushes the restrictions the Constitution placed on the Federal government to the State governments (so there goes any State-level mandated religion) but even then, it still says nothing about a government displaying Nativity scenes.

Knowing nothing more about the “Rosman Nativity Scandal of 2012” than what's presented in the opinion letter, it wouldn't surprise me if the said local church didn't fill out the appropriate forms and pay the appropriate fees to host a public display on public property, allowing the church members to exercise the other part of the First Amendment:

… or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.

I'm sure that had a group wanted to present a Kwanzaa celebration on public property they could fill out the proper forms and paid the proper fees just like the church members probably did.

Sheesh! It's not like the Constitution is hard to read.

Thursday, January 03, 2013

The Average Human, Part II

Disney asking its female employees to to help design an animated male lead reminded me of the average human, a post I made a few years ago.

Amazingly enough, the Face Research Make an Average page still exists, only with a new crop of pictures.

I thought it might be interesting to compare how averages made with a fresh set of pictures would hold up to the previous set of pictures and remarkedly, they are very similar in nature.

The 2007 pictures appear first, then then current, 2013 pictures. And then, the average of both men and women to create—

I wonder how closer we are to my predictions of digital actors?

Friday, January 04, 2013

Rebooting commercial jetliners

I think my friend Gregory will like these: cold start of an Airbus 320 and a cold start of a Boeing 737 (links via Flutterby).

As I was viewing the Airbus 320 video, I was struck that landing the plane appeared to be vastly easier than starting booting it up! And watching how complicated it was for the planes makes the episode where Picard & Co. reboot the Enterprise seem all the more deus ex machina than it already was.

Saturday, January 05, 2013

It's rough being The Computer Guy

Reason #9—Every Conversation You Have Is Roughly The Same

When the computer guy dares to mention what he does for a living, the typical response is, “I have a question about my home computer”

Or when the computer guy first hears about a widespread problem within the computer network he's responsible for, he can barely begin to assess the problem before a dozen other people call to report the same problem.

Or when the computer guy explains a certain process on a computer to a user who is incapable of retaining the process, he will inevitably need to reinstruct the user of this same process—indefinitely.

Via The Endeavour, 10 Reasons It Doesn't Pay To Be “The Computer Guy”

When I read this, I was taken back to Gregory's rant about the user community (which was more of a rant about people who ask the same questions over and over again).

What really struck home, though, was this comment at The Endeavour:

No thank you. I dropped residential support and told all my businesses we put file servers in their business and we re-image workstations at the first hint of trouble. I also emphasize to businesses that work machines should be work machines.

Those jobs began to actually be worth the time.

It doesn't pay to be the computer guy

Fortunately, my exposure to residential support was brief (back in the mid 90s I worked at a local ISP and I was stuck on the front line of support) and since then, I've been able to obtain jobs where talking to residential customers isn't a concern—it pays to be a programmer.

Sunday, January 06, 2013

The lengths I go to for your hypertext environnment for reading pleasure

One of the many reasons I stopped blogging for so long (a post in May, a few in August, then nothing til January 1^st) is that it felt too much like work. And in looking back, I see I've been bitching about this topic for at least ten years now, and sadly, the process I go through making an entry today is the same as it was ten years ago. It's the process of writing raw HTML to be posted on the blog.

At least the actual act of submitting a post to be published by mod_blog (the engine that runs my blog) is automated.

Another issue is that I can't quite type fast enough to get my thoughts out. Couple that with my desire to actually use HTML as it was intended (you know, to provide a hypertext environment for reading) causes me to stumble. It also doesn't help that I'm also thinking about several different things at once [okay, don't forget to link back to the 2002 bitch entry] [Done. —Editor] [also, don't forget to talk about throwww and geeze, there's already thirteen tabs just for this entry alone] [Also done. —Editor]. No wonder my Lovely and Talented Copy Editor has her work cut out for her.

Sure, I could use a simpler markup langauge, like say, Markdown [make sure I get back and make a link here] [Done. —Editor] but I know HTML; why bother learning a half-baked markup language that isn't really designed for a hypertext environment for reading (in my opinion).

Throwww [oh, almost forgot to mention that I found the link at Hacker News] appears to be a system where you can edit the page directly (much like the original idea Tim Berners-Lee had when he wrote the first web browser [and here I would go off and try to find pages to link to about Tim Berners-Lee's ideas about editing webpages directly in the browser, but I don't want to digress and … oh, lost the thought … oh, wait, got it back.] [four, count them, four, tabs to find a suitable link. —Editor] but there doesn't appear to be a way see the rendered results and it appears to use its own half-baked markup language.

Blech.

Distraction Free Writing (also via Hacker News) also uses Markdown (what? Is HTML too damn hard for people to use that we must use a watered down half-baked ill-specified [don't forget to link to the author of Markdown refusing to standardize it] [Done. —Editor] formatting language that isn't nearly as expressive as HTML?) and does have a way of previewing the results (that's nice) but it's hardly distraction free what with the exceedingly loud typing noises it makes (at the time—it seems the author of Distrction Free Writing received enough negative feedback to have fixed that) exceeds the loud typing noises my keyboard [don't forget a link there either] [Done. —Editor] makes.

To compound the problems, a solution I had [link to 2008/12/03.1] [Done. —Editor] no longer works, as sometime over the past year the version of Firefox [yet another link] [Nah, forget the link to Firefox. It caused us enough pain. —Editor] autoupdated itself into oblivion (that is—the last update broke on my Linux system so bad I can't even run it) so now I'm forced to use Firefox 1.5 that came installed on the system (yes yes, I'm running a version of Firefox that existed when the dinosaurs roamed the earth, but then again, anything software related that is older than twenty minutes is as old as the dinosaurs and I'll stop here before I digress on the obessiveness of constant upgrades our industry forces upon us) and thus, the program I used to construct <BLOCKQUOTE>s doesn't like Firefox 1.5.

Crap.

So now it's time to go back through some twenty tabs, adding links to make that oh so nice hypertext environment for reading.

[A total of twenty-three webpages were referenced (not necessarily linked to) in the making of this post for your hypertext environment for reading pleasure. —Editor]

[The Editor here is not to be confused with the Lovely and Talented Copy Editor. –Editor]

Monday, January 07, 2013

I think a more serious question is—why do I know about Zeta Reticuli and the Reptilians from Draco?

As I was taking out the garbage (and yes, this happens very early in the morning) I noticed an unusually bright light in the western sky changing colors, and not moving. The rest of the stars weren't twinkling that much, and besides, I've never seen a twinkling star change colors quite like the light I was observing.

I went back inside, grabbed my old-school telescope (based off a design that Galileo used) but it proved to be a bit difficult to use. Bunny then came outside, wondering what I was looking at with the telescope. I showed her, and we both came back inside for some binoculars.

Well, that didn't resolve the issue any. It still looked like a point of light rapidly changing colors. A few minutes later, it winked out of existence.

Okay, so technically what I saw was an unidentified flying object—an object, in the sky, that was unidentifiable. Was it an actual alien spacecraft from the Zeta Reticuli system? No, I doubt it; everybody knows it's the Reptilians from the Draco constellation that rule us here).

Okay, seriously, it was probably an airplane, headed due west (thus it didn't appear to be moving relative to our position at Chez Boca) that we saw.

Tuesday, January 08, 2013

This day in history

Interesting! Not only is today the birthday of the Supreme Commander of the cleanest race on the planet, the Sagacious Leader of the Democratic People's Republic of Korea, Kim Jong Un, but it is also the feast day of Our Lady of Prompt Succor, the day a UFO landing left phystical evidence (France, 1981), and the only time in the history of the United States our debt was fully paid off (back in 1835!).

---

I'm enjoying a lovely cup of tea

For some strange reason (I doubt it's to celebrate Kim Jong Un's birthday) I received a beautiful tea mug with infuser, and several ounces of Golden Monkey tea, which is quite good.

Wednesday, January 09, 2013

Stupid Twitter Tricks III

The lack of activity here has been so severe, I didn't notice that the Twitter feed stopped showing up in the Obligatory Sidebar™. It took some time to track down the issue, and it appears that the Twitter API changed locations (and I find it weird that APIs can now refer to URLs, but perhaps that's my old-school mentality showing). At least it was an easier fix than the last time Twitter puked.

Sigh.

Thursday, January 10, 2013

I reject your reality and substitute my own!

REMOVING her ex-husband from more than a decade of memories may take a lifetime for Laura Horn, a police emergency dispatcher in Rochester. But removing him from a dozen years of vacation photographs took only hours, with some deft mouse work from a willing friend who was proficient in Photoshop, the popular digital-image editing program.

Like a Stalin-era technician in the Kremlin removing all traces of an out-of-favor official from state photos, the friend erased the husband from numerous cherished pictures taken on cruises and at Caribbean cottages, where he had been standing alongside Ms. Horn, now 50, and other traveling companions.

“In my own reality, I know that these things did happen,” Ms. Horn said. But “without him in them, I can display them. I can look at those pictures and think of the laughter we were sharing, the places we went to.”

“This new reality,” she added, “is a lot more pleasant.”

…

After her father died several years ago, Theresa Newman Rolley, an accountant in Williamsport, Pa., hired Wayne Palmer, a photographic retoucher, to create a composite portrait of the two of them because she had no actual one of them together.

That photograph—of a moment that never happened—now hangs in her living room. It still brings tears to her eyes, she said.

“It's the only picture of my dad and me together,” Ms. Rolley said, adding, “If the only reason I can get one is cropping it in, it still means the same to me.”

I Was There. Just Ask Photoshop.

I remember back in FAU the drawing instructor told us to draw what we saw half the time, and the other half, told us to ignore any distracting details that didn't pertain to the subject at hand. So in a sense, we were manipulating reality as we saw it.

Also back at FAU, the photography instructor (and this was back before everything turned digital so we were using 35mm cameras, and even then we weren't photographing reality. If we were, we wouldn't be using black-and-white film (real life is in color these days) has us manipulating reality. We had the power to adjust the shutter speed (a fast shutter to freeze the action; a slow shutter to convey speed of motion), the f-stop (a low setting to blur the foreground and background around the subject; a high setting to keep everything in focus) and even the sensitivity of the film (a slow film speed would produce super crisp pictures but required tons of light; a fast film speed could do wonders in low light but the results are grainy); all “adjustments” we could do, in camera, to modify “reality” (and when you get to color—then you have the ability to manipulate the color temperature; make the scene cool and impersonal, warm and inviting, or totally alien in nature).

And once the film was developed (less development, less contrast, a softer picture; more development, more contrast, a harsher look), you could futher manipulate the resulting images; dodging and burning, cropping, even the paper used to expose the image (matt finish, high gloss), as well as the minor touch-ups with a fine brush and ink to remove any white spots due to dust on the enlarging lens or dark lines due to scratches on the film.

And the stuff mentioned in the article? Just a more modern version of scissors removing that ex-person from your life.

Or a gross violation of reality.

Take your pick.

Because visual representations of reality have always been m anipulated in one way or another.

Friday, January 11, 2013

Did I repeat myself? Did I repeat myself? Did I repeat myself? Let me fix that.

“Did you mean to post four copies of your post?” asked Bunny.

“No, why?”

“Because there are four copies of your post, that's why,” said Bunny.

“How do you link to the post in question like that when you speak?” I asked. She shook her head and walked away. “Bunny?”

---

So yes, there was a glitch last night. The only way for four copies to show up like that would be for four copies to be sent via email (which is my preferred interface). And in checking the logs, I did indeed see four copies show up:

Jan 11 05:03:58 brevard postfix/local: 084412EB72E4: to=<XXXXXXXXXXXXXXXXXXXXX>, relay=local, delay=3, status=deferred (Command died with signal 6: "/home/spc/web/sites/boston.conman.org/htdocs/boston.cgi --config /home/spc/web/sites/boston.conman.org/journal/boston.cnf --email --cmd new". Command output: *** glibc detected *** double free or corruption (fasttop): 0x09cf8d38 *** )
Jan 11 05:27:10 brevard postfix/local: 084412EB72E4: to=<XXXXXXXXXXXXXXXXXXXXX>, relay=local, delay=1395, status=deferred (Command died with signal 6: "/home/spc/web/sites/boston.conman.org/htdocs/boston.cgi --config /home/spc/web/sites/boston.conman.org/journal/boston.cnf --email --cmd new". Command output: *** glibc detected *** double free or corruption (fasttop): 0x08646d38 *** )
Jan 11 06:00:29 brevard postfix/local: 084412EB72E4: to=<XXXXXXXXXXXXXXXXXXXXX>, relay=local, delay=3394, status=deferred (Command died with signal 6: "/home/spc/web/sites/boston.conman.org/htdocs/boston.cgi --config /home/spc/web/sites/boston.conman.org/journal/boston.cnf --email --cmd new". Command output: *** glibc detected *** double free or corruption (fasttop): 0x0934bd38 *** )
Jan 11 07:07:09 brevard postfix/local: 084412EB72E4: to=<XXXXXXXXXXXXXXXXXXXXX>, relay=local, delay=7394, status=sent (delivered to command: /home/spc/web/sites/boston.conman.org/htdocs/boston.cgi --config /home/spc/web/sites/boston.conman.org/journal/boston.cnf --email --cmd new)

The fact that four showed up showed that mod_blog crashed when exiting. I find it disturbing that I've been running the latest version since the 1^st and this is the first time it crashed. Not only that, but it crashed three times in a row then worked.

Such bugs are terrifying.

So, inspired by the crash reports in Redis, I decided to roll my own and it was surprisingly easy given the output (from a test of the code):

CRASH: pid=6103 signal='Aborted'
CRASH: reason='Unspecified/untranslated error'
CRASH: CS=0073 DS=C02E007B ES=007B FS=0000 GS=0033
CRASH: EIP=00B767A2 EFL=00000246 ESP=BFE162F8 EBP=BFE1630C ESI=000017D7 EDI=00CBBFF4
CRASH: EAX=00000000 EBX=000017D7 ECX=000017D7 EDX=00000006
CRASH: UESP=BFE162F8 TRAPNO=00000000 ERR=00000000
CRASH: STACK DUMP
CRASH: BFE162F8:                         C5 78 BB 00 F4 BF CB 00 
CRASH: BFE16300: F4 BF CB 00 00 00 00 00 C0 C6 F0 B7 3C 64 E1 BF 
CRASH: BFE16310: 29 93 BB 00 06 00 00 00 20 63 E1 BF 00 00 00 00 
CRASH: BFE16320: 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE163A0: A7 C4 CB 00 A8 C4 CB 00 F4 BF CB 00 A7 C4 CB 00 
CRASH: BFE163B0: 60 C4 CB 00 D8 63 E1 BF 73 D0 C4 00 F4 BF CB 00 
CRASH: BFE163C0: 4F E7 BE 00 02 00 00 00 B8 C6 54 08 42 00 00 00 
CRASH: BFE163D0: B8 C6 54 08 42 00 00 00 14 64 E1 BF 9C E9 BE 00 
CRASH: BFE163E0: 60 C4 CB 00 B8 C6 54 08 42 00 00 00 32 30 31 34 
CRASH: BFE163F0: A7 C4 CB 00 00 00 00 00                         
CRASH: STACK TRACE
CRASH:        /home/spc/source/boston/build/boston[0x805d3c4]
CRASH:        /home/spc/source/boston/build/boston[0x805daa2]
CRASH:        /lib/tls/libc.so.6[0xbb79b0]
CRASH:        /lib/tls/libc.so.6(abort+0xe9)[0xbb9329]
CRASH:        /lib/tls/libc.so.6(__assert_fail+0x101)[0xbb0e41]
CRASH:        /home/spc/source/boston/build/boston[0x8053f30]
CRASH:        /home/spc/source/boston/build/boston[0x805d0b9]
CRASH:        /home/spc/source/boston/build/boston(ChunkProcessStream+0xc8)[0x805d32e]
CRASH:        /home/spc/source/boston/build/boston(ChunkProcess+0xcb)[0x805d240]
CRASH:        /home/spc/source/boston/build/boston(generic_cb+0x74)[0x8053a68]
CRASH:        /home/spc/source/boston/build/boston(pagegen_days+0x1c4)[0x8057f5c]
CRASH:        /home/spc/source/boston/build/boston(generate_pages+0xd0)[0x8057bfc]
CRASH:        /home/spc/source/boston/build/boston[0x80501bb]
CRASH:        /home/spc/source/boston/build/boston(main_cli+0x1e5)[0x80500cd]
CRASH:        /home/spc/source/boston/build/boston(main+0x133)[0x8051eb3]
CRASH:        /lib/tls/libc.so.6(__libc_start_main+0xd3)[0xba4e93]
CRASH:        /home/spc/source/boston/build/boston[0x804cdbd]

POSIX allows you to register a signal handler with additional contextual information. This contextual information includes the contents of the CPU registers and it's just a matter of finding the structure definition in a header file. Once I have the registers, it's not hard to get the stack dump.

The stack trace was even eaiser than that. It seems that the Linux C library comes with a few functions to do just that—backtrace() collects the return addresses off the stack; backtrace_symbols() will resolve the addresses back to function names and backtrace_symbols_fd() does the same, only it writes the informtaion to a file. I ended up using backtrace_symbols_fd() since I'm executing in the context of a signal handler, and I know that open(), read() and close() are marked as “async-signal-safe” (that is, safe to call in a signal handler) by POSIX, but malloc() and free() are not, and backtrace_symbols() is known to call malloc() (and thus, I would need to call free()) and I don't want to risk a crash in the crash report code. I'm taking a risk because backtrace() and backtrace_symbols_fd() could call non-async-signal-safe functions, but I'm doing as much as I can to get as much context as I can as safely as I can.

---

The utility of a crash report

Well, that was fast. I installed the new version of mod_blog, went out to dinner with Bunny, and came home to over 80 crash reports, reason: “Address not mapped for object.” Or in other words, “the program tried to access memory that didn't belong to it.”

Bad news—it wasn't every request, just certain ones. Even worse news—the logging server and the web server have a 3′50″ difference in time (that took me about fifteen minutes to realize). I also realized that even with stack dump and trace, there still wasn't enough contextual information to figure out what was going on.

I decided to dump the command line arguments (a hack—I have to store the arguments in a global variable and make crashreport(), a library function, depend upon these global variables, which is bad form) and the environment variables (less of a hack—the system already has these in a global variable). With that, I was able to track down the issue.

The issue—some script kiddies are trying to hit the webpage that creates webpages. This isn't normally an issue, since the POST method requires authentication, but these script kiddies were trying to create entries via GET, and the error that was generating wasn't being properly checked (since that shouldn't happen—oops). And because of that, some unititialized variables were dereferenced and boom! A crash report.

The surprising thing about all this is that Apache not once reported the CGI program crashing. To me, that seems fairly obvious, but apparently not. So there's no telling how long this has been happening (yeah, you can tell I check the logs often, can't you?).

I do need to think about how to handle command line arguments in crashreport(). It's library code and thus, shouldn't rely on global variables. I have to think about this …

Saturday, January 12, 2013

A few notes about yesterday's crashes

I wrote crashreport() in an attempt to find out why glibc was reporting a double free (or memory corruption), so imagine my surprise when I found other crashes happening. I did find the root causes for the crashes yesterday, but I have yet to figure out why the memory corruption happened.

First off, no points to Apache for failing to report the unexpected termination of a child process. I can certainly understand that the Apache developers don't expect anyone to use CGI anymore, and if people do, to use a CGI developed in a scripting language that probably won't core dump. But still, they make the CGI module, and that the program the CGI module executes can be written in anything and hiding the fact that a program crashed due to SIGSEGV or SIGABRT is, to me, inexcusable.

Had Apache logged the crash, I probably would have found the error a few years ago (seriously). The actual crash only happened after the output was generated and sent to the browser, so I never saw anything unusual. And because Apache never said anything about a crash and well … everything is okay, right?

Second, the code path with the crash was in a seldom used code path—specifically, when the addentry.html page was requested. I normally use email to create entries, not the web interface. But it's not like I never use the web interface, but I can safely count on two hands the number of times I've used it over the past thirteen years.

So to say it doesn't get a lot of use is an understatement.

Now, are there features I don't use? Yes. And such code is currently commented out. That code was written at a time when I expected other people might use the codebase, but alas, only one other person ever used mod_blog (only to stop blogging due to personal reasons) and now, as far as I know, I'm the only one who uses this codebase. That doesn't bother me, but it does indicate that I should probably remove the code that I don't use.

But the web interface? I use it just enough to justify its existence in the codebase.

Third, the addtion of command line and evironment variables to the output of crashreport() (and I solved the global variable issues I had) certainly helped with the diagnosis. It revealed a request that would reliably crash the program (the aforementioned addentry.html page) and with a reliable way to crash the program, it's easy to isolate the buggy code (if a bit tedious).

And to tell the truth, the bug has existed since May 26^th, 2009, when I made the following commit:

Basically, I rewrote the core blogging engine over the past twelve hours. I still have yet to support adding new entries via the engine, but until I get that fixed, I can add them manually.

only I didn't quite update all the code properly. And since the code path in question isn't executed except when called as a CGI program (I should note that mod_blog can be run from the command line as well), and Apache never logs CGI programs that crash, no wonder I never saw this bug.

Sunday, January 13, 2013

One more final note about the crashes

So far so good—no more crashes since yesterday.

It's also interesting to know that there's a Reddit post on obtaining stack traces on excpetions under Windows, Linux and Mac OS-X. The approach taken is similar to my approach (which I “borrowed” from Redis) with the exception of using a separate signal stack—apparently, Mac OS-X won't point to the right stack when running a signal handler—nice to know.

So, hopefully ,there won't be any more cra—

ha ha, only ki

Monday, January 14, 2013

The significance of this is that you can build parsing expressions on the fly …

I found Meta II to be an interesting approach to parsing, and the closest modern equivilent to that are parsing expression grammars (PEGs), and the easiest one to use I've found is the Lua implementation LPeg.

What's interesting about LPeg is that it isn't compiled into Lua, but into a specialized parsing VM, which makes it quite fast. Maybe not as fast as lex and yacc but certain easier to understand and vastly easier to use.

Let me amend that: I find the re module to be easier to use (which is build on LPeg), as I find this:

local re = require "re"

parser = re.compile [[
	expr		<- term (termop term)*
	term		<- factor (factorop factor)*
	factor		<- number
			/  open expr close

	number		<- space '-'? [0-9]+ space	
	termop		<- space [+-] space
	factorop	<- space [*/] space
	open		<- space '(' space
	close		<- space ')' space
	space		<- ' '?
]]

to be way easier to read and understand than

local lpeg = require "lpeg"

local space    = lpeg.P" "^0
local close    = space * lpeg.P")" * space
local open     = space * lpeg.P"(" * space
local factorop = space * lpeg.S"*/" * space
local termop   = space * lpeg.S"+-" * space
local number   = space * lpeg.P"-"^-1 * lpeg.R"09"^1 * space

local factor , term , expr = lpeg.V"factor" , lpeg.V"term" , lpeg.V"expr"

parser = lpeg.P {
  "expr",
  factor = number
         + open * expr * close,
  term   = factor * (factorop * factor)^0,
  expr   = term   * (termop   * term)^0
}

As such, I've been concentrating on using the re module to brush up on my parsing skills to the point that I've been ignoring a key compent of LPeg—expressions!

Sure, raw LPeg isn't pretty, but as you can see from the above example, it is built up out of expressions. And that's a powerful abstraction right there.

For instance, in mod_blog, I have code that will parse text, converting certain sequences of characters like --- (three dashes) into an HTML entity &mcode;. So, I type the following:

``The name of our act is---The Aristocrats! ... Um ... hello?''

which is turned into

“The name of our act is—The Aristocrats! … Um …
hello?”

to be rendered on your screen as:

“The name of our act is—The Aristocrats! … Um … hello?”

Now, I only support a few character sequences (six) and that takes 160 lines of C code. Adding support for more is a daunting task, and one that I've been reluctant to take on. But in LPeg, the code looks like:

local lpeg  = require "lpeg"

local base =
{
  [ [[``]] ] = "“" ,
  [ [['']] ] = "”" ,
  [ "---"  ] = "—" ,
  [ "--"   ] = "–" ,
  [ "..."  ] = "…",
  [ ".."   ] = "‥" ,
}

function mktranslate(tab)
  local tab   = tab or {}
  local chars = lpeg.C(lpeg.P(1))
  
  for target,replacement in pairs(tab) do
    chars = lpeg.P(target) / replacement + chars
  end

  for target,replacement in pairs(base) do
    chars = lpeg.P(target) / replacement + chars
  end
  
  return lpeg.Ct(chars^0) / function(c) return table.concat(c) end
end

Now, I could do this with the re module:

local re   = require "re"
local R    = { concat = table.concat }
local G    = --[[ lpeg/re ]] [[

text    <- chars* -> {} -> concat

chars   <- '`'   -> '&ldquo;'
        /  "''"   -> '&rdquo;'
        /  '---'  -> '&mdash;'
        /  '--'   -> '&ndash;'
        /  '...'  -> '&helip;'
        /  '..'   -> '&#8229;'
        /  { . }

]]

filter = re.compile(G,R)

But the former allows me to pass in an additional table of translations to do in addition to the “standard set” programmed in, for example:

translate = mktranslate {
  ["RAM"]  = '<abbr title="Random Access Memory">RAM</abbr>',
  ["CPU"]  = '<abbr title="Central Processing Unit">CPU</abbr>',
  ["(tm)"] = '&trade;'
}

And I would want this why? Well, I have Lua embedded in mod_blog, so using Lua to do the translations is straightforward. But, now when I make an entry, I could include a table of custom translations for that entry. Doing it this way solves a problem I saw nearly a decade ago.

Tuesday, January 15, 2013

Linux vs. Windows. Two operating systems go in, only one comes out!

It quickly become apparent that Windows has no package management to speak of. I had to actually go directly to software vendors' websites and manually download, unpack, and install software. In some cases, the packages would come as a RAR file, and there was no unrar to extract the files with. The worst thing though, is that Widows wanted me to reboot it every time I installed something new.

I finally got my hardware to work after four hours of research, downloading, and rebooting. By that time, I was already feeling like a slave to the OS.

Via Hacker News, Branko's Thought Dump: On the state of Windows on the desktop

My friend Gregory is a Windows person who has had his share of problems with Linux. Well Gregory, here's the story of a Linux person who's had his share of problems with installing Windows.

Now, to avoid accusasions of being biased, I will also include the views of a Windows user on installing Linux:

When I installed Windows the process worked properly: The installer looked at my computer for a long time, then copied some files and rebooted. Then it had another long ponder before rebooting again. Then it booted into Windows, announced there were critical updates, and asked to boot again. Then once I installed my graphics drivers it wanted another boot. I guess my only complaint is that it stopped to ask about that fourth reboot. That was kind of odd. I mean, why would I say no?

On the other hand, the Linux installer feels like it?s missing parts or something. I ran the installer and it rebooted just once, right into Linux. After that I installed some updates, but the machine still didn?t ask to reboot itself. It just sat there like it was ready for me to start using it. I waited ten minutes just to be sure, but it never did ask to restart.

Eventually I had to bring in the laptop and reboot that, just so I could feel like the job was done properly.

Linux vs. Windows—Twenty Sided

See, I'm being fair here …

---

Leviathan

Before you could get a building permit, however, you had to be approved by the Zoning Authority. And Zoning—citing FEMA regulations—would force you to bring the house "up to code," which in many cases meant elevating the house by several feet. Now, elevating your house is very expensive and time consuming—not because of the actual raising, which takes just a day or two, but because of the required permits.

Kafka would have liked the zoning folks. There also is a limit on how high in the sky your house can be. That calculation seems to be a state secret, but it can easily happen that raising your house violates the height requirement. Which means that you can't raise the house that you must raise if you want to repair it. Got that?

Roger Kimball: This Metamorphosis Will Require a Permit

Somehow, I get the feeling that we've reached peak government.

Wednesday, January 16, 2013

Life at The Corporation

I've certainly peered deeper into the Abyss of XXX XXX's implementation of SS7 and come to a deeper understanding of the Lovecraftian nature of it.

A comment about my “goal” on my self-evaluation for The Corporation.

I'm finding myself in an odd position at The Corporation.

And it's not a bad thing at all.

I was hired to do performance measurements and testing of “Project: Wolowizard” and my first manager was R, who started (and still runs) the Ft. Lauderdale Office of The Corporation (the main office being in Seattle). My role fell into the “Quality Assurance” department and in 2011, I was transferred to the QA department, and my manager became E, who worked at Corporate Headquarters in Seattle (while I still work in the Ft. Lauderdale Office).

I ended up being the only QA employee in the Ft. Lauderdale Office. But more crucially, I'm the only QA employee doing QA on call processing (the part that happens between cell phones) in the entire company. So my weekly meeting (read: phone call) with the QA team would go something like:

E

So, random QA person, what have you been working on this week?

Random QA Person #1

[Long rambling technobabble about setting up unit tests for a smart phone application.]

E

Very good. Now, other random QA person, what have you been working on this week?

Random QA Person #2

[Even longer rambling technobabble about helping Random QA Person #1 set up unit tests, with a long digression about Ruby programming.]

E

Very good, very good. What about you, yet another random QA person?

Random QA Person #3

[Exposition about the difficulties in testing Random Java Technology for yet another smart phone application.]

Random QA Person #1

[Interjects into what Random QA Person #3 was saying about a possible fix for the difficulties in testing Random Java Technology.]

Random QA Person #3

Okay, I think I'll try that.

E

Wonderful. What about you, some other random QA person?

Random QA Person #4

[SEAN strains to hear Random QA Person #4 over the phone, but Random QA Person #4 mumbles so softly, SEAN cannot make out what Random QA Person #4 is saying, so SEAN spaces out for a few minutes.]

E

…Sean? Are you there?

Sean

[Realizes it's his turn to speak.] Oh yeah … um … this week I've [technobabble about running the latest “Project: Wolowizard” regression test and some of the difficulties in testing the Protocol Stack From Hell™.]

E

Okay … well then … see you all next week.

Nothing against E—E is a very nice person, but no one else on the QA team deals with call processing. Nor do they program in C, C++ or Lua. Java, Javascript and Ruby, yes. But the rest of the QA team in the Corporation deal with applications for cell phones and smart phones; I'm the only one that deals with call processing.

And because of that, it was felt by The Powers That Be™ that I would be better served under a manager in the Ft. Lauderdale Office (this was a few months ago). So now I'm working under J. J's a great guy, but he too, knows nothing about call processing (he's actually managing other, non-call processing cell phone related projects). Except for a meeting every week or two (or like today, for a performance review) I've just been plugging away at testing and staring into the Abyss of The Protocol Stack From Hell™.

It could be worse. I could have daily meetings like my fellow office mates.

(But it's not like I'm the only one there who understands call processing. There are three others, all at the Ft. Lauderdale Office. R, who is a VP in The Corporation and runs the Ft. Lauderdale Office; M, the lead developer who is now enmeshed in writing a smart phone client for “Project: Wolowizard,” and S, who does for Ops (call processing stuff) what I do for QA (call processing stuff) and is probably in a similar situation as me.)

Thursday, January 17, 2013

Texting from the other side of the table

My dad has often lamented the scene he sees whenever he goes out to a restaurant. It is inevitable that a nearby table of two or more diners will all be sitting in rapt silence, staring into small lit screens of smart phones and not engaging in conversation at all. Or rather, conversation with the fellow diners—for all he knows, they could be communicating with others, maybe at other restaurants.

It's amusing to note that this isn't a modern phenomenon.

Friday, January 18, 2013

Notes from a conversation outside a restaurant

As Bunny is getting out of the car, a kid walks up with a handful of leaflets. “Here,” he says, handing one to Bunny.

Bunny takes it and glances at it. From my vantage point, I can see that it's a leaflet for a psychic reading, presumably at some local establishment. Bunny takes one look at it, and hands it back. “No thank you. Not today,” she said. The kid takes it back, and wanders off.

“You know,” I said, “he should have seen that coming.”

Saturday, January 19, 2013

Trees are freaking awesome!

I did not know that trees create a negative pressure to force water up through their trunks (link via Hacker News commentary on a NewScientist blurb about the limits to tree growth.)

---

So, what date did October 23, 4004 BC fall on?

Who needs machine readable dates? As far as I can see there are two target audiences for this operation. The first is obviously social applications that have to work with dates, and where it can be useful to compare dates of two different events. An app must be able to see if two events fall on the same day and warn you if they do.

However, as a target audience social applications are immediately followed by historians (or historical, chronological applications). After all, historians are (dare I say it?) historically the most prolific users of dates, until they were upstaged by social applications.

…

Let’s go another eight hundred years back and land just in time to see Hannibal victorious against the Romans at Cannae. This historical battle, sources assure us, took place on 2 August 216 BC. We don’t have a prayer of re-mapping this date to a proleptic Gregorian or a Julian one.

The ancient Roman year had 355 days, and in theory every second year ought to have a so-called intercalary month of 22 or 23 days. The problem was that these months were inserted irregularly, and no chronologist ancient or modern has ever taken the trouble to track down the exact use of the intercalary month. (Besides, the sources are just not there.)

This means that we will never know exactly on which proleptic Gregorian date the battle of Cannae took place. The best we can say is that it took place in high summer; probably in July or August.

…

Before Dionysius introduced his reform, people used the old Roman system, in which every year was named after its two consuls.

After the Romans had discarded their monarchy in 509 BC they were forced to stop using regnal years. They needed a new naming system, and they decided to allow their two chief magistrates, the consuls, to give their names to the year.

Thus, “in the consulate of Cn. Pompeius Magnus and M. Licinius Crassus Dives” is a historically valid alternative to “70 BC.” In fact, BC or AD years may be considered a convenient shorthand for the “semantically” more correct consular years.

Although the consuls lost all political power after Augustus founded the Empire in 27 BC, the title was still given out to aristocrats who’d deserved a plum, as wel as to the Emperor himself, until the office was abolished in 541 AD. The consuls continued to give their names to the year. (In return they were graciously allowed to squander their fortunes on organising circus games.)

Via Hacker News, M aking <time> safe for historians

This is an amazing article (long, but well worth reading) about the difficulties historians have with time. Unfortunately, not only are calendars complicated, but even the concept of time is non-intuitive.

Sunday, January 20, 2013

Leviathan II

Finally, you have to guess what the law is. There is so much “discretion” [3,4] afforded to regulatory agencies that the threat of fines and seizures over bizarre interpretations of the law by a Carmen Ortiz- style ambitious regulator is never far from your mind. Example [5]:

[Newsweek:] What exactly would constitute a “medical claim?” Would pointing people to medical research papers [qualify]?

[FDA]: It depends. There are rules as to how one can do that … Those rules are actually worked out pretty well, and they just would need to make sure they’re staying within the rules.

[Newsweek:] Are those rules on the Web?

[FDA]: I don’t know where the policy is. I would have to get it for you. It’s an agencywide policy. I would have to find it for you. And it won’t be that easy for people to follow it …

…

Look, finally, how they claim in an official court filing against family farms producing raw milk that you have "No Generalized Right to Bodily and Physical Health" [12], where they approvingly cite the case of Cowan vs. US, where a terminal cancer patient was denied access to experimental medication, denied the right to opt-out of the FDA:

There is No Generalized Right to Bodily and Physical Health.

Plaintiffs' assertion of a “fundamental right to their own bodily and physical health, which includes what foods they do and do not choose to consume for themselves and their families” is similarly unavailing because plaintiffs do not have a fundamental right to obtain any food they wish. In addition, courts have consistently refused to extrapolate a generalized right to “bodily and physical health” from the Supreme Court's narrow substantive due process precedents regarding abortion, intimate relations, and the refusal of lifesaving medical treatment.

…

I know it sounds surreal, but they are arguing here that you only control your own body with respect to abortion, intimate relations, and euthanasia. Everything else is controlled by the FDA, yea even unto your death from cancer.

Comment at Hacker News

The whole comment should be read, if only to read the extensive footnotes provided in the comment (those numbers in brackets). But it does show just how nanny-esque (and not to mention risk-averse) our government has become.

I'm telling you—Leviathan, peak government.

Monday, January 21, 2013

One peculiar router …

I decided to poke around a bit with the home router, a Cisco WRVS4400N Wireless-N Gigabit Security Router and I must say, it's an odd router from a management point of view.

So I get a list of interfaces (via SNMP) on the device:

Interface Dump

Status	Interface	Bytes In	Bytes Out
up	lo	207454	207454
down	ipsec1	0	0
down	ipsec2	0	0
down	ipsec3	0	0
up	br0	304475952	467833506
up	ppp0	100525553	80535424
up	sit1	0	0
up	eth0	246917723	273606969
up	eth1	119391487	88802941
up	eth2	109882441	191904431
down	teql0	0	0
up	sit0	0	0
down	tunl0	0	0
down	gre0	0	0
up	ipsec0	0	0

Okay, lo is the local loopback device, I have no idea what br0 is, but I see eth0, eth1 and eth2 which are obviously the Ethernet ports in use. So, what do I have plugged in where? Okay, that's easy to determine—unplug a device, see which interface is marked as “down”. This is typical Cicso behavior, right?

Okay, I unplug the Mac from the network and I see:

Interface Dump

Status	Interface	Bytes In	Bytes Out
up	lo	207454	207454
down	ipsec1	0	0
down	ipsec2	0	0
down	ipsec3	0	0
up	br0	304488023	467841666
up	ppp0	100526059	80536312
up	sit1	0	0
up	eth0	246926120	273617295
up	eth1	119393091	88804157
up	eth2	109888649	191905881
down	teql0	0	0
up	sit0	0	0
down	tunl0	0	0
down	gre0	0	0
up	ipsec0	0	0

So. It's not going to mark one of the Ethernet ports as being down. Lovely. Looks like I'm going to have to do this the old fasioned way:

Physically trace each Ethernet cable. Okay, in the above image, the left-most cable goes out to the Intarwebs. The right-most cable (the blue one, in the port labeled “4”) goes to the Mac. The one to the left of the blue cable (in port “3”) goes to my Linux system. The one in port “2” goes to a computer that is currently turned off. And the one in port “1” (the second on the left) wraps around and is hanging in front of the shelves (I use that one for the laptop).

Wait a minute … let me look at the front again …

We have the Internet; Mac is at gigabit speed; my Linux system, and … the turned off system? Really?

Okay, it appears that the Ethernet card in the currently off system receives just enough power to maintain a connection status; there's probably a “wake-on-LAN” feature on its Ethernet card.

Okay, now that's that's straightened out … um …

eth0, eth1 and eth2 …

Um …

The external ports don't match up with the internal ports. And given that there's a computer that is turned off, shouldn't one of the Ethernet ports return no data? And where's eth3?

What exactly is going on?

Okay, put that aside for now. What's the routing table look like?

Routing Table

Dest	Mask	NextHop	Proto	Metric	Age	Interface
0.0.0.0	0.0.0.0	70.XXXXXXXXXX	local	0	0	ppp0
127.0.0.0	255.255.0.0	0.0.0.0	local	0	0	sit1
192.168.1.0	255.255.255.0	0.0.0.0	local	0	0	br0
239.0.0.0	255.0.0.0	0.0.0.0	local	0	0	br0
70.XXXXXXXXXX	255.255.255.255	0.0.0.0	local	0	0	ppp0

Um, 70.XXXXXXXXXX isn't my IP address; it's 74.XXXXXXXXX (I pay extra for a static IP address because of work issues). Okay, it appears that 70.XXXXXXXXXX is the remote side of my connection, but that routing (while it works) just looks odd to me. So, what IP addresses are assigned to which interfaces?

Interfaces

Destination	Mask	Interface
127.0.0.1	255.0.0.0	lo
192.168.1.1	255.255.255.0	br0
74.XXXXXXXXX	255.255.255.255	ipsec0

Okay, there's my IP address, but it's … ipsec0? Weird. And it seems that br0 is a grouping of all the Ethernet ports.

But really, the mislableled Ethernet ports, the turned off computer sending and receiving traffic, it just has me skeeved out a bit. And the whole mess makes it difficult to monitor the network (not that I need to monitor my network, but Cisco is selling this as a “Small Business” device and a “Small Business” might want to monitor its network).

Oh, I just thought of something … the wireless interface—it's missing! I mean, it's missing in the interface list; physically it's there or Bunny wouldn't be able to use her laptop on the Intarwebs.

This is one strange router …

Tuesday, January 22, 2013

More on that peculiar router of mine …

From

Mark Grosberg <XXXXXXXXXXXXXXXXX>

To

spc@conman.org

Subject

Router

Date

Tue, 22 Jan 2013 11:10:49 -0500 (EST)

To explain your router: You have three actual hardware Ethernet MACs:

• The WAN (probably eth0)
• Internal (probably eth1)
• WIFI (probably eth2)

The br0 interface is a software bridging between eth1 and eth2. This way your wireless computer is on the same network as your wired computers. But it's not a real physical device but represents packets from either eth1 or eth2 to the IP stack. The multiplexing is done in the driver for that interface.

As for the IP address discrepancy: eth0 does not actually act as your Internet. That's not how DSL from XXXXXXXX works. eth0 uses managed IP's to bring up a PPPoE session. So ppp0 will likely have your public IP address. Those packets get tunnled within the other set of IP's. That's a guess since I don't have XXXXXXXXX DSL though.

The thing with software, even embedded software these days is that there are so many layers of existing crap (that don't need to be there but skittish managers are loathe to remove things they don't grok) it's amazingly difficult to grasp the whole picture since there is often little rationale behind it.

That actually explains the behavior I'm seeing on the router. So I can monitor the Intarweb traffic on eth0, overall LAN traffic on eth1 and the WiFi network via eth2.

Okay then.

---

Cellphone guts for the curious …

M thought I might find the following email he sent to some of his fellow cow-orkers interesting (and I did). He also gave me permission to post it here.

So, if you were ever interested in the internal processing guts of a cell phone, well, here you go …

From

MXXXXXXXXXXXX <XXXXXXXXXXXXXXXXXXXXX>

To

XXXXXXXX XXXX XXXXXXXXXXXXXXXXXXXXXX

Subject

XXXXXXXXXX firmware guts for the curious …

Date

Sun, 20 Jan 2013 19:07:00 -0500

So I found myself poking around inside the firmware of my personal cell phone, an HTC EVO 3D. This phone is based on the XXXXXXXXXXXXXXXXXXX chipset. In investigating something about how my phone worked I decided to disassemble the cellular firmware on it and see how some of the EVDO stuff was done. I found some interesting things for those of you who are curious as to how some of these things work beyond the Linux side of Android. If you aren't then feel free to skip this e-mail.

A cell phone running something like Android had a complete separation between the application processor (the CPU running Linux, your apps, the GUI, all that stuff) and the cell phone processor (runs the mobile data stack, voice codecs, speakerphone echo cancellation and all other manner of hard real-time stuff). The (somewhat incorrect) terms of “application processor” and “baseband processor” were assigned to these two CPUs, respectively. So why the separation in the first place? Well Linux isn't very good at things like voice or real-time. Running a cellular radio is quite a bit of real time code. And all the speech pathways of the actual “phone” part are also a bit much for Linux to deal with. Linux, as with much “desktop” style software out there gives little thought to things like interrupt latency—making it a terrible OS to do anything hardware. Plus it makes sense that when smartphones first started popping up everywhere that you had a bunch of grizzly EE guys doing the RF stuff who didn't want to mess with Linux and all that “high level crap.” (And yes, I fall squarely in that camp). So the dual-CPU approach keeps the different types of engineers from getting in each others' way.

I used to think that the baseband firmware was pretty traditional, monolithic piece of embedded software. Turns out I was dead wrong. I can only guess at the reasons for their architecture since I have no real contacts at XXXXXXXX. But the firmware is actually quite a hodgepodge of things. Those of you who fancy yourselves software archaeologists may find this informative.

The software behind the baseband CPU is quite interesting and totally undocumented. But I dug in and found a fascinating world. First everything in the XXXXXXXXXX baseband runs under a hypervisor—which they call a “microvisor.” For those of you unfamiliar with what this means it's a bit like running VMware on a bare computer and then running Linux and Windows simultaneously. Only in this case the hypervisor is a commercial port of the L4 microkernel. The hypervisor divides the processor into “cells” that each run their own distinctive world. Most of the cellular modem software runs under an OS called Iguana. Iguana wraps an older RTOS used by XXXXXXXX called REX that now runs inside an L4 task and makes it a message-based server. Those of you who worked with me at XXXXXXX may remember how we ran [one operating system] on top of [another operating system] on our XXXXXXXXXX XXXXX PBX—this is kind of the same thing. Only the ARM architecture is far saner to deal with than x86 when it comes to virtualization.

The cellular software running under these layers is called XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX—and it's a quite typical monolithic embedded blob of software. But it's had a few interfaces shoved in it and now communicates with the rest of the firmware through a message based interface.

So why did they go to all the trouble to run their embedded firmware under L4? Well there is more running on the baseband processor than you may think. It's grown in complexity and now contains several large pieces of disjoint software, so the microkernel approach starts to make sense when you consider how much legacy code they have. Poking around I found out the XXXXXXXXXX firmware also uses pForth. Mostly it looks likes for remote diagnostics the cellular carrier can send blobs of FORTH code right to the radio. The radio firmware also seems to have an IP stack (with TCP) so it can do its own interesting things (both bad and good). This TCP stack is what talks to the cellular network. The rmnet0 interface actually goes over shared memory and talks to this IP stack (which is where PPP is actually running).

The GPS functionality of your phone is also quite a huge pile of code and again the Linux side of things sees a very simple interface where coordinates are fed into the application processor. The huge amount of code necessary to do the actual positioning (and the associated WAAS & aGPS stuff) also runs as a task under L4. There are also a pile of DSP's running their own kernel that communicate with the L4 kernel for management purposes.

I think XXXXXXXX is planning on eventually removing the distinct baseband processor—maybe in their low-cost chipsets. You can run Linux under L4. This means the entire Android application processor could in theory be virtualized and run alongside the baseband software on a single core. I could see some advantages to this and I wonder if XXXXXXXX is working on (or implemented) this. A quick peek at the various device drivers in the open source kernels for the XXXXXXXXXX shows the drivers just fill in buffers and queue messages in a very high level manner compared to a typical device driver. Since the hardware is actually managed by stuff running under Iguana. Since this is just a message interface it wouldn't be terribly difficult to move everything to a single core design.

Wednesday, January 23, 2013

dew umopapisdn

I pop open Google Maps, hit the arrow and I find myself contemplating a new outlook on Lower Sheol:

At first, I thought Google decided to put an end to the Eurocentralism of world maps and force a new perspective on us, but then I realized that earlier, I had simply rotated the map to face the direction I was travelling.

Ah well …

Thursday, January 24, 2013

But if anyone could have faked it, Stanley Kubrick is a good choice

S. G. Collins' take on the conspriacy that the Apollo moon landing was faked (link via Jason Kottke) is an interesting perspective on it. No, Mr. Collins doesn't think the moon landings were fake, but his argument for it being real is one I've never seen presented.

I think it's worth watching.

Friday, January 25, 2013

So charactures are drawn with peripheral vision

You have to try this (link from my friend Hoade). I've done it multiple times and each time, it just amazes me.

Human vision is weird sometimes.

---

Even more optical illusions

And speaking of optical illusions, Wlofie sent along a link to Impossible World, a collection of optical illusions and impossible images.

Saturday, January 26, 2013

Playing around with QR codes

QR codes have been around for quite some time and I've been meaning to play around with them, but lacked the software to generate them. Just recently though, I came across QArt Coder, a web based application to generate QR codes.

I slapped in http://boston.conman.org/ and the result was amazing:

Who'd have thought the QR code for my own site would look that way?

Sunday, January 27, 2013

In the old days, this was probably much harder

From a link on FaceGoogleBookPlus is this video of a mind reader revealing his tricks and let me tell you, it's quite revealing, not only in how it was done, but just how much information was available.

---

I have to wonder how my college friend Rebel One deals with this crap?

Over the past week, a number of users of the popular photo sharing app Instagram and parent company Facebook have been locked out of their accounts and prompted by both services to upload images of their government issued photo IDs to regain access, as CNET first reported on Tuesday.

Via Hacker News, Instagram Asking For Your Government Issued Photo IDs Now, Too.

I'm not sure how I feel about this. I mean, there was the Google Plus real name controversy a year and a half ago and that didn't go over so well and now we have FaceBook (who owns Instagram) possibly, maybe, pushing rather silently, a “real name” policy for its site.

On the one hand, it's a company, and a company can run its site as it sees fit, and we, as potential users, can use or not use that company as we see fit. But the whole FaceGoogleBookPlus social space does seem to be taking over the Internet these past few years and it's my fear that the only way to use the Intarwebs will be through one of these large social networking sites.

I think overall, I find it spooky.

(Oh, and Rebel One? That's his real name. Just look on his driver's license. Okay, okay, it's not really Rebel One—that's just a nickname for his real name, The Rebellious One.)

Monday, January 28, 2013

Notes from a weekly gathering of food trucks in Hollywood, Florida

Bunny, Gregory and Amy have wandered off to get some more food at the Food Trucks @ Young Circle; I'm standing guard (literally—there are no chairs) at the table (no chairs) we've been using.

A gentleman, alone, walks up and stops at the table. In one hand he's holding a small empty cup, in his other hand he sets down a large chocolate shake. He then grabs the straw sticking out of his shake, covers the top with a finger, and proceeds to transfer a sample of his shake to the small cup.

At no point does he acknowledge my existence (and to be fair, I didn't acknowledge back). He just intently transfers some of the chocolate shake from one container to the other via the straw.

After transfering some three or four strawsful, he picks up both containers, and walks off.

---

Antagonistic phrases

I titled the previous post as “Notes from a bimonthly gathering of food trucks in Hollywood, Florida.” I noted that the Food Trucks @ Young Circle were bimonthly from this sentence:

Starting tonight and every other Monday …

Food Trucks at Young Circle Every Monday, Starting Tonight (and yes, I didn't notice the huge headline at the top of the article)

To me, the phrase “every other Monday” translates to “this Monday, not this Monday, this Monday, not this Monday”—in other words, you skip a Monday.

Bunny informed me that no, it does not mean that; as phrased it meant “this Monday, and all the Mondays after that”—all other Mondays.

Aaaaaaah!

Biweekly—is that “twice a week” or “every two weeks?”

Even the word I used, “bimonthly”, can appear to mean two different things.

Man, I hate antagonistic phrases …

Tuesday, January 29, 2013

Cellphone guts for the curious, Part II

My friend Brian forwarded this comment about cellphone guts to me (the lateness is due to getting permission to post this email from a third party):

From

Brian Yoder <XXXXXXXXXXXXXXXXXXXXXXX>

To

Sean Conner <sean@conman.org>

Subject

Fwd: Cellphone guts for the curious … - The Boston Diaries - Captain Napalm

Date

Wed, 23 Jan 2013 17:49:23 -0500

Sean,

I passed along your blog post to my friend Chip, and here is his response.

Regards,
Brian

Begin forwarded message:

From

XXXXX XXXXX

To

Brian Yoder <XXXXXXXXXXXXXXXXXXXXXXX>

Subject

Re: Cellphone guts for the curious … - The Boston Diaries - Captain Napalm

Date

January 23, 2013, 1:50:52 AM EST

They must have done a very good virtualization work, because FCC regulations specify that the computer that runs the transmit recieve be separate from the rest of the phone and with a ROM.

It drove cell phone manufactures nuts because of cost, but the FCC did want any one to hack in to phone and change the radio software.

Ah … politics trumps technology …

---

I'm terribly upset that GCC didn't start NetHack

Ah yes, undefined behavior of C. It's easy to see in retrospect, but it's still a bit surprising. The code:

#include <limits.h>

int foo(int a,int b)
{
  return a / b;
}

int main(void)
{
  return foo(INT_MIN,-1);
}

And when you compile with the right options …

[spc]lucy:/tmp>gcc -std=c99 -O0 crash.c
[spc]lucy:/tmp>./a.out
Floating point exception (core dumped)
[spc]lucy:/tmp>

What's actually going on here? Well, I compiled the code with crashreport(), so I could capture the crash:

CRASH(9372/000): pid=9372 signal='Floating point exception'
CRASH(9372/001): reason='Integer divide-by-zero'
CRASH(9372/002): pc=0x804883d
CRASH(9372/003): CS=0073 DS=007B ES=007B FS=0000 GS=0033
CRASH(9372/004): EIP=0804883D EFL=00010296 ESP=BFFC370C EBP=BFFC3710 ESI=BFFC37C4 EDI=BFFC3750
CRASH(9372/005): EAX=80000000 EBX=00CBBFF4 ECX=BFFC371C EDX=FFFFFFFF
CRASH(9372/006): UESP=BFFC370C TRAPNO=00000000 ERR=00000000
CRASH(9372/007): STACK DUMP
CRASH(9372/008):        BFFC370C:                                     1C 37 FC BF 
CRASH(9372/009):        BFFC3710: 38 37 FC BF 7C 88 04 08 00 00 00 80 FF FF FF FF 
CRASH(9372/010):        BFFC3720: F4 BF CB 00 F4 BF CB 00 F8 A9 04 08 F4 BF CB 00 
CRASH(9372/011):        BFFC3730: 00 00 00 00 A0 CC B8 00 98 37 FC BF 93 4E BA 00 
CRASH(9372/012):        BFFC3740: 01 00 00 00 C4 37 FC BF CC 37 FC BF 26 22 B8 00 
CRASH(9372/013):        BFFC3750: F4 BF CB 00 00 00 00 00 50 37 FC BF 98 37 FC BF 
CRASH(9372/014):        BFFC3760: 40 37 FC BF 55 4E BA 00 00 00 00 00 00 00 00 00 
CRASH(9372/015):        BFFC3770: 00 00 00 00 D4 CF B8 00 01 00 00 00 80 87 04 08 
CRASH(9372/016):        BFFC3780: 00 00 00 00 60 21 B8 00 B0 2C B8 00 D4 CF B8 00 
CRASH(9372/017):        BFFC3790: 01 00 00 00 80 87 04 08 00 00 00 00 A1 87 04 08 
CRASH(9372/018):        BFFC37A0: 47 88 04 08 01 00 00 00 C4 37 FC BF CC 92 04 08 
CRASH(9372/019):        BFFC37B0: 20 93 04 08 B0 2C B8 00 BC 37 FC BF 92 9A B8 00 
CRASH(9372/020):        BFFC37C0: 01 00 00 00 BB A9 FF BF 00 00 00 00 C3 A9 FF BF 
CRASH(9372/021):        BFFC37D0: E4 A9 FF BF F4 A9 FF BF FF A9 FF BF 0D AA FF BF 
CRASH(9372/022):        BFFC37E0: 34 AA FF BF 51 AA FF BF 79 AA FF BF 95 AA FF BF 
CRASH(9372/023):        BFFC37F0: A7 AA FF BF B8 AA FF BF CE AA FF BF EC AA FF BF 
CRASH(9372/024):        BFFC3800: F5 AA FF BF 04 AB FF BF C7 AC FF BF             
CRASH(9372/025): STACK TRACE
CRASH(9372/026):        ./a.out[0x804889c]
CRASH(9372/027):        ./a.out[0x8049078]
CRASH(9372/028):        /lib/tls/libc.so.6[0xbb79b0]
CRASH(9372/029):        ./a.out[0x804887c]
CRASH(9372/030):        /lib/tls/libc.so.6(__libc_start_main+0xd3)[0xba4e93]
CRASH(9372/031):        ./a.out[0x80487a1]
CRASH(9372/032): DONE

And from there, we can load up the program and do some disassembly:

[spc]lucy:/tmp>gdb a.out
GNU gdb Red Hat Linux (6.3.0.0-1.132.EL4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"…Using host libthread_db
library "/lib/tls/libthread_db.so.1".

(gdb) disassemble 0x804883d
Dump of assembler code for function foo:
0x08048828 <foo+0>:     push   %ebp
0x08048829 <foo+1>:     mov    %esp,%ebp
0x0804882b <foo+3>:     sub    $0x4,%esp
0x0804882e <foo+6>:     mov    0x8(%ebp),%edx
0x08048831 <foo+9>:     lea    0xc(%ebp),%eax
0x08048834 <foo+12>:    mov    %eax,0xfffffffc(%ebp)
0x08048837 <foo+15>:    mov    %edx,%eax
0x08048839 <foo+17>:    mov    0xfffffffc(%ebp),%ecx
0x0804883c <foo+20>:    cltd   
0x0804883d <foo+21>:    idivl  (%ecx)
0x0804883f <foo+23>:    mov    %eax,0xfffffffc(%ebp)
0x08048842 <foo+26>:    mov    0xfffffffc(%ebp),%eax
0x08048845 <foo+29>:    leave  
0x08048846 <foo+30>:    ret    
End of assembler dump.
(gdb) 

It faulted on the IDIV instruction, but it wasn't technically an “integer division-by-zero.” The Intel 80386 (and the Pentium™ in my computer is little more than a glorified Intel 80386) book I have describes IDIV as:

An 80386 interrupt zero (0) [which is reported as an “Integer division-by-zero”] is taken if a zero divisor or a quotient too large for the destination register is generated. [emphasis added]

Now, EAX is -2,147,483,648 (80000000 in hexadecimal notation, which can be represented in 32-bits (we're running 32-bit code here—the issue still happens on 64-bit systems but the value will be vastly larger), but -2,147,483,648 divided by -1 should be 2,147,483,648, but 2,147,483,648 cannot be respresented in 32-bits [Technically, the value can be represented in 32 bits, but the instruction in question, IDIV is a signed instruction, and because of the way Intel does signed integer math, the signed quantity 2,147,483,648 cannot be represented as a 32-bit signed quantity in 32-bits. —Editor] and thus, because the quotient is then considered “too large” we get the fault which ends the program.

This is fine as far as C goes, because C says such behavior is “undefined” and thus, anything goes.

Simple once you know what's going on.

(And for the 99% of my readership who don't get the NetHack reference in the title … )

Wednesday, January 30, 2013

… and a scientist does science!

It is no longer enough to simply ask someone if they are a programmer. Saying a programmer writes programs is like saying a scientist does science. The difference is that botanists don't design nuclear reactors.

Via Reddit, Erik McClure: "Programmer" is an Overgeneralization

It took me some time to convince my dad that, while I am a programmer, I do not use, nor understand, Microsoft Windows; there is more to programming than just The Redmond Monopolist Operating System. The above article goes into just a few different areas that all fall under “programming.”

Even system administration (the job of keeping computers fed and happy) has disciplines—you have those that run Microsoft Windows (and even there, you have several different flavors of Windows to manage) and those that run Unix (and here you can even specialize even further—Solaris, BSD, any number of Linux distributions).

And even if you know how to run Linux, there's a quantitative difference between running a small network (say, less than 100 machines) and something on the scale of Google (100,000 machines? 500,000 machines? A million?).

(If you are curious, I'm clearing out the backlog here … )

Thursday, January 31, 2013

Parsers vs. regular expressions? No contest

I'm finding that where once in Lua to its regular expressions for parsing, I am now turning to LPeg—or rather, the re module, as I find it easier to understand the code once written.

For instance, the regression test program I wrote for work outputs the results of each test:

1.a.8 0-0 16-17 scp:  ASREQ (1) ASRESP (1) LNPHIT (1) SS7COMP (1) SS7XACT (1) tps:  ack-cmpl (1) cache-searches (1) cache-updates (1) termreq (1)

Briefly, the first three fields are the test case ID, and indications if certain data files changed. The scp field indicates which variables of the SCP (you can think of this as a service on a phone switch) were modified (these just happen to be in uppercase) and then the tps field indicates which TPS (our lead developer does have a sense of humor) were modified. But if a variable is added (or removed—it happens), the order can change and it makes checking the results against the expected results a bit of a challenge.

The result is some code to parse the output and check against the expected results. And for that, I find using the re module for parsing:

local re   = require "re"

G = [[
line		<- entry -> {}
entry		<- {:id: id :} 			 %s
		   {:seriala: serial :}	         %s
		   {:serialb: serial :}	         %s
		   'scp:' {:scp: items* -> {} :} %s
		   'tps:' {:tps: items* -> {} :}
id		<- %d+ '.' [a-z] '.' %d+
serial		<- %d+ '-' %d+
items		<- %s* { ([0-9A-Za-z] / '-')+ %s '(' %d+ ')' }

]]

parser = re.compile(G)

to be more understandable than using Lua-based regular expressions:

function parse(line)
  local res = {}
  
  local id,seriala,serialb,tscp,ttps = line:match("^(%S+)%s+(%S+)%s+(%S+)%s+scp%:%s+(.*)tps%:%s+(.*)")
  
  res.id      = id
  res.seriala = seriala
  res.serialb = serialb
  
  res.scp = {}
  res.tps = {}
  
  for item in tscp:gmatch("%s*(%S+%s%(%d+%))%s*") do
    res.scp[#res.scp + 1] = item
  end
  
  for item in ttps:gmatch("%s*(%S+%s%(%d+%))%s*") do
    res.tps[#res.tps + 1] = item
  end
  return res
end

with both returning the same results:

{
  scp =
  {
    [1] = "ASREQ (1)",
    [2] = "ASRESP (1)",
    [3] = "LNPHIT (1)",
    [4] = "SS7COMP (1)",
    [5] = "SS7XACT (1)",
  },
  id = "1.a.8",
  tps =
  {
    [1] = "ack-cmpl (1)",
    [2] = "cache-searches (1)",
    [3] = "cache-updates (1)",
    [4] = "termreq (1)",
  },
  serialb = "16-17",
  seriala = "0-0",
}

Personally, I find regular expressions to be an incomprehensible mess of random punctuation and letters, whereas the re module at least lets me label the parts of the text I'm parsing. I also find it easier to see what is happening six months later if I have to revisit the code.

Even more importantly, this is a real parser. Would you ranther debug a regular expression for just validating an email address or a grammar that validates all defined email headers (email address validation starts at line 464)?

Friday, February 01, 2013

Leviathan III

Why are the Nordic countries doing this? The obvious answer is that they have reached the limits of big government. “The welfare state we have is excellent in most ways,” says Gunnar Viby Mogensen, a Danish historian. “We only have this little problem. We can’t afford it.” The economic storms that shook all the Nordic countries in the early 1990s provided a foretaste of what would happen if they failed to get their affairs in order.

Via Hacker News, Norther lights | The Economist

You know … if the Nordic countries are saying they've reached peak government, do you think we might actually have reached peak government?

Saturday, February 02, 2013

Groundhog Day

Ah, Groundhog Day, the day when a large squirrel pops up and if it sees its shadow, we'll have six more weeks of winter, or, if you are Phil Connors, 34 years of purgatory.

Sunday, February 03, 2013

Head in the clouds

Shocked by the way intuition had abandoned him, Crane began to ask questions. For years he got no intelligent answers. Veterans of the military and the airmail service still insisted they could fly "by the seat of the pants," and they thought less of those who could not. Their self- deception now seems all the more profound because the solution to the problem of flying in clouds and darkness—a gyroscope adapted to flying—was already widely available.

…

One of the earliest cloud flights with a turn indicator was made by William Ocker, an Army pilot, in 1918. Though he, too, spiraled out of overcast, he concluded correctly that his mistake had been to favor sensation over the instrument's indications. During the 1920s a few Post Office pilots began to fly by instruments. When Charles Lindbergh crossed the Atlantic, in 1927, a turn indicator kept him from spiraling into the sea when he met fog. Two years later Jimmy Doolittle made a "blind" landing, after flying a complete circuit around an airport in a special biplane modified with a domed cockpit from which he could not see outside. … More significant were the special devices that made the precisely flown circuit possible. The airplane was equipped with navigational radios, an airspeed indicator, an improved altimeter, a turn indicator, and two new gyroscopic instruments from Elmer Sperry—a gyroscopic compass and an artificial horizon. This combination was so effective that it still forms the core of instrument panels today. Doolittle compared the artificial horizon to cutting a porthole through the fog to look at the real horizon. Devising technology was the easy part. The more stubborn problem of belief remained. As late as 1930 one of the airlines wrote to Sperry complaining about a mysterious problem: the instruments worked fine in clear air, but as soon as they were taken into clouds, they began to indicate turns.

The Turn 93.12

It's a rare piece of writing that causes an intense bout of self- introspection, but given my past views on the whole “crutch vs. tools” debate, this might be the one thing to possibly change my views.

Or at lest temper them a bit.

My problem with technology like GPS has been the blind trust in the technology, but here is an article about people that have to have blind trust in the technology when in the clouds least they die a horrible and (I assume) painful death. It's also about how our “intuition” can lead us astray. Granted, we don't have thousands of years of expience to guide evolution in honing our “intuition” for flying, which is why our “intuition” leads us astray when flying, and thus, we have to augment our “intuition,” as well as our senses, with tools.

So we can fly.

Safely.

When we can't see.

Because our heads are in the clouds.

Hmm.

Monday, February 04, 2013

What happened to the great Super Bowl ads?

I must say that the this year's Super Bowl ads were not all that great. Not a single one was all that memorable, unlike, say, 1984, or herding cats.

Tuesday, February 05, 2013

Just wait until I reveal the truth behind the bees

Don't you think it's weird that our children come home from the school cafeteria caring more about poverty than about the real hazards that plague their daily lives? In this case, poverty is, quite literally, the “opiate” of the masses.

Via Hacker News, Verified Facts About Poverty

The people who know me well know that I tend to enjoy the occasional conspriacy theory or two (or twenty- three) and the Verified Facts website is a hoot!

Yes, the “facts” there are made up (probably based off a Marko v Chain) but I still find the results entertaining. I mean, how can I not love this?

Do you know about the shocking connection between The Boston Diaries and fluoride? No? Well, your innocence is about to be destroyed.

Gun control advocates use fluoride at rates almost 300% higher than other citizens.

Verifie d Facts About The Boston Diaries and Flouride

---

I don't bother with news—I figure, if it's important enough, it'll filter through, and apparently, it does to a rather frightening degree

I don't know if I should be proud of answering all the questions correctly, or horrified that I, someone who does not watch the news nor read any newspapers (except for a very local newspaper from a town I don't even live in) managed to outscore 92% of other people who took the test.

Wednesday, February 06, 2013

Notes from an overheard conversation at some ungodly hour of the morning (as if there were such a thing as a godly hour)

“… ate!”

“Ug … um … I haven't eaten anything yet!”

“No, I said you are going to be late!”

“Um … really?”

“Your alarm clock had been alarming for half an hour.”

“It was?”

“Yes.”

“Oh … so it was.”

“You need to get up!”

“Who knew the iPhone alarm would give up after half an hour?”

“You are going to be late!”

“Ug … five more minutes?”

“UP!”

Thursday, February 07, 2013

1997 called. They want your F**K YOU! USE THIS BROWSER! site back.

A friend (whom I'm not naming because he had no idea this would happen and I'd rather not drag him into this) sent me this link. I'd like to read the article, but no—what do I get?

What? No link to the article anyway? Just a large “Go away or we shall taunt you a second time?”

Fine.

But that doesn't mean I won't taunt you a second time.

Friday, February 08, 2013

Sailing the Corporate Seas, Part II

How many people know how the “Disney system” of comics works? When I describe this to some fans when asked about it, they often think I'm kidding them or lying. Or they are outraged. But it's an unfortunate fact that there have never been, and I ultimately realized there never will be, any royalties paid to the people who write or draw or otherwise create all the Disney comics you've ever read. We are paid a flat rate per page by one publisher for whom we work directly. After that, no matter how many times that story is used by other Disney publishers around the world, no matter how many times the story is reprinted in other comics, album series, hardback books, special editions, etc., etc., no matter how well it sells, we never receive another cent for having created that work. That's the system Carl Barks worked in and it's the same system operating today.

How can such an archaic system still be in operation in the 21st Century when royalties have been paid in other creative publishing endeavors for literally centuries? All book authors, musicians, actors, singers, non- Disney cartoonists, even people who act in TV commercials … they all receive royalties if success warrants it. Even Disney pays normal royalties to creators and performers in its own movie and TV and book and music businesses. As near as I can tell, correct me if I'm wrong, but it's only the creators of Disney comics who have no chance to receive a share of the profits of the success of the work they create. And yet Disney comics have never been produced by the Disney company, but have always been created by freelance writers and artists working for licensed independent publishers, like Carl Barks working for Dell Comics, me working for Egmont, and hundreds of others working for numerous other Disney licensees.

Why is this? I don't know.

Via Hacker News, Don Rosa Collection | An Epilogue by Don Rosa

My very first post was about the importance of creator owned IP. This is more of the same, although it's from the other Good Duck Artist™ (the Good Duck Artist™ being Carl Barks).

It's sad that this still happens to this day.

It's also sad that artists still sign their lives away like this too.

Saturday, February 09, 2013

Wonderful pulp fiction magazines from a past future that never happened

A subset of my friends will find this incredibly cool:

That same subset will also enjoy making their own pulp magazine covers.

Sunday, February 10, 2013

I've been doing this for thirteen years

As I was reading about the expression problem, I couldn't help but think it sounded very familiar. And then it hit me, I wrote about this nine years ago.

This might be another reason why I took some time off—I've already written about a lot of stuff.

Twice this year, I've wanted to write about some topic, only to find I've already written about said topic. For instance, there's this silly series about solving sudoku problems that I've written about before (seriously—I thought this was satire at first, until I realized that this was a serious article by Ron Jeffries, who is a consultant selling this programming methodology).

The second came about because of the alarm clock incident where I was reminded of this weird dream I once had.

It's sometimes hard to come up with new material.

Monday, February 11, 2013

640G … we're up to 640G, right? That should finally be enough, right?

What I am suggesting is that disks came about because of limited RAM. Now that RAM limitations can be of increasing greater size, we should explore new freedoms. What follows may seem a little far-fetched, but may also be just around the corner.

First, we may take it that a one megabyte RAM is not likely to be filled with a BASIC or machine code program of anything near that length. The debugging alone would take too long! This leaves us with other possibilities.

We could fill a lot of the RAM with a wide range of programs, and call up any of the whole suite, instantaneously, from a special menu program.

We could have as many programming aids in our machine as we could conceivably wish for, and barely scratch the surface of our new-found capacity.

We could have a vast range of help screens available for instantaneous recall when in trouble.

We could call in a whole succession of high resolution pictures, which are usually slow to load from disk, so rapidly that even animation would be possible.

We could have split processing in one machine. After all, it is common for two processors to be in one machine, so why not a schizoid machine with each part operating independently?

We could have a really enormous amount of text in our word processor at any one time, and have many different text areas. Our word processor could perhaps interact with our accounting and data base programs in RAM.

Accounting suites of programs could be truly integrated, so that final accounts are updated after every transaction.

Via Flutterby, Guest Commentary: Is RAM Memory A Status Symbol?

Hmm … sounds a lot like Microsoft Windows.

Or a smart phone for that matter.

I should note that this was written in 1983, when 512K went for $550.00 (cheapest price I could find— today that would be anywhere from $1,080.00 to $2,350.00, depending on how you calculate inflation) and there were no personal computers that could hold more than a megabyte of memory (which would shortly change over the next few years).

Oh, and this comment from Flutterby is priceless:

With modern machines having gigabytes of RAM, one can only assume that debugging has been completely abandoned … and assumption that gets validated every time Scrabble crashes on my Android phone.

Comm ent

Tuesday, February 12, 2013

Head in the clouds

Okay, I received the following email from a friend of mine (not saying who, as I don't want to drag him into this) about cloud storage. It was a bit odd, in that it was addressed to “undisclosed-recipients” but even if he did send this with good intentions, well … given that he knows my views on this, I wonder why he included me in the email.

I'm not given to sending mass-emails, so when I send one out, it's a biggie. And this is a biggie.

I'm a huge proponent of cloud-based storage for convenience and backup. And, joy of joys, an online cloud storage provider, box.com, has teamed up with Dell to give away the deal of the century: 50G of online storage—for life!

As near as I can tell, there's no catch.*

Plus, you can access your files from anywhere: From you Windows PC or Mac, from your iPhone, or iPad, or from your Android phone or tablet. There are sync clients available for all the major platforms—even FTP. I use Box.com myself. It's really easy to use.

This is a tremendously good thing to have, if for no other reason than to store your important pictures. So many times over the course of my career have people come to me with dead hard drives and said: “Please, I don't care about anything else—just save my pictures.” I can tell you from personal experience that the very safest place to store copies of your important files is the cloud. With Box, you will never, ever have to worry about losing your important files again. Plus, you can access your files from anywhere, and collaborate as well.

Click the link below to get started:

https://www.box.com/signup/o/dell_50gb_give_get

* Uh-oh, the dreaded asterisk: There actually is a small catch—the size of any one individual file in your box.com account cannot exceed 250 megabytes (250 megabytes = 250,000,000 bytes, or a quarter of a gigabyte). 250 megabytes is enormous, and is quite literally ten times larger than you are normally allowed to send over most email systems.

I am NOT a proponent of cloud-based storage. First off, there's that bit about “online storage—for life!”. The implication is your life, but more likely the company will die long before you do. And it's not just limited to small companies that go under, even large companies like Yahoo and Google (who now has more money than God and Microsoft) have shutdown “online storage—for life!”

No, really—you can lose access to your “cloud-services” for the darndest of reasons and have absolutely no recourse.

Yes, box.com, this time it will be different.

The next item, the 250M file limit. Yes, it's large, but personally, I have about two dozen files that exceed that limit. Okay, they're videos and CD images of various Linux distributions, and one archive (a zip file) of MP3s (the soundtrack to an online game someone made). It's not a horrible limit, but there are legitimate files that exceed that limit. This isn't necessarily a deal breaker, but it does limit what I can backup on this so-called “cloud service.”

The next reason I dislike these “cloud services” will definitely come across as “tin-foil haberdashery” but—you may not have Fourth Amendment protections for data in the clouds. Even if it's served with a warrant, Google may not be able to notify you of the warrant it received to search your email stored there. Even having nothing to hide is no reason to give up your rights to hide the fact you have nothing to hide (told you this was “tin-foil haberdasery”—furthermore, you might want to be careful what you post at GoogleFacePlusBook, I'm just saying).

One last reason I don't like “cloud-services”—I can access all my files here at Chez Boca from anywhere, thanks to the miracle of the Internet. I don't need a third party to manage my own data. Of course, it's a bit of work to do so on the modern Internet (what with dynamic IP addresses and that horrible hack known as NAT, which broke the true peer-to-peer notion of the original Internet, but I digress) but it is possible.

But in the mean time, just be aware of what you can expect from “cloud services” (no privacy) before using it.

And remember this: if you're not paying for the product, you are the product.

---

1994 called. They want your tag soup site back.

I received a comment from Catherine B. on GoogleFacePlusBook about the iPad-only site with a link to an embroidery trouble shooting guide. Aside from the 1976 design asthetic (red and blue on a white background) the other odd thing about the site is that … well … just scroll down, and keep scrolling down. It's … amusing.

Wednesday, February 13, 2013

A lot of kids of my generation had a similar dream and all felt that New Zealand was probably the safest place to live.

Missile Command cannot be fully understood without taking into account its historical context. It is a direct product of its time, a cathartic experience dealing with the fears of the Cold War era. To attenuate its bleak tone, unique in video game history, geographical references to the Californian coast were ultimately dropped from the game.

“It was pretty scary. During the project and for 6 months after the project, I'd wake up in a cold sweat because I'd have these dreams where I'd seen the missile streak coming in and I'd see the impact. I would be up on top of a mountain and I'd see the missiles coming in, and I'd know it would be about 30 seconds until the black hit and fried me to a crisp.”

Dave Theurer
Creator of Missile Command

Just before release, the title of the game was changed to Missile Command. It was originally meant to be titled Armageddon.

Retro Sabotage - Missile Command | Docudrama - Flash Game [ver.9.0 req.]

An interesting bit of historical context for Missile Command. The whole Retro Sabotage site is a series of unique takes on a bunch of old video games. In fact, the second Pac-Man game in Mockumentary is actually a really clever twist on the game, and one that took me an embarrassing amount of time to figure out—you control the ghosts, and believe me, it's a lot harder than it appears to trap Pac-Man when you control all four ghosts at the same time with the same controls (you move up, all four ghosts attempt to move up at the same time).

All the games presented all have a weird or odd twist to them.

Thursday, February 14, 2013

I have been informed that it is indeed, a holiday today

Friday, February 15, 2013

Heads in the clouds, II

Curious. Three days after I post about my dislike of cloud services, Posterous, one of those “free for life, share anything” type sites is closing down (link via Hacker News).

Why am I not surprised?

Anyway, what I realized is that my previous post did not address possible solutions to the problems that “cloud storage” suposedly solves.

Backup

Easiest solution: an external harddrive. This is literally a “plug-n-play” solution as you plug in the external harddrive and copy your files to it. You don't need much in the way of software—all you need is something that can copy a mass collection of files from one location to another.

This also makes restoring your files easy—just copy them back off the external harddrive.

Disaster recovery

As was pointed out by my first cousin once removed Roger, there's also the concern about the house burning down. And there's an easy solution for that—buy two external harddrives and use both to backup your files.

Of course, what you really want to do is keep one of the harddrives off-site, like at the office, or a friend's house, or a bank deposit box. Just make sure that periodically, you swap the two drives.

An easy scheme—keep one drive at the office, one at home. One day a week (say, Wednesday), take the one at home to the office, and bring the one at the office back home. Keep backing up, and if the house burns down, at most you lose one week's worth of files.

Which is better than nothing.

Which leaves:

Access and collaboration

Admittedly, this isn't quite as easy.

It could be done by running a webserver on your home computer, and forwarding traffic from your router to your PC, but that involves quite a bit of software and if you get it wrong, the wrong people may gain access to stuff you don't want.

Then again, with cloud storage, that could still happen.

But another thought occured to me—more and more people are using laptops so they pretty much have all their files with them. So there's that solution.

But collaboration, and the closely related synchronization, is a very hard problem (it's called “cache invalidation”)—it's one of the two hardest in Computer Science (the other one being naming and off-by-one errors). You might as well use a cloud-service for this, but be aware of security implications as well as accessibility issues.

Saturday, February 16, 2013

There's just enough atmosphere to make it interesting

Bunny sent me this video of landing a rover on Mars. She liked it because the rover reminded her of Wall-E (one of her favorite films). It's also strikes me as silly that NASA would seriously consider covering the rover with beach balls and have to bounce to a landing.

Then again, considering what it takes to land on Mars, maybe it wasn't such a silly idea after all.

---

This could make for one killer spy computer

The Cotton Candy Computer (link via Wlofie) is an interesting computer in that it's a USB sized computer. Granted, it's a bit more expensive than the Raspberry Pi but the form factor is way smaller and could make (possibly) a decent environment for developing Android applications.

Sunday, February 17, 2013

It's not quite a Prisoner's Dilemma as there was no incentive to defect

The students in Professor Peter Froehlich's “Intermediate Programming” and “Introduction to Programming for Scientists and Engineers” (a Python language class) classes, boycotted their finals last December. The former initially organized the boycott and the latter followed suit.

To avoid the stress of taking their exam, the students decided to capitalize on a loophole in Froehlich's grading system.

“In my courses, all grades are relative to the highest actually achieved score. Thus, if no one showed up and everyone got 0 percent, everyone would be marked as 100 percent,” Froehlich wrote in an email to The News- Letter.

Via Reddit, Computer science students successfully boycott class final | The Johns Hopkins News- Letter

As remarkable as all the students in class deciding not to take the exam, it's even more remarkable that the instructor kept his word and awarded everyone 100 percent on the exam.

And I'm bummed that I never had an instructor in school make such an offer.

Monday, February 18, 2013

It's all a matter of perspective

I feel I have a pretty good spatial sense, and I adore the works of M. C. Escher, but playing a game based on Escherian perceptions of reality?

Yeah, that's a bit much, even for me.

Tuesday, February 19, 2013

Bunny has been having a heel of a time

Since last Monday I've been working from home (ah, gotta love telecommuting) to help Bunny recover from stabalizing her partial talotarsal dislocation in her left foot—effectively, her podiatrist shoved a metal pin into the side of her foot to keep her arches from collapsing (ouch).

Even though it's an outpatient procedure, she has very limited mobility (enough to avoid the dreaded bedpan) and she has to wear this freakishly large boot to keep her ankle mostly immobile. And I've been working from home to help tend to Bunny, as well as chauffeur her to the podiatrist for her checkup visits.

So far, her foot is healing fine, but she won't be really mobile until late this week or early next week.

Wednesday, February 20, 2013

“Just sell public information,” they said …

If we're offering advice from the peanut gallery (and I'm no smartphone developer, so I'm way up in the peanut gallery), why not spend that time on a program that can generate trivial apps.

Extract data [legally, now!] from some source (eg wikipedia), and skin it with some boilerplate and some generated UI elements. I think it'd be a more interesting project to implement the more general version, and once it's nailed down it's probably a better source of revenue too.

If we're offering advice from the peanut gallery (and I'm no smartphone develope… | Hacker News

This reminds me of a job I had years ago.

And given that it was probably over ten years since I did this job, I think I'm now safe enough to talk about the job, which involved scraping public information from a website so my employer could sell the information.

It was a freelance job. The gentleman who hired me was a lawyer specializing in the financial industry. The idea was to download public information, package it up as a book to sell it to financial market insiders. All I needed to do was to scrape the public information from a website.

The Public Disclosure Program discloses the following information on firms:

• the firm's name, CRD number, SEC identification number, “applicant name” or legal name, NASD district office which oversees the firm, and the firm's main and mailing addresses
• the types of business in which the firm is currently engaged
• details of the firm's legal status (i.e, corporation, partnership, etc.), state or country and date of formation, and its fiscal year end
• all approved registrations
• disclosure matters involving certain criminal charges and convictions, regulatory actions, civil judicial actions, and certain financial actions (e.g., bankruptcies, unsatisfied judgment/liens.)

Easy stuff. Just submit a form, get the output. Pass it along to the lawyer. And when I found out how much he was going to charge (around $1,200 a copy) I was kicking myself for not thinking of this on my own.

But there were problems. First off, the site in question took a dim view of my scraping (even though I wasn't hitting the site all that hard—maybe a request every few minutes) that they changed how the results were returned. Now I had to set up email accounts to accept the results.

Then I had to learn how to manage emails with attachments.

They then changed the sumbmission form multiple times.

In all of this, I was told that the information is public and that there is no question of legality involved with this. Remember, my employer was a lawyer and well … okay, it's public information about securities firms.

So I kept up with all the changes and kept handing over the files to my employer.

Then I received a call from their lawyers.

Oh.

I immediately told them I was just a hired gun and that the person they really wanted to talk to was my employer. Thankfully, I never did hear back from them, and I never had to appear in court nor did I receive a summons. At least my employer kept those lawyers off my back and bore the brunt of a lawsuit against him for selling “public information” (he ultimately lost).

In hindsite, I was mighty glad I didn't have the idea to do that. Not only would I have had trouble selling such a book, given that I knew absolutely nothing about the industry, nor did I know anyone in the industry, but I was shielded from a lawsuit.

Yes, the idea is nice in theory, but in practice, you had an organization that wasn't thrilled with someone actually trying to use the “public information” and made their intentions known.

This is just something to keep in mind if you ever get a similar idea.

Thursday, February 21, 2013

Peanut butter and chocolate? Wonderful. Raspberry and chocolate? Divine. All three?

“Noooooooooooooooooooooooooo!” I yelled.

“What happened?”

“I dropped the mint chocolate truffles on the floor,” I said, scrambling as fast as I could, but alas, I exceeded the alloted five seconds and all but one lone truffle still sitting cleanly in its paper cup were truly contaminated.

“What about the five—”

“Alas, I exceeded the time,” I said. “Besides, this is the garage floor we're talking about.” I kept the mint chocolate truffles in the freezer out in the garage. They were a Christmas gift and I was slowly working my way through the box; I still had over half the box left.

“Oh,” said Bunny. “You could try one of these,” Bunny said, limping slowly into the garage. She then handed me a Trader Joe's PB & J Milk Chocolate Bar. The bar was part of a care package her brother sent from Seattle. “I was going to have it, but it sounds like you could use it.”

“A peanut butter and jelly chocolate bar,” I said, somewhat skeptically. “I'll try it if you try it.”

“Okay,” said Bunny.

We headed back to the family room. Bunny opened the bar and broke off two pieces. She handed one piece to me and ate the other one. I then ate the piece I had. And well … it was interesting. Now, I like peanut butter and chocolate (Bunny knows better than to buy Reese's Peanut Butter Cups™ else I gorge myself) so it's got that going for it. I also like raspberry and chocolate (the jelly here is raspberry jelly) so it's got that going for it. But all three? I don't think that worked.

Oh, it was edible, don't get me wrong; no spit-takes here. But it just didn't work for me (or for Bunny). I think it actually was the peanut butter and raspberry that failed in this case. It wasn't bad enough not to finish, but we wouldn't go out of our way to get another one (and we have two more of the bars to go through).

Friday, February 22, 2013

Aliens the Musical. Enough said

What more can I say? Aliens the Musical (via Twenty Sided).

---

This is one of those “hard” problems, isn't it?

Depending on how you want to think about it, it was funny or inevitable or symbolic that the robotic takeover did not start at MIT, NASA, Microsoft or Ford. It started at a Burger-G restaurant in Cary, NC on May 17. It seemed like such a simple thing at the time, but May 17 marked a pivotal moment in human history.

Burger-G was a fast food chain that had come out of nowhere starting with its first restaurant in Cary. The Burger-G chain had an attitude and a style that said “hip” and “fun” to a wide swath of the American middle class. The chain was able to grow with surprising speed based on its popularity and the public persona of the young founder, Joe Garcia. Over time, Burger-G grew to 1,000 outlets in the U.S. and showed no signs of slowing down. If the trend continued, Burger-G would soon be one of the “Top 5” fast food restaurants in the U.S.

The “robot” installed at this first Burger-G restaurant looked nothing like the robots of popular culture. It was not hominid like C-3PO or futuristic like R2-D2 or industrial like an assembly line robot. Instead it was simply a PC sitting in the back corner of the restaurant running a piece of software. The software was called “Manna”, version 1.0^*.

Manna's job was to manage the store, and it did this in a most interesting way.

Via a comment at Hacker News, Manna, Chapter 1, by Marshall Brain

It's an interesting piece of fiction, and certainly timely as there's a fear of computers taking over our jobs. Even I'm not immune as I've felt for quite some time that there are simply not enough jobs for everyone, and that most jobs are “fluff” or “make busy” jobs just to keep people employed. And then I imagine the productivity gains if computers were really used to their potential instead of playing solitare and posting cat videos— unemployment would probably be worse than it is now.

And then I come across stuff like this (granted, this was nearly twenty years ago, but still):

Making a work schedule in a place like this is a fiendishly difficult task. Each employee has a schedule of availability that is unique to them. Alice can only work weeknights before nine. Bob can only work Monday, Wednesday, and Friday, and can't work after dark because he still has a junior license which prohibits nighttime driving. Carl can't work on Sundays, won't work before three in the afternoon, and his mother doesn't want him to work more than ten hours a week. Dave and Ellen can't work at the same time, since they're living together and one of them needs to be home with the baby. Furthermore, some employees are not suited to some tasks. Fred is too dumb and rude to work the register. Gretchen can work the register but hasn't yet been trained on preparing food. Additionally, you must make sure to give all of the employees the right number of hours for the week. You can't let the full-timers drop below forty hours or they won't be able to pay their bills. You can't make any of the minors work above a certain limit or it's a federal offense. You have to give everyone at least a few hours or they'll quit. Above all, you can't give anyone more than forty hours or corporate will mete out harsh judgement on you for allowing people to earn overtime pay.

Using this list of restrictions, exceptions, and limitations, you have to fit these employees into a schedule that gives you exactly as many people as you'll need at any given time of the day. This task ends up being an hours-long puzzle where there isn't guaranteed to be a solution.

At McDonald's this was done by hand. Here at Taco Bell, the schedule is first done by a computer, and then a manager has to come along and completely re-build it. The computer can't generate a usable schedule because the rules of each individual employee are so complex that there's no way to explain them to the computer.

Autoblography Part 34: The Systems Analyst - Twenty Sided

(It gets worse from there with upper management trusting the computer models over reality.)

And I don't know what to think. Are we doomed to live our lives under our robotic overlords? Or are we simply doomed to corporate dysfunction (“We put the ‘fun’ in dysfunction!”) and the collapse of civilization?

Saturday, February 23, 2013

But it still doesn't help when the DSL goes down

The Corporation has been reorganizing the network again (which seems to coincide with corporate reorganizations but I'm sure they're not related at all) so I thought I would help my fellow cow-orkers figure out if it's down for everybody or just them.

Sunday, February 24, 2013

Musical links

I have a few links for Bunny. The first is an alternative piano keyboard layout (link via Microclesia) that to me, would seem to make piano playing easier, as each scale is played in an identical manner, instead of having to learn a dozen different fingerings.

The second one is this unique musical visualization of jazz music (link also via Microclesia). I personally have a hard time “hearing” music when looking at its printed representation, so I find these visualizations helping in “seeing” music.

It's actually quite amusing—I saved a link to the laptop orchestra that my dad sent me over four years ago (yes, I have quite the backlog) that now, I just don't find all that interesting (and I don't think Bunny would find interesting either) but the following posts from that blog I found extremely interesting.

Monday, February 25, 2013

Something you never want to hear from an airline pilot

Ladies and gentlemen, this is your captain speaking. We have a small problem. All four engines have stopped. We are doing our damnedest to get them going again. I trust you are not in too much distress.

British Airways Flight 9

And this reminds me of the following quote: “Superior pilots will use their superior judgement to avoid sitations that require their superior skills.”

---

Some impressions about LuaRocks

I've gone off against “control panels” (and “package managers”) a few times so it's odd that I find myself playing around with LuaRocks, a package manager for installing Lua-based software (Perl has CPAN, Ruby has RubyGems, so it's not uncommon for package management to exist for languages).

So far, it hasn't been that horrible. One neat aspect of LuaRocks is that the specification for a “rock” can exist outside the source code that comprises a “rock.” For instance, the following “rockspec:”

package = "OrgConman"
version = "1.0.0-1"

source =
{
  url    = "git://github.com/spc476/lua-conmanorg.git",
  branch = "master",
}

description =
{
  homepage   = "https://github.com/spc476/lua-conmanorg",
  maintainer = "Sean Conner",
  license    = "LGPL",
  summary    = "Useful modules.  A lot of useful modules",

  detailed = [[ 
	A lot of useful routines to manipulate tables.  Yes, I should write
	more text here.  But I'm still playing aorund with this stuff.
  ]],
}

supported_platforms =
{
  "linux",
  "macosx",
  "freebsd",
  "unix"
}

dependencies =
{
  "lua ~> 5.1"
}

external_dependencies =
{
  TCC     = { header  = "libtcc.h" },
  OPENSSH = { library = "crypto"   },
  MAGIC   = { header  = "magic.h"  },
  RT      = { library = "rt"       },
}

build =
{
  type = "make",
  copy_directories = {},
  variables =
  { 
	CC     = "$(CC) -std=c99", 
	CFLAGS = "$(CFLAGS)",
	LFLAGS = "$(LIBFLAG)",
	LUALUA = "$(LUADIR)",
	LUALIB = "$(LIBDIR)",
  },
}

can be used, as is, to install the package, even though no existing “rock” exists, nor has it been published anywhere. All it takes is saving it in a file called orgconman-1.0.0-1.rockspec, and running luarocks build orgconman-1.0.0-1.rockspec—it will then download the code from github, build it and install it (assuming you meet the dependencies listed).

And once published, it will be a simple luarocks install orgconman to install the code (this will check the “rock servers” listed in the configuration file for the named “rock”, download the “rockspec” and then build and install it).

Even nicer, you can specify different locations to install the resulting “rocks” beside the default /usr/local/lib/lua/ (such locations can be specified in the configuration file, or on the command line).

So far, the only problematic aspect of LuaRocks I've encountered is actually making a “rock.” luarocks pack rockspec doesn't quite work out of the box:

[spc]lucy:/tmp>luarocks pack orgconman-1.0.0-1.rockspec
Initialized empty Git repository in /tmp/luarocks_orgconman-1.0.0-1-7168/lua-conmanorg/.git/
remote: Counting objects: 41, done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 41 (delta 11), reused 11 (delta 2)
Receiving objects: 100% (41/41), 60.55 KiB, done.
Resolving deltas: 100% (11/11), done.
sh: line 0: cd: lua-conmanorg: No such file or directory
Packed: /tmp/orgconman-1.0.0-1.src.rock

It seems like it's not switching into the temporary build directory when trying to make the “rock.” I can work around it (by checking out the code at the same level as the “rockspec”), but I still find that bug annoying.

Another issue—there is no easy way to specify the language level (C89 or C99) in a platform independent way. Here, I have to assume the given platforms I do support use gcc, but not all platforms I currently use Lua on have gcc (Solaris comes to mind). Again, I'm working around the issue, but I feel it is something that should be addressed somehow.

This is also having me rethink how I develop the modules I have developed. Right now, they all exist in a single repository, but all the modules are mostly independent. But there isn't a simple way to checkout specific files using git, so I may have to make sepearate repositories for each module (so far, 22 existing modules, two additional ones I could add).

Tuesday, February 26, 2013

PEBKAC, of course

This issue with LuaRocks:

[spc]lucy:/tmp>luarocks pack orgconman-1.0.0-1.rockspec
Initialized empty Git repository in /tmp/luarocks_orgconman-1.0.0-1-7168/lua-conmanorg/.git/
remote: Counting objects: 41, done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 41 (delta 11), reused 11 (delta 2)
Receiving objects: 100% (41/41), 60.55 KiB, done.
Resolving deltas: 100% (11/11), done.
sh: line 0: cd: lua-conmanorg: No such file or directory
Packed: /tmp/orgconman-1.0.0-1.src.rock

Yeah, that one. Totally my fault and nothing at all to do with LuaRocks. Between help from Ignacio Burgueño (via email) and Hisham Muhammad (via a bug report) I was able to isolate the issue to a Lua module I installed 3½ years ago that LuaRocks will use if it's installed.

The only problem—the module in question was compiled with some … um … let's say “questionable compiler options” (specified by yours truly).

Ugh.

Anyway, that issue has been resolved.

Sheesh.

Wednesday, February 27, 2013

Who needs critics with publishers like this?

An independent bookseller I know landed a major bestselling author for a rare in-store signing. He got the word out, took advance phone and internet orders for signed copies, and called his sales rep at the publisher to make sure the books would reach him in plenty of time.

“You’ve ordered 450 copies,” the rep told him. “I’m afraid we can only ship you 200.”

Why, for God’s sake? Hadn’t they printed enough?

“No, it’s policy,” he was told. “Two hundred books is our maximum order. We can’t take the chance of huge returns, or credit problems.”

“But the copies are sold,” the store owner said. “I’ve got prepaid orders for them, and I’ll pay in advance myself, and take them from you on a non-returnable basis. There’s no risk, and there won’t be any returns, and that’s 450 copies of a $30 book at the usual 40% off, which makes it an $8100 cash order. So what’s the problem?”

He got nowhere.

Via InstaPundit, Great Moments in Contemporary Publishing | LB's BLOG

You know, twenty years ago forgoing a publisher for the “self-publisher” route was a sign of vanity on the part of the writer. But these days?

These days, I'm thinking your better off avoiding the traditional publishers, as they seem to be tripping themselves into oblivion.

Thursday, February 28, 2013

The scary cuisine of Japan

From

Mark Grosberg <XXXXXXXXXXXXXXXXX>

To

sean@conman.org

Subject

Japanese food

Date

Thu, 28 Feb 2013 09:39:40 -0500 (EST)

Hey Sean,

I know you've never liked Japanese food because it's out there. But I recently stumbled upon a Japanese cooking show on YouTube. Search for username runnyrunny999. He tries to dispell the myth about Japanese food. His food is cooked and some of it looks good although I suspect if you try any of his recipes you'll have to find an Asian market for some of the ingridients.

The show is also funny. It's probably worth your time checking out a few episodes.

-MYG

It's been twenty years, but the experience I had of a “traditional Japanese lunch” is still seared into my brain. I was invited along by the Japanese Language Department at FAU (I was friends with about half the department, which wasn't hard considering it consisted of about six people) to attend “lunch” at a nearby Japanese restaurant.

Now, I've always been a bit leary of Japanese food because of sushi (“Squid eyeballs, people! Squid eyeballs!”) but I was assured that sushi would not be served. What was served, however …

I've blocked most of the food from that “lunch,” but there are still a few items that still strike out in my nightmares from time to time. There was the “soup,” which consisted of a small bowl with about a ½ inch of liquid and … cubes … of … stuff … sitting in the liquid. The liquid wasn't deep enough for “floating.” And yes, these were perfectly cut cubes. Of what, I have no idea. Vegetable matter most likely.

I hope.

Anyway, the only non-cube item in the bowl was a bit of deep fried tofu. Extremely crunchy on the outside, totally liquid on the inside, and gag-inducing in me. Most unpleasant.

Then again, I'm not the most adventuresome of eaters.

Okay, with the “soup” out of the way, the next nightmare item I remember was some form of egg dish. Only, it wasn't in a dish but a desert glass (a cross between a Sundae and a champagne glass). The eggs were … scrambled, I guess. With … stuff. That's the thing—most of the food that day was “stuff.” What? I have no idea. Just … supposedly edible things with off-putting textures and flavors.

I remember a dish with some chicken that was okay (the chicken; not the rest of the dish) but otherwise, the entire “lunch” was a horror show of horrible textures and questionable flavors.

I was lucy to make it out alive.

So it was with trepidation that I viewed runnyrunny999's cooking videos. The curry soba noodles were—oh sweet Jesus that's what those cold slimy dark colored noodles with the questionable white “sauce” at that “lunch” were—soba noodles!

Aaaaaaaaaaaaah!

Oh, wait … sorry about that. Flashbacks. Where was I? Oh yes, the curry soba noodles were (I can get through this) interesting (I didn't realize that curry paste was a curry spiced roux) but I'm not sure if I would eat the resulting dish. Shudder.

The tonkatsu looked edible and I might actually make beef bento lunch box. So far so good. So let's see what's involved with a Japanized-Korean style pizza? I mean, it can't be that bad bad, can—WTF? Did runnyrunny999 just say “bait?” Yes! He said “bait!” He said “whitebait.”

And what exactly is this shirako stuff? It's … um … I just lost my appetite. Yea, I definitely lost my appetite.

Friday, March 01, 2013

Parsing—it's not just for compilers anymore

I've been playing around with LuaRocks and while I've made a rock of all my modules, I've been thinking that it would be better if I made the modules individual rocks. That way, you can install just the modules you want (perhaps you want to embed a C compiler in your Lua program) instead of a bunch of modules most of which you won't use.

And that's fine. But I like the ability to pull the source code right out of the repository when making a rock. Now, given that the majority of my modules are single files (either in Lua or C) and the fact that it's difficult to checkout a single file with git (or with svn for that matter) I think I'd be better served having each module be its own repository.

And that's fine, but now I have a larger problem—how do I break out the individual files into their own repositories and keep the existing revision history? This doesn't seem to be an easy problem to solve.

Sure, git now has the concept of “submodules”—external repositories referenced in an existing repository, but that doesn't help me here (and git's handling of “submodules” is quirky at best). There's git-filter-branch but that's if I want to break a directory into its own repository, not a single file. But there's also git-fast-export, which dumps an existing repository in a text format, supposedly to help export repositories into other version control systems.

I think I can work with this.

The resulting output is simple and easy to parse, so my thought is to only look at bits involving the file I'm interested in, and generating a new file that can then be imported into a fresh resposity with git-fast-import.

I used LPeg to parse the exported output (why not? The git export format is documented with BNF, which is directly translatable into Lpeg), and the only difficult portion was handling this bit of syntax:

'data' SP <count> LF
<raw> LF?

A datablock contains the number of bytes to read starting with the next line. Defining this in LPeg took some thinking. An early approach was something like:

data = Ct(				-- return parse results in table
	   P'data '			-- match 'data' SP
	   * Cg(R"09"^1,'size')		-- get size, save for later reference
	   * P'\n'			-- match LF
	   * Cg(			-- named capture
	         P(tonumber(Cb('size'))) -- of 'size' bytes characters
		 ,'data'                -- store as 'data'
	     )
	   * P'\n'^-1			-- parse optional LF
	)

lpeg.P(n) states that it matchs n characters, but in my case, n wasn't constant. You can do named captures, so I figured I could capture the size, then retrieve it by name, passing the value to lpeg.P(), but no, that didn't work. It generates “bad argument #1 to 'P' (lpeg-pattern expected, got nil)”—in other words, an error.

It took quite a bit of playing around, and close reading of the LPeg manual before I found the solution:

function immdata(subject,position,capture)
  local size  = tonumber(capture)
  local range = position + size - 1
  local data  = subject:sub(position,range)
  return range,data
end

data = Ct(
	   P'data '
	   * Cg(Cmt(R"09"^1 * P"\n",immdata),'data')
	   * P'\n^-1
	)

It's the lpeg.Cmt() that does it. It calls the given function as soon as the given pattern is matched. The function is given the entire object being parsed (one huge string, in this case the subject parameter), the position after the match (the position parameter), and the actual string that was matched (the capture parameter). From there, we can parse the size (tonumber(), a standard Lua functionm, ignores the included line feed character), then we return what we want as the capture (the variable amount of data) and the new position where LPeg should resume parsing.

And this was the hardest part of the entire project, trying to match a variable number of unknown characters. Once I had this, I could read the exported respository into memory, find the parts relating to an individual file and generate output that had the history of that one file (excluding the bits where the file may have moved from directory to directory—those wheren't needed) which could then be imported into a clean git repository.

Saturday, March 02, 2013

Stupid GitHub tricks

All that work parsing git repositories was for naught—it seems you can link to individual files on GitHub (go figure!). So now I can create a rockspec per module, like:

package = "org.conman.tcc"
version = "1.0.0-0"

source =
{
  url = "https://raw.github.com/spc476/lua-conmanorg/1.0.0/src/tcc.c"
}

description =
{
  homepage = "http://...",
  maintainer = "Sean Conner <sean@conman.org>",
  license    = "LGPL",
  summary    = "Lua wrapper for TCC",
  detailed   = [[
	Blah blah blah
  ]]
}

dependencies =
{
  "lua ~> 5.1"
}

external_dependencies =
{
  TCC = { header = "libtcc.h" }
}

build =
{
  type = "builtin",
  copy_directories = {},
  modules =
  {
    ['org.conman.tcc'] = 
    {
      sources   = { 'tcc.c' },
      libraries = { "tcc" },
    }
  },

  variables =
  {
    CC = "$(CC) -std=c99",
    CFLAGS = "$(CFLAGS)",
    LFLAGS = "$(LIBFLAG)",
    LUALIB = "$(LIBDIR)"
  }
}

(this particular module embeds a C compiler in Lua, which is something I do need to talk about).

But it isn't like I wasted time on this. No, I don't view it that way at all. In fact, I learned a few things—how to parse git repositories, how to parse a variable amount of data in LPeg and I have code to extract a single file into its own git repository if I ever have that need again.

Sunday, March 03, 2013

I mean, Washington may be a wretched hive of scum and villiany, but …

From

"Agent Chris Swecker"<laura@nwclbj.com>

To

undisclosed-recipients:;

Subject

*****SPAM***** UNITED STATES DEPARTMENT OF JUSTICE

Date

Sun, 3 Mar 2013 03:38:06 +0800

Spam detection software, running on the system “DiskStation”, has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see admin for details.

Content preview: Federal Bureau of Investigation (FBI) Counter-terrorism Division and Cyber Crime Division J. Edgar. Hoover Building Washington DC Dear Beneficiary, Series of meetings have been held over the past 7 months with the secretary general of the United Nations Organization. This ended 3 days ago. It is obvious that you have not received your fund which is to the tune of $8.5,000.000.00 due to past corrupt Governmental Officials who almost held the fund to themselves for their selfish reason and some individuals who have taken advantage of your fund all in an attempt to swindle your fund which has led to so many losses from your end and unnecessary delay in the receipt of your fund. […]

Content analysis details: (6.9 points, 5.0 required)

pts	rule name	description
-1.4	ALL_TRUSTED	Passed through trusted hosts only via SMTP
1.8	SUBJ_ALL_CAPS	Subject is all capitals
0.0	MONEY_BACK	BODY: Money back guarantee
0.0	HTML_MESSAGE	BODY: HTML included in message
1.7	MIME_HTML_ONLY	BODY: Message only has text/html MIME parts
0.0	FORGED_OUTLOOK_TAGS	Outlook can't send HTML in this format
0.7	MSOE_MID_WRONG_CASE	MSOE_MID_WRONG_CASE
0.0	FORGED_OUTLOOK_HTML	Outlook can't send HTML message only
4.2	FORGED_MUA_OUTLOOK	Forged mail pretending to be from MS Outlook

The original message was not completely plain text, and may be unsafe to open with some email clients; in particular, it may contain a virus, or confirm that your address can receive spam. If you wish to view it, it may be safer to save it to a file and open it with an editor.

You know … if you're going to try a Nigerian 419 scam, it might be wise to ensure your email isn't flagged as spam on the way out of your own email server! I'm just saying …

Monday, March 04, 2013

Ideas in parsing the command line

For any non-trivial script, even for personal consumption, it's necessary to supply usage text. The novelty of Lapp is that it starts from that point and defines a loose format for usage strings which can specify the names and types of the parameters.

An example will make this clearer:

	-- scale.lua
	  require 'lapp'
	  local args = lapp [[
	  Does some calculations
	    -o,--offset (default 0.0)  Offset to add to scaled number
	    -s,--scale  (number)  Scaling factor
	     <number> (number )  Number to be scaled
	  ]]

	  print(args.offset + args.scale * args.number)

lua-users wiki: Lapp Framework

The thought of parsing the usage text for parsing the command line never occured to me, and I think it's brilliant.

Now, when I want to modify the command line of a program I wrote (and this is mostly in C, by the way), there are four locations I have to edit:

1. An enumeration specifying the “short” form of the command line option
2. A structure describing both the short and the long forms of a command line option
3. A switch statement that processes the command line options from getopt_long()
4. The text printed out describing the command line options

This method though, there's only one area I would have to edit.

Now granted, this is only for Lua, but I can't see why something similar for other languages can't be done.

---

An unholy mashup

I know some of my college friends will find this herectical, but what happens when you take Carly Rae Jepsen's “Call Me Maybe” and mix it with Nine Inch Nails' “Head Like A Hole”?

Oddly enough, you get something that works (link via GoogleFacePlusBook), kind of, maybe, in that “train-wreck cross-genre musical mashup” kind of way.

Tuesday, March 05, 2013

Papers please

I've seen Constitutional Free Zone map multiple times, and I've tried to substantiate the claims made by the ACLU but everything I've found so far points back to the ACLU; I've yet to find anything that doesn't point back there.

But then I saw this video of DHS checkpoint refusals (link via Flutterby), all of which took place inside the United States (and not at the border), one of which was at least 30 miles from the border. So perhaps there is something to that ACLU article.

Another odd thing about that video—most of the agents would not, refused, or tried to talk about the question “am I being detained?” Oh, and the automatic assumption of guilt when one refuser plead the Fifth. I also found it offensive when the officers admonished the refusers for making their job more difficult.

Wednesday, March 06, 2013

Peak government, I tell ya! Peak government!

Take Eddie Leroy Anderson, a retired logger from Idaho whose only crime was loaning his son “some tools to dig for arrowheads near a favorite campground of theirs,” according to the Wall Street Journal. Anderson and his son found no arrowheads, but because they were unknowingly on federal land at the time they were judged to be in violation of an obscure Carter- era law called the Archaeological Resources Protection Act.

The government showed no mercy. Wendy Olson, the Obama appointee prosecuting the case, saw to it that father and son were fined $1,500 apiece and each sentenced to a year's probation. “Folks do need to pay attention to where they are,” she said.

Statutory law in America has expanded to the point that government's primary activity is no longer to protect, preserve and defend our lives, liberty and property, but rather to stalk and entrap normal American citizens doing everyday things.

After identifying three federal offenses in the U.S. Constitution— treason, piracy and counterfeiting—the federal government left most matters of law enforcement to the states. By the time President Obama took office in 2009, however, there were more than 4,500 federal criminal statutes on the books.

Via Instapundit, Op-Ed: How to end overcriminalization | WashingtonExaminer.com

Remember, ignorantia juris non excusat, so better start reading.

Thursday, March 07, 2013

Plugging leaky memory

Just because a language has garbage collection doesn't mean you still can't leak memory—you can easily leak memory, since quite a few modern langauges that have garbage collection have ways of calling into libraries written in C, and those can leak.

With that said, reading “Tracking down a memory leak in Ruby's EventMachine (link via Hacker News) was quite informative. Looking for patterns in the leaked memory as a means of tracking down what was being leaked was brilliant (“Well, as mentioned, 95+% of our program’s memory footprint is leaked objects. So if we just take a random sample of bits of memory, we will find leaked objects with very good probability.”). And I did not know you could call C functions from within gdb.

This is something I'll have to keep in mind for work.

Friday, March 08, 2013

Um … why couldn't the giant eagles fly the Ring into Mordor?

When I first saw How the Lord of the Rings Should Have Ended, I though, “Yeah, why didn't they use the giant eagles to fly the Ring to Mount Doom?”

One of the arguments I've heard is that the Nazgûls' fell beasts would have taken out the giant Eagles once they entered into Mordor. But Sean Chist felt otherwise—he thinks it's a plot hole left by Tolkien (link via Hacker News).

But personally, I find this theory much more plausible and a much better explanation for why “Operation: AirDrop One” wasn't done.

Saturday, March 09, 2013

Shave and a haircut, definitely not two bits

Bunny decided it was time for me to get a haircut. Normally, she does the cutting, but after the last haircut I received at a real barber shop (in Brevard—real barber pole, barbers, wood panelling, the works) she felt that the professionals did a much better job at it then she.

But we live in Boca Raton, not quite your Small Town, USA™ so we had to make do with something a bit more upscale, The Man Cave. At the appointed time, I walked into the Man Cave.

“Welcome, Sean,” said the hostess.

“Um … how did you–”

“You're here for your four o'clock appointment,” she said. “Would you care for wine? Or perhaps an imported beer from the Continent?”

“Oh. Um. No, I'm fine.”

“Very well. Chel will take care of you,” she said. “Chel! You're four o'clock is here.” She pointed over to the chairs, nestled among oversized high-contrast portraits of James Dean and Marlon Brando. “This way,” she said.

“Hello,” said Chel, walking over to lead me to her chair. “Please, take a seat. Short, over the ears, close cropped shave.”

“How did—”

“Shh, just sit back and relax,” she said, tying a paper collar about my neck and adjusting the snap-on tarp. “Glasses,” she said.

“Oh, yes,” I said. I took off my glasses, and she placed them gently on the nearby counter. She then started clipping my hair. It was the typical motions—snip here, snip there, reposition my head, more snipping, use the electric razer here and there and before long, she had apparently finished with cutting my hair.

She then lowered the back of the chair so I was nearly lying horizontally. “Please, relax,” Chel said, as she lowered a folded, steaming hot towel across the lower half of my face, then raised the folded part to cover my entire face. Oddly enough, even though I could see the steam rising off of it (even without my glasses) it wasn't scalding. In fact, it felt nice. It was wisked off, then she massaged my face, then another towel, then various gels and what not were rubbed into my face, then another hot towel, then more gels and finally, the shave with an honest-to-god straight razor. That was weird. I could feel it (felt like a sharp pencil against my skin) and hear it scrap the hair off my face.

And with that, I was done.

It was not cheap. But it was a fun experience. And certainly a different experience from a small town barber shop.

---

I have a sad story to tell you, it might hurt your feelings a bit

Bunny and I found ourselves at The Rock Steady Jamaican Bistro in Boca Raton for dinner. An early dinner. Or a late lunch. Or a very late brunch. Dunch, if you will.

So we were eating dunch at The Rock Steady Jamaican Bistro (the food was quite good, but you better like it jerked) and for a change, we were listening to reggae. Not because we wanted to listen to raggae, but beause we were eating at a Jamaican restaurant.

And it could have been worse—it could have been country reggae.

But we were listening to reggae, when it struck me, the reggae music—it was a song I haven't heard in a long time …

“I have a sad story to tell you.”

I was trying to place where I heard it …

“It might hurt your feelings a bit.”

It's been years … and the heavy reggae beat wasn't helping, mon.

“I stepped in the bathroom last night.”

What was it?

“And stepped in a pile of shhhhhhhhh—”

Shaving Cream! It was the Shaving Cream song! Only with a very heavy reggae beat and a thick Jamaican accent. But yes, it was the Shaving Cream song!

Incredible!

---

A Liar's Autobiography

Bunny and I watched “A Liar's Autobiography: The Untrue Story of Monty Python's Graham Chapman” and I must say—it's a very odd film, even by Monty Python standards. Animated, in fourteen different styles (some of it very beautifully done) with most of Monty Python doing voice work (and no, it's not Graham Chapman that isn't in it), it's a completely made up story of the life of Graham Chapman.

Well, mostly made up. It does cover his homosexuality and drinking problems. But everything else is made up. Well, except for him working for Monty Python. But short of the homosexuality, the drinking problems, and working with Monty Python, it's all made up. Except his name really is Graham Chapman.

Okay, let me start over again.

Except for his name, his homosexuality, drinking problems and working with Monty Python, the movie is completey and utterly false.

Except he did study to become a doctor.

Damn!

Well, it's a very weird film and you should watch it because it's kind of true, except when it isn't.

There.

Sunday, March 10, 2013

Visions of a future past

Ah, 70's space colony concept art (link via InstaPundit)—what's not to like?

---

Alternative keyboards

This association of the keyboard with the synthesizer eased its entry into the world of music, but it also placed limitations on how the instrument is played that its designers didn't intend. The limitations of the piano keyboard have been recognized since long before the synthesizer existed. The biggest problem that the keyboard has always had is that, due to the two-row layout with all of the naturals on the bottom row and all of the accidentals on the top row, the performer must usually change fingering in order to transpose a chord from one key to another. This frustrates what should be a simple operation; the guitar player playing a barred chord can transpose it simply by moving up and down the neck, but the keyboard player must keep shifting fingers around to insure that each finger hits on the correct row. The additional manual dexterity and muscle memory requirement makes learning the different keys on the piano a slow and frustrating process. From my own experience, it also introduces the temptation to use teaching shortcuts that cause the student problems later on: a common technique is to start the beginning student out learning the C-major scale, which is played all on the white keys. This introduces a sort of fear or puzzlement at the black keys—what are the for? When does one use them? And then when the teacher starts introducing other scales, the use of the black keys seems arbitrary and unsystematic, and the student gets a bit freaked out. By contrast, guitar pedagogy treats the accidentals as simply other notes in the chromatic scale, which they are, and the guitar student has relatively little trouble understanding how to play different scales and keys.

Via Hacker News (via Hacker News), Sequence 15: Alternative Keyboards

I'm actually puzzled with (musical) electronic keyboards. Sure, they have a layout like a traditional piano, and yes, the C-major scale is played with all white keys, so why can't if you decide to play, say, a F-major scale, why can't you just remap the frequency of the keys so you can still play it with all white keys? C-major the keys can go C-D-E-F-G-A-B while in F-major, they go F-G-A-B♭-C-D-E and for A♭-major they go A♭-B♭-C-D♭-E♭-F-G. The same fingering, regardless of scale.

Yes, I know you can't do this on a traditional piano, but I'm not talking about a traditional piano here. You can't change the layout of a typewriter, yet it's trivial to change the layout of a computer keyboard (used to type text)—it's a matter of changing the software and boom—you have a Colmak layout!

But short of that, I am facinated by alternative music keyboards, probably because I'm not a musician and to me, these alternative music keyboards seem to show the patterns inherent in music must better than a piano keyboard.

Monday, March 11, 2013

Paper

For all the things an iPad can do, there's still a place for paper (link via dad).

Tuesday, March 12, 2013

Flowing water, stationary

Brusspup filmed water going through a speaker running at 24Hz (link via Hacker News) and the result is incredible—the water just sits there, in mid-air. As he adjusts the sound wave, the water appears to move slowly down, then reverses itself.

It is a trick, though it's not special effects—it's all done in real time with a camera. But it's the camera that causes the effect—it's acting like a strobe light, catching the action at just the right moment. And it's still impressive though.

Wednesday, March 13, 2013

Why does “Enterprise Software” universally suck?

I work in the QA Department of The Corporation. The majority of the team is stationed in the Seattle Office whereas I am the only member of QA in The Ft. Lauderdale Office. The Seattle Office tests the actual cell phones, whereas not only do I test call processing (and even though I might complain about the Protocol Stack From Hell™, I can automate my tests—muahahahahahaha!) but I am the only person who tests call processing in The Corporation.

I bring this up because the QA Department is now using Gusty, a “Real-Time Test Management System” as it calls itself. And so far, I am seriously unimpressed with it. It's written in Flash and tries its darndest to imitate a Microsoft Windows interface, but it's a far cry from Microsoft Windows. And because it tries to imitate Microsoft Windows, it's quite Microsoft-centric.

But, because it tries so hard to be cross platform, it's almost, but not quite, cute. It screams that it was a Visual Basic application ported to Flash to sell to non-Microsoft shops. There's no way to change the font (which is borderline too small, even for me). It's hard to resize the windows. The scrolling is wonky. It's just an overall clunky user interface.

And that would be fine if it actually helped with my job. But it falls down there too. There are two main objects—requirements and testcases. A test case can have multiple requirements, and a requirement can apply to several testcases. You use different windows to create requirements and testcases.

Now, when you view the details of, say, a requirement, you can get a list of testcases it applies to, but it's a tool-tip like element—meaning, it pops up a small text window with a list of testcases. Can you click on one to go to the testcase? No. Can you select one? No. Can you copy the text with mouse? No. Does it remain up for any length of time? No. Is this list in anyway useful? No. Well … okay, you can memorize the ID before it goes away. So if you want more details on the a particular testcase for a requirement, you have to go to the testcase window and manually search for it.

Oh, there is a search facility (it was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying “Beware of the leopard”) but searching by ID doesn't work. You see, it's a text only search, and IDs being numbers, aren't text …

Yeah.

And the Microsoft-esqueness of the program means that this is really geared towards manual tests. Oh, they pay lipservice to automation and in theory, you can run automated tests, but in theory, there is no difference between theory and practice, but in practice …

In practice, you install some Java client on the machine to run the tests and somehow get this tool to run the tests. And okay, that's fine.

Only my test program runs eight programs (which spawn like a bazillion processes) on four computers, and the programs need to start in a particular order, with particular data. Somehow, I don't think the Gusty tool was built with that type of testing in mind (and when I said the tests I run are automated? Yes. They are. But the setup isn't, as there are a few steps that have security implications involving root).

Now, I'm sure that Gusty is a fine tool within certain limitations (large testing teams manually testing software using Microsoft Windows) but for what I do, it doesn't work at all.

Thankfully, I can continue with my job without having to use Gusty, as I'm practically my own department.

Thursday, March 14, 2013

Happy π Day, I guess …

It seems that several of my peeps on GoogleFacePlusBook are making some noise about it being π Day or something like that. Well … it's 3/14/13, or 3.1413, which is 0.0003 too small. I think it might be better to wait until March 14^th, 2015. It's just too bad it isn't March 14^th, 1592.

Friday, March 15, 2013

Rolling them bones

On GoogleFacePlusBook, Jeff linked to an article about non-transitive dice—three dice where, on average (meaning—many rolls) die A will win over die B, die B will win over die C, but die C will win over die A (kind of like rock-paper-scissors). Even weirder, if you double the dice, two A's against two B's against two C's, the order reverses! (And the accompanying video shows a series of five dice with an even weirder dual-non-transitive ordering).

This page remineded me of a set of “go-first-dice”—a set of twelve sided dice where, say, four people each have a ¼ chance of rolling the highest number, a ¼ chance of rolling the second highest number, a ¼ chance of rolling the third highest number and a ¼ chance of rolling the lowest number. In this case, the dice form a strict ordering (there is no chance of a tie).

Monday, March 18, 2013

You know, you might want to check the calendar for March 2002

Okay, I've seen this particular meme multiple times now on GoogleFacePlusBook:

THIS IS THE ONLY TIME WE WILL SEE AND LIVE THIS EVENT

Calendar for March 2013

March 2013

Sun	Mon	Tue	Wed	Thr	Fri	Sat
					1	2
3	4	5	6	7	8	9
10	11	12	13	14	15	16
17	18	19	20	21	22	23
24	25	26	27	28	29	30
31

This year March has 5 Fridays, 5 Saturdays and 5 Sundays. This happens once every 823 years. This is called money bags. So, share this to your friends and money will arrive within 4 days. Based on Chinese “Feng Shui.” The one who does not share … will be without money.

The first time I saw it, I thought, Weird—is that true? Really? 823 years since the last time? Okay, let's write some code to answer this—which years has March 1^st fallen on a Friday?

#!/usr/bin/env lua

function dayofweek(date)
  local a = math.floor((14 - date.month) / 12)
  local y = date.year - a
  local m = date.month + 12 * a - 2

  local d = date.day
          + y
          + math.floor(y / 4)  
          - math.floor(y / 100)
          + math.floor(y / 400)
          + math.floor(31 * m / 12)
  return (d % 7) + 1
end

for year = 1190,2014 do
  if dayofweek { year = year , month = 3 , day = 1 } == 6 then
    print(year)
  end
end  

1190 is 823 years ago. Let's see what we get …

1191 1196 1202 1213 1219 1224 1230 1241 
1247 1252 1258 1269 1275 1280 1286 1297 
1309 1315 1320 1326 1337 1343 1348 1354 
1365 1371 1376 1382 1393 1399 1405 1411 
1416 1422 1433 1439 1444 1450 1461 1467 
1472 1478 1489 1495 1501 1507 1512 1518 
1529 1535 1540 1546 1557 1563 1568 1574 
1585 1591 1596 1602 1613 1619 1624 1630 
1641 1647 1652 1658 1669 1675 1680 1686 
1697 1709 1715 1720 1726 1737 1743 1748 
1754 1765 1771 1776 1782 1793 1799 1805 
1811 1816 1822 1833 1839 1844 1850 1861 
1867 1872 1878 1889 1895 1901 1907 1912 
1918 1929 1935 1940 1946 1957 1963 1968 
1974 1985 1991 1996 2002 2013

(okay, I cleaned up the output a bit)

Well … it's happened 118 times since 1190, and it didn't happen 823 years ago, but 822 years ago, and 817 years ago and … you get the picture.

Hardly a rare occurrence and in thinking about it, any month with 31 days will have three days that happen five times a month (five Sundays, five Mondays and five Tuesdays for instance).

Now, I can hope this puts this particularly medium meme (it's not rare, and it's not well done) out of commission. But alas, I doubt it …

Tuesday, March 19, 2013

It was an inside job

“On that fateful day, the day that the Empire was attacked, all the crew, staff and guests including security personel and Imperial officials with the exception of one were killed when the terrorist organization known only as the ‘Rebel Alliance’ blew up the Death Star with a squad of single man fighters.”

That much is known. But really, how does a farm boy with no combat experience blow up the most fortified and heavily armed vehicle in the Empire's arsenal? You do really believe the “official story?”

Nah, it was an inside job (link via GoogleFacePlusBook).

Wednesday, March 20, 2013

Simple

You just went to the Google home page.

Simple, isn't it?

What just actually happened?

Well, when you know a bit of about how browsers work, it's not quite that simple. You've just put into play HTTP, HTML, CSS, ECMAscript, and more. Those are actually such incredibly complex technologies that they'll make any engineer dizzy if they think about them too much, and such that no single company can deal with that entire complexity.

Let's simplify.

Via Hacker News, Jean- Baptiste Queru - Google+ - Dizzying but invisible depth

It's hard to simplify how modern computers work. Sure, I could say something along the lines of “a computer consists of three components, memory, which stores a program and data, the CPU, which executes the program out of memory and input/output devices, like a keyboard, the monitor.” But while I'm not outright lying, I am grossly simplifying and skipping a lot of details (for instance, memory could be considered an input/output device, but it's treated differently than other input/output devices, except when it isn't).

For instance:

Let's say you've just bought a MacBook Air, and your goal is to become master of the machine, to understand how it works on every level.

…

The total of all of this is 79 pages shy of eleven thousand. I neglected to include man pages for hundreds of system utilities and the Xcode documentation. And I didn't even touch upon the graphics knowhow needed to do anything interesting with OpenGL, or how to write good C and Objective-C or anything about object-oriented design, and …

A Complete Understanding is No Longer Possible

And those 11,000 pages exclude documentation on the hardware. For comparison, the Color Computer (my first computer). The TRS-80 Color Computer Technical Reference Manual, which covers the hardware, is 69 pages; the technical reference for the MC6809 (the CPU) is 35 pages; the reference for the MC6821 (an I/O adaptor) is 11 pages; the reference for the MC6847 (the video graphics chip) is 26 pages, and the EDTASM+ manual (the assembler) is 68 pages.

So, in 209 pages, you will know enough to program the Color Computer (assuming you know how to program in assembly to begin with—else tack on another 294 pages for TRS-80 Color Computer Assembly Language Programming for a little over 500 pages).

Even “Project: Wolowizard” is rather insane, requiring knowledge of SS7, IP, HTTP, Solaris, Linux, C, C++, Javascript, Java, Lua, Ruby, Python, SQL, HTML, CSS, XML and that's just the “off-the- shelf” stuff we're using (and all the documentation for that probably exceeds 11,000 pages quite easily; there are probably over 2,000 pages just for SS7 and IP alone)—I'm not mentioning the file formats and protocols we've developed for “Project: Wolowizard” (just the test plan for one component is over 150 pages, and there's at least seven components just on the backend).

It's simple.

Thursday, March 21, 2013

A good idea in theory marred by the terrible reality of practice

I get the feeling sometimes that not enough is written about failed ideas—not bad ideas, the ones that shouldn't be done but the class of ideas that can't be done for one reason or another.

Today I had one such idea, but first, some back story.

Sometime last year, R, who runs the Ft. Lauderdale Office of The Corporation, was listening to me lament about The Protocol Stack From Hell™ and how I had this magical ability to break it by thinking bad thoughts about it (an amazing feat when you consider that the physical computers are several miles away in a data center and that any bad thoughts I had towards it had to travel over a remote command line interface).

R explained that SS7 networks are different than IP networks in that any SS7 endpoint that bounces up and down will effectively be ignored by the rest of the SS7 network (and will typically require manual intervention to re-establish a connection), so was there any way I could keep my testing program up and running.

I countered that I didn't think so, seeing how I had to test the testing program and as such, I had to stop and start the program as I found bugs in my own code while I was using it to find bugs in the code I was paid to find bugs in. R conceeded the point and that was that. I would keep doing what I was doing and if the SS7 stack on the machines needed to be restarted because I borked The Protocol Stack From Hell™ yet again, so be it.

Then today, I read about the reliability of the Tandem computer (link via programming is terrible).

Hi, is this Support? We have a problem with our Tandem: A car bomb exploded outside the bank, and the machine has fallen over … No, no it hasn't crashed, it's still running, just on its side. We were wondering if we can move it without breaking it.

Apocraphal story about a Tandem computer

[One other apocraphal story about the Tandem. About fifteen years ago I worked at a company that had a Tandem computer. It was said that one day a cooling fan for the Tandem computer just showed up at the receptionist's desk with no explaination. When she called Tandem about the apparent mistaken delivery, they said that the Tandem computer had noticed its cooling fan was marginal and had ordered a replacement fan.]

I had an idea.

I can't say exactly what triggered the idea—it just hit me.

The idea was to write a very small, and very simple program that established an SS7 endpoint—a “master control program” if you will. It would also listen in on a named pipe for commands. One command would start the testing program, passing the SS7 endpoint to the testing program to use (another command would be to stop the testing program). The SS7 endpoint that is created is a Unix file descriptor (a file descriptor is an integer value used to refer to an open file under Unix, but more importantly, we have the source code to The Protocol Stack From Hell™ and the fact that the SS7 endpoint is a file descriptor is something I can verify). Open file descriptors are inherited by child processes. Closing a file descriptor in a child process does not close it in the parent process, so the test program can crash and burn, but because the SS7 endpoint is still open in the “master control program” it's still “up” to the rest of the SS7 network.

It's a nice idea.

It won't work.

That's because the user library we use to establish an SS7 endpoint keeps static data based on the file descriptor (and no, it doesn't use the integer value as an index into an array, which would be quick—oh no, it does a linear search, multiple times for said private data—I really need a triple facepalm picture for this) and there's no way to establish this static data given an existing file descriptor.

Sigh.

Friday, March 22, 2013

Preloading Lua modules

I'm tasked with testing the call processing on “Project: Wolowizard.” M suggested, and I concurred, that using Lua to manage the testing scripts would be a Good Thing™. Easier to write and modify the tests as needed. So over the past few years I've written a number of modules to handle the files and protocols used in the project (one side effect: by re-implemeting the code to read/write the various data files helped to verify the specification and flush out architectural dependencies in the binary formats).

But one problem did exist: Not all the systems I need to run the test on have Lua installed, and LuaRocks has … um … “issues” on our Solaris boxes (otherwise, it's not that bad a package manager). So I decided to build what I call “Kitchen Sink Lua”—a Lua interpreter that has the 47 modules required to run the testing scripts (okay, eight of the modules are already built into Lua).

It took some time to wrangle, as some of the modules were written in Lua (so the source needed to be embedded) and I had to figure out how to integrate some third party modules (like LuaCURL) into the build system, but perhaps the hardest bit was to ensure the modules were initialized properly. My first attempt, while it worked (mostly by accident) wasn't technically correct (as I realized when I read this message on a mailing list).

I then restructured my code, which not only made it correct, but smaller and clearer.

#include <stdlib.h>
#include <assert.h>

#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>

/**************************************************************************/

typedef struct prelua_reg
{
  const char   *const name;
  const char   *const code;
  const size_t *const size;
} prelua_reg__t;

/*************************************************************************/

int	luaopen_org_conman_env		(lua_State *);
int	luaopen_org_conman_errno	(lua_State *);
int	luaopen_org_conman_fsys		(lua_State *);
int	luaopen_org_conman_math		(lua_State *);
int	luaopen_org_conman_syslog	(lua_State *);
int	luaopen_org_conman_hash		(lua_State *);
int	luaopen_org_conman_string_trim	(lua_State *);
int	luaopen_org_conman_string_wrap	(lua_State *);
int	luaopen_org_conman_string_remchar (lua_State *);
int	luaopen_org_conman_process	(lua_State *);
int	luaopen_org_conman_net		(lua_State *);
int	luaopen_org_conman_dns		(lua_State *);
int	luaopen_org_conman_sys		(lua_State *);
int	luaopen_org_conman_uuid		(lua_State *);
int	luaopen_lpeg			(lua_State *);
int	luaopen_LuaXML_lib		(lua_State *);
int	luaopen_cURL			(lua_State *);

/***********************************************************************/

	/*---------------------------------------------------------------
	; Modules written in Lua.  The build system takes the Lua code,
	; processes it through luac (the Lua compiler), then creates an
	; object file which exports a character array containing the byte
	; code, and a variable which gives the size of the bytecode array.
	;---------------------------------------------------------------*/

extern const char   c_org_conman_debug[];
extern const size_t c_org_conman_debug_size;
extern const char   c_org_conman_getopt[];
extern const size_t c_org_conman_getopt_size;
extern const char   c_org_conman_string[];
extern const size_t c_org_conman_string_size;
extern const char   c_org_conman_table[];
extern const size_t c_org_conman_table_size;
extern const char   c_org_conman_unix[];
extern const size_t c_org_conman_unix_size;
extern const char   c_re[];
extern const size_t c_re_size;
extern const char   c_LuaXml[];
extern const size_t c_LuaXml_size;

	/*----------------------------------------------------------------
	; Modules written in C.  We can use luaL_register() to load these
	; into package.preloaded[]
	;----------------------------------------------------------------*/

const luaL_Reg c_preload[] =
{
  { "org.conman.env"		, luaopen_org_conman_env		} ,
  { "org.conman.errno"		, luaopen_org_conman_errno		} ,
  { "org.conman.fsys"		, luaopen_org_conman_fsys		} ,
  { "org.conman.math"		, luaopen_org_conman_math		} ,
  { "org.conman.syslog"		, luaopen_org_conman_syslog		} ,
  { "org.conman.hash"		, luaopen_org_conman_hash		} ,
  { "org.conman.string.trim"	, luaopen_org_conman_string_trim	} ,
  { "org.conman.string.wrap"	, luaopen_org_conman_string_wrap	} ,
  { "org.conman.string.remchar"	, luaopen_org_conman_string_remchar	} ,
  { "org.conman.process"	, luaopen_org_conman_process		} ,
  { "org.conman.net"		, luaopen_org_conman_net		} ,
  { "org.conman.dns"		, luaopen_org_conman_dns		} ,
  { "org.conman.sys"		, luaopen_org_conman_sys		} ,
  { "org.conman.uuid"		, luaopen_org_conman_uuid		} ,
  { "lpeg"			, luaopen_lpeg				} ,
  { "LuaXML_lib"		, luaopen_LuaXML_lib			} ,
  { "cURL"			, luaopen_cURL				} ,
  { NULL			, NULL					}
};

	/*---------------------------------------------------------------
	; Modules written in Lua.  These need to be loaded and populated
	; into package.preloaded[] by some code provided in this file.
	;----------------------------------------------------------------

const prelua_reg__t c_luapreload[] =
{
  { "org.conman.debug"		, c_org_conman_debug	, &c_org_conman_debug_size	} ,
  { "org.conman.getopt"		, c_org_conman_getopt	, &c_org_conman_getopt_size	} ,
  { "org.conman.string"		, c_org_conman_string	, &c_org_conman_string_size	} ,
  { "org.conman.table"		, c_org_conman_table	, &c_org_conman_table_size	} ,
  { "org.conman.unix"		, c_org_conman_unix	, &c_org_conman_unix_size	} ,
  { "re"			, c_re			, &c_re_size			} ,
  { "LuaXml"			, c_LuaXml		, &c_LuaXml_size		} ,
  { NULL			, NULL			, NULL				}
};

/*************************************************************************/

void preload_lua(lua_State *const L)
{
  assert(L != NULL);
  
  lua_gc(L,LUA_GCSTOP,0);
  luaL_openlibs(L);
  lua_gc(L,LUA_GCRESTART,0);
  
  /*---------------------------------------------------------------
  ; preload all the modules.  This does does not initialize them, 
  ; just makes them available for require().  
  ;
  ; I'm doing it this way because of a recent email on the LuaJIT
  ; email list:
  ;
  ; http://www.freelists.org/post/luajit/Trivial-bug-in-bitops-bitc-luaopen-bit,4
  ;
  ; Pre-loading these modules in package.preload[] means that they're be
  ; initialized properly through the require() statement.
  ;---------------------------------------------------------------------*/
  
  lua_getglobal(L,"package");
  lua_getfield(L,-1,"preload");
  
  luaL_register(L,NULL,c_preload);
  for (size_t i = 0 ; c_luapreload[i].name != NULL ; i++)
  {
    int rc = luaL_loadbuffer(L,c_luapreload[i].code,*c_luapreload[i].size,c_luapreload[i].name);
    if (rc != 0)
    {
      const char *err;
      
      switch(rc)
      {
        case LUA_ERRRUN:    err = "runtime error"; break;
        case LUA_ERRSYNTAX: err = "syntax error";  break;
        case LUA_ERRMEM:    err = "memory error";  break;
        case LUA_ERRERR:    err = "generic error"; break;
        case LUA_ERRFILE:   err = "file error";    break;
        default:            err = "unknown error"; break;
      }
      
      fprintf(stderr,"%s: %s\n",c_luapreload[i].name,err);
      exit(EXIT_FAILURE);
    }
    lua_setfield(L,-2,c_luapreload[i].name);
  }
}

/*************************************************************************/

Yes, this is the code used in “Project: Wolowizard” (minus the proprietary modules) and is a good example of the module preload feature in Lua. The modules in C are easy to build (the following is from the Makefile):

obj/spc/process.o : $(LUASPC)/src/process.c     \
                $(LUA)/lua.h                    \
                $(LUA)/lauxlib.h
        $(CC) $(CFLAGS) -I$(LUA) -c -o $@ $<

While the Lua-based modules are a bit more involved:

obj/spc/unix.o : $(LUASPC)/lua/unix.lua $(BIN2C) $(LUAC)
        $(LUAC) -o tmp/unix.out $<
        $(BIN2C) -o tmp/unix.c -t org_conman_unix tmp/unix.out
        $(CC) $(CFLAGS) -c -o $@ tmp/unix.c

These modules are compiled using luac (which outputs the Lua byte code used by the core Lua VM), then through a program that converts this output into a C file, which is then compiled into an object file that can be linked into the final Kitchen Sink Lua interpreter.

---

Musings on the Current Work Project Du jour

So I have this Lua code that implements the cellphone end of a protocol used in “Project: Wolowizard.” I need to ramp up the load testing on this portion of the project so I'm looking at what I have and trying to figure out how to approach this project.

The protocol itself is rather simple—only a few messages are defined and the code is rather straightforward. It looks something like:

-- Pre-define these
state_receive = function(phone,socket) end
state_msg1    = function(phone,socket,remote,msg) end
state_msg2    = function(phone,socket,remote,msg) end

-- Now the code

state_receive = function(phone,socket)
  local remote,packet,err = socket:read()
  if err ~= 0 then
    syslog('err',string.format("error reading socket: %s",errno[err]))
    return state_receive(phone,socket)
  end

  local msg,err = sooperseekritprotocol.decode(packet)
  if err ~= 0 then
    syslog('err',string.format("error decoding: %s",decoderror(err))
    return state_receive(phone,socket)
  end

  if msg.type == 'MSG1" then
    return state_msg1(phone,socket,remote,msg)
  elseif msg.type == "MSG2" then
    return state_msg2(phone,socket,remote,msg)
  else
    syslog('warn',string.format("unknown message: %s",msg.type))
    return state_receive(phone,socket)
  end
end

state_msg1 = function(phone,socket,remote,msg)
  local reply = ... -- code to handle this msg
  local packet = sooperseekritprotocol.encode(reply)
  socket:write(remote,packet)
  return state_receive(phone,socket)
end

state_msg2 = function(phone,socket,remote,msg)
  local reply = ... -- code to andle this msg
  local packet = sooperseekritprotocol.encode(reply)
  socket:write(remote,packet)
  return state_receive(phone,socket)
end

Don't worry about this code blowing out the call stack—Lua optimizes tail calls and these effectively become GOTOs. I found this feature to be very useful in writing protocol handlers since (in my opinion) it makes the state machine rather explicit.

Now, to speed this up, I could translate this to C. As I wrote the Lua modules for The Kitchen Sink Lua interpreter, I pretty much followed a bi-level approach. I have a C interface (to be used by C code) which is then mimicked in Lua. This makes translating the Lua code into C more or less straightforward (with a bit more typing because of variable declarations and what not).

But here, I can't rely on the C compiler to optimize tail calls (GCC can, but only with certain options; I don't know about the Solaris C compiler). I could have the routines return the next function to call and use a loop:

while((statef = (*statef)(phone,sock,&remote,&msg) != NULL)
  /* the whole state machine is run in the previous line;

But just try to define the type of statef so the compiler doesn't complain about a type mismatch. It needs to define a function that takes blah and returns a function that takes blah and returns a function that takes blah and returns a function that … It's one of those recurisive type definitions that produce headaches when you think too much about it.

Okay, so instead, let's just have a function that returns a simple integer value that represents the next state. That's easier to define and the main driving loop isn't that bad:

while(state != DONE)
{
  switch(state)
  {
    case RECEIVE: state = state_receive(phone,socket,&remote,&msg); break;
    case MSG1:    state = state_msg1(phone,socket,&remote,&msg); break;
    case MSG2:    state = state_msg2(phone,socket,&remote,&msg); break;
    default:      assert(0); break;
  }
}

Okay, with that out of the way, we can start writing the C code.

Clackity-clackity-clack clackity-clack clack clack clackity-clackity-clackity-clack clack clack clack clack …

Man, that's boring drudgework. Okay, let's just use the Lua code and maybe throw some additional threads at this. I don't think that's a bad approach. Now, Lua, out of the box, isn't exactly thread-safe. Sure, you can provide an implemention of lua_lock() and lua_unlock() but that might slow Lua down quite a bit (there are 62 locations where the lock could be taken in the Lua engine). We could give each thread its own Lua state—how bad could that be?

How big is a Lua state? Let's find out, shall we?

#include <stdio.h>
#include <stdlib.h>
#include <lua.h>
#include <lauxlib.h>

int main(void)
{
  lua_State *L;

  L = luaL_newstate();
  if (L == NULL)
  {
    perror("luaL_newstate()");
    return EXIT_FAILURE;
  }
   
  printf("%d\n",lua_gc(L,LUA_GCCOUNT,0) * 1024);
  lua_close(L);
  return EXIT_SUCCESS;
} 

When compiled and run, this returns 2048, the amount of memory used in an empty Lua state. That's not bad at all, but that's an empty state. What about a more useful state, like the one you get when you run the stock Lua interpreter?

-- ensure any accumulated garbage is reclaimed
collectgarbage('collect')
collectgarbage('collect')
collectgarbage('collect')
print(collectgarbage('count') * 1024)

Okay, when I run this, I get 17608. Eh … it's not that bad per thread (and I do have to remind myself—this is not running on my Color Computer with 16,384 bytes of memory). But I'm not running the stock Lua interpreter, I'm running the Kitchen Sink Lua with all the trimmings—how big is that state?

I run the above Lua code and I get 4683963.

Four and a half megs!

Ouch.

I suppose if it becomes an issue, I could always go back to writing C …

Saturday, March 23, 2013

Preloading Lua modules, part II

Well, four and a half megs per Lua state in the Kitchen Sink Lua interpreter. I thought about it, and I had Yet Another Idea™.

Lua not only has an array for preloaded modules, but an array of functions used to locate and load modules. So the idea I had was to insert two custom load functions—one to search for C based Lua modules, and one for Lua-based Lua modules. The code is pretty much straight forward:

#include <stdlib.h>
#include <string.h>
#include <assert.h>

#include <lua.h>
#include <lauxlib.h>
#include <lualib.h>

/**************************************************************************/

typedef struct prelua_reg
{
  const char   *const name;
  const char   *const code;
  const size_t *const size;
} prelua_reg__t;

/*************************************************************************/

int	luaopen_org_conman_env		(lua_State *);
int	luaopen_org_conman_errno	(lua_State *);
int	luaopen_org_conman_fsys		(lua_State *);
int	luaopen_org_conman_math		(lua_State *);
int	luaopen_org_conman_syslog	(lua_State *);
int	luaopen_org_conman_hash		(lua_State *);
int	luaopen_org_conman_string_trim	(lua_State *);
int	luaopen_org_conman_string_wrap	(lua_State *);
int	luaopen_org_conman_string_remchar (lua_State *);
int	luaopen_org_conman_process	(lua_State *);
int	luaopen_org_conman_net		(lua_State *);
int	luaopen_org_conman_dns		(lua_State *);
int	luaopen_org_conman_sys		(lua_State *);
int	luaopen_org_conman_uuid		(lua_State *);
int	luaopen_lpeg			(lua_State *);
int	luaopen_LuaXML_lib		(lua_State *);
int	luaopen_cURL			(lua_State *);

/***********************************************************************/

extern const char   c_org_conman_debug[];
extern const size_t c_org_conman_debug_size;
extern const char   c_org_conman_getopt[];
extern const size_t c_org_conman_getopt_size;
extern const char   c_org_conman_string[];
extern const size_t c_org_conman_string_size;
extern const char   c_org_conman_table[];
extern const size_t c_org_conman_table_size;
extern const char   c_org_conman_unix[];
extern const size_t c_org_conman_unix_size;
extern const char   c_re[];
extern const size_t c_re_size;
extern const char   c_LuaXml[];
extern const size_t c_LuaXml_size;

const luaL_Reg c_preload[] =
{
  { "LuaXML_lib"		, luaopen_LuaXML_lib			} ,
  { "cURL"			, luaopen_cURL				} ,
  { "lpeg"			, luaopen_lpeg				} ,
  { "org.conman.dns"		, luaopen_org_conman_dns		} ,
  { "org.conman.env"		, luaopen_org_conman_env		} ,
  { "org.conman.errno"		, luaopen_org_conman_errno		} ,
  { "org.conman.fsys"		, luaopen_org_conman_fsys		} ,
  { "org.conman.hash"		, luaopen_org_conman_hash		} ,
  { "org.conman.math"		, luaopen_org_conman_math		} ,
  { "org.conman.net"		, luaopen_org_conman_net		} ,
  { "org.conman.process"	, luaopen_org_conman_process		} ,
  { "org.conman.string.remchar"	, luaopen_org_conman_string_remchar	} ,
  { "org.conman.string.trim"	, luaopen_org_conman_string_trim	} ,
  { "org.conman.string.wrap"	, luaopen_org_conman_string_wrap	} ,
  { "org.conman.sys"		, luaopen_org_conman_sys		} ,
  { "org.conman.syslog"		, luaopen_org_conman_syslog		} ,
  { "org.conman.uuid"		, luaopen_org_conman_uuid		} ,
};

#define MAX_CMOD (sizeof(c_preload) / sizeof(luaL_Reg))

const prelua_reg__t c_luapreload[] =
{
  { "LuaXml"			, c_LuaXml		, &c_LuaXml_size		} ,
  { "org.conman.debug"		, c_org_conman_debug	, &c_org_conman_debug_size	} ,
  { "org.conman.getopt"		, c_org_conman_getopt	, &c_org_conman_getopt_size	} ,
  { "org.conman.string"		, c_org_conman_string	, &c_org_conman_string_size	} ,
  { "org.conman.table"		, c_org_conman_table	, &c_org_conman_table_size	} ,
  { "org.conman.unix"		, c_org_conman_unix	, &c_org_conman_unix_size	} ,
  { "re"			, c_re			, &c_re_size			} ,
};

#define MAX_LMOD (sizeof(c_luapreload) / sizeof(prelua_reg__t))

/*************************************************************************/

static int luaLReg_cmp(const void *needle,const void *haystack)
{
  const char     *key   = needle;
  const luaL_Reg *value = haystack;
  
  return (strcmp(key,value->name));
}

/*************************************************************************/

static int preloadlua_cloader(lua_State *const L)
{
  const char     *key;
  const luaL_Reg *target;
  
  key    = luaL_checkstring(L,1);
  target = bsearch(key,c_preload,MAX_CMOD,sizeof(luaL_Reg),luaLReg_cmp);
  if (target == NULL)
    lua_pushnil(L);
  else
    lua_pushcfunction(L,target->func);
  return 1;
}

/************************************************************************/

static int preluareg_cmp(const void *needle,const void *haystack)
{
  const char          *key   = needle;
  const prelua_reg__t *value = haystack;
  
  return (strcmp(key,value->name));
}

/*************************************************************************/

static int preloadlua_lualoader(lua_State *const L)
{
  const char          *key;
  const prelua_reg__t *target;
  
  key = luaL_checkstring(L,1);
  target = bsearch(key,c_luapreload,MAX_LMOD,sizeof(prelua_reg__t),preluareg_cmp);
  if (target == NULL)
    lua_pushnil(L);
  else
  {
    int rc = luaL_loadbuffer(L,target->code,*target->size,target->name);
    if (rc != 0)
      lua_pushnil(L);
  }
  return 1;
}

/***********************************************************************/

void preload_lua(lua_State *const L)
{
  assert(L != NULL);
  
  lua_gc(L,LUA_GCSTOP,0);
  luaL_openlibs(L);
  lua_gc(L,LUA_GCRESTART,0);
  
  /*---------------------------------------------------------------
  ; modify the package.loaders[] array to include two new searchers:
  ;
  ; 1) scan for a C based module, return luaopen_*()
  ; 2) scan for a Lua based module, return the result of luaL_loadbuffer()
  ;---------------------------------------------------------------------*/
  
  lua_getglobal(L,"package");
  lua_getfield(L,-1,"loaders");
  
  int len = lua_objlen(L,-1);
  
  /*-----------------------------------------------------------------------
  ; insert the two new functions at the start of the package.loaders[]
  ; array---this way, we get first crack at loading the modules and don't
  ; waste time with expensive disk lookups.
  ;----------------------------------------------------------------------*/

  for (int i = len + 2 ; i > 3 ; i--)
  {
    lua_rawgeti(L,-1,i - 2);
    lua_rawseti(L,-2,i);
  }
  
  lua_pushinteger(L,1);
  lua_pushcfunction(L,preloadlua_cloader);
  lua_settable(L,-3);
  
  lua_pushinteger(L,2);
  lua_pushcfunction(L,preloadlua_lualoader);
  lua_settable(L,-3);    
}

And a quick test of the new Kitchen Sink Lua interpeter on this:

-- ensure any accumulated garbage is reclaimed
collectgarbage('collect')
collectgarbage('collect')
collectgarbage('collect')
print(collectgarbage('count') * 1024)

reveals a nice usage of 17,618 bytes—on par with the stock Lua interpreter. What's happening here is that the modules are no longer being shoved into the Lua state regardless of use (and there's one module that accounts for about 3½ megabytes—it's rarely used, but I do need it in some circumstances); they're now loaded into the Lua state as needed.

This also lead me to the concept of compressing the Lua written modules with zlib to save space in the executable (and it does help). I'll leave that code to the reader as an exercise.

Interestingly enough, the only hard part of this was trying to figure out how to insert two elements at the start of an array using the C API of Lua—there is no equivalent function to the Lua table.insert() function. I resorted to checking the source code to table.insert() to see how it was done.

The only other problem I had was debugging the zlib-based version of this code—a typo (two missing characters—sigh) lead me on a multi-hour bug chase.

But it works now, and I've decreased memory usage quite a bit with some few simple changes to the code, which is always nice.

Monday, March 25, 2013

On tail call optimization in certain C compilers

From

Mark Grosberg <XXXXXXXXXXXXXXXXX>

To

Sean Conner <sean@conman.org>

Subject

Tail calls.

Date

Sun, 24 Mar 2013 12:35:18 PM -0500

But here, I can't rely on the C compiler to optimize tail calls (GCC can, but only with certain options; I don't know about the Solaris C compiler). I could have the routines return the next function to call and use a loop:

I haven't verified it but it probably does. It's a pretty easy optimization (compared to what compilers do today) so I'd be surprised if the Sun C compiler doesn't handle this. At least for C code (C++ exceptions can throw some wrinkles in this in some cases).

-MYG

I decided to check. And yes, the Solaris C compiler does support tail call optimizations. So I figured I would play around with this a bit, both under gcc and the Solaris C compiler.

Final results: gcc and the Solaris C compiler both support tail call optimizations (some restrictions apply; void where prohibited; your mileage may vary; results presented are not typical and yours might vary; use only as directed; this program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE; do not taunt Happy Fun Ball).

First off, the number of parameters must match among the functions; the types don't appear to matter that much, just the number. Second, the number (or size) of locally defined variables also matters. I'm not sure what the upper size (or number) for variables is (and it may differ between the two compilers) but it does appear to be a factor. Third, the only safe way to determine if tail call optimizations are being performed is to check the assembly code and check for calls (or, just run it and see if it crashes after a period of time).

So I can, kind of, assume tail call optimization in C code. It'll be something to keep in mind.

Tuesday, March 26, 2013

I wonder what IPO I'll be invited to this time?

I need to check what I used as a certain field in my Lua unix module so I thought I would do this through the Lua interpreter:

[spc]saltmine:~>lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> unix = require "org.conman.unix"
lua: src/env.c:54: luaopen_org_conman_env: Assertion `value != ((void *)0)' failed.
Aborted (core dumped)
[spc]saltmine:~>

What the … um … what's going on with that code?

int luaopen_org_conman_env(lua_State *L)
{
  luaL_register(L,"org.conman.env",env);

  for (int i = 0 ; environ[i] != NULL ; i++)
  {
    char   *value;
    char   *eos;

    value = memchr(environ[i],'=',(size_t)-1);
    assert(value != NULL);
    eos   = memchr(value + 1,'\0',(size_t)-1);
    assert(eos   != NULL);

    lua_pushlstring(L,environ[i],(size_t)(value - environ[i]));
    lua_pushlstring(L,value + 1,(size_t)(eos - (value + 1)));
    lua_settable(L,-3);
  }

  return 1;
}

No! It can't be! Really?

value = memchr(environ[i],'=',10000);

[spc]saltmine:~>lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> unix = require "org.conman.unix"
> 

Yup. It can be! Really!

XXXX! I encountered this very same bug fifteen years ago! The GNU C library, 64 bit version.

Back then, the maintainers of the GNU were making an assumption that any value above some already ridiculously large value was obviously bad and returning NULL, not even bothering to run memchr(). But I was using a valid value.

You see, I have a block of data I know an equal sign exists in. If it doesn't exist, I have bigger things to worry about (like I'm not in Kansas a POSIX environment anymore). But I don't know how much data to look through. And instead of just assuming a “large enough value” (which may be good enough for today, but then again, 640K was enough back in the day) I decided to use a value, that converted to a size_t type, basically translates to “all of memory”.

And on a 32-bit system, it worked fine. But on the GNU C library, 64-bit version, it failed, probably because the maintainers felt that 18,446,744,073,709,551,615 bytes is just a tad silly to search through.

And the only reason I remember this particular bug, is because it apparently was enough to get me invited to the RedHat IPO (it was either that, or my work on porting pfe to IRIX back in the mid-90s).

I did a bit more research (basically—I tried two 64-bit Linux distributions) and I found a really odd thing—glibc version 2.3 does not exhibit the behavior (meaning, my code works on a version released in 2007) but crashes under 2.12 (the code changed sometime between 2007 and 2010).

Sigh. Time to investigate if this is still a problem in 2.17 and if so, report it as a bug …

Thursday, March 28, 2013

The end result was a computer producing vast amounts of nothing very slowly

So, I run this loadtest program on my work computer. It's going, I can see the components I'm testing registering events (via the realtime viewer I wrote for syslogintr). Everything is going fine … and … … then … … … t … h … e … … c … o … m … p … … u … … … t … … … … e … … … … … r … … … … … … … … s … … … … … … … … … … l … … … … … … … … … … … … … … … o … … … … … … … … … … … … w … … … … … … … … … … … … … … … … … … … … … … … … … … … … … … s …

It takes about ten minutes to type and run, but this:

[spc]saltmine:~>uptime
 14:44:20 up 6 days, 23:12, 10 users,  load average: 2320.45, 1277.98, 546.61

was quite amusing to see (usually the load average is 0). Perhaps it was just a tad ambitious to simulate 10,000 units on the work computer (each unit its own thread, running a Lua script—yes, even after the modifications to the Lua interpreter).

Also amusing was this:

[spc]saltmine:~>free
             total       used       free     shared    buffers     cached
Mem:       3910888     640868    3270020          0      35568     185848
-/+ buffers/cache:     419452    3491436
Swap:     11457532     544260   10913272

Yes, eleven gigabytes of memory were shoved out to the disk, so most of the slowless was due to thrashing.

Perhaps I should find some fellow cow-orker's computer to run this on …

Friday, March 29, 2013

A meta configure file

I'm tired of changing the configuration files as I test under different systems. It also seemed silly that I needed to replicate the configuration files for each system (or set of systems). What I wanted was a configuration for the configuration and that notion set off an alarm bell in my head.

The poster child for the “a configuration file for the configuration file” is sendmail, the only program I know of that has a thousand page tome dedicated to describing the configuration file, and it's little wonder when the syntax makes Perl look sane:

# try UUCP traffic as a local address
R$* < @ $+ . UUCP > $*          $: $1 < @ $[ $2 $] . UUCP . > $3
R$* < @ $+ . . UUCP . > $*      $@ $1 < @ $2 . > $3

# hostnames ending in class P are always canonical
R$* < @ $* $=P > $*             $: $1 < @ $2 $3 . > $4
R$* < @ $* $~P > $*             $: $&{daemon_flags} $| $1 < @ $2 $3 > $4
R$* CC $* $| $* < @ $+.$+ > $*  $: $3 < @ $4.$5 . > $6
R$* CC $* $| $*                 $: $3
# pass to name server to make hostname canonical
R$* $| $* < @ $* > $*           $: $2 < @ $[ $3 $] > $4
R$* $| $*                       $: $2

# local host aliases and pseudo-domains are always canonical
R$* < @ $=w > $*                $: $1 < @ $2 . > $3
R$* < @ $=M > $*                $: $1 < @ $2 . > $3
R$* < @ $={VirtHost} > $*       $: $1 < @ $2 . > $3
R$* < @ $* . . > $*             $1 < @ $2 . > $3

It's so bad that there does indeed exist a configuration file for sendmail.cf that's not ugly in a “line noise” way, but ugly in a “needlessly verbose” way:

include(`/usr/share/sendmail-cf/m4/cf.m4')dnl
VERSIONID(`setup for Red Hat Linux')dnl
OSTYPE(`linux')dnl
dnl #
dnl # default logging level is 9, you might want to set it higher to
dnl # debug the configuration
dnl #
dnl define(`confLOG_LEVEL', `9')dnl
dnl #
dnl # Uncomment and edit the following line if your outgoing mail needs to
dnl # be sent out through an external mail server:
dnl #
dnl define(`SMART_HOST',`smtp.your.provider')
dnl #
define(`confDEF_USER_ID',``8:12'')dnl
dnl define(`confAUTO_REBUILD')dnl
define(`confTO_CONNECT', `1m')dnl
define(`confTRY_NULL_MX_LIST',true)dnl
define(`confDONT_PROBE_INTERFACES',true)dnl
define(`PROCMAIL_MAILER_PATH',`/usr/bin/procmail')dnl
define(`ALIAS_FILE', `/etc/aliases')dnl
define(`STATUS_FILE', `/var/log/mail/statistics')dnl
define(`UUCP_MAILER_MAX', `2000000')dnl
define(`confUSERDB_SPEC', `/etc/mail/userdb.db')dnl
define(`confPRIVACY_FLAGS', `authwarnings,novrfy,noexpn,restrictqrun')dnl
define(`confAUTH_OPTIONS', `A')dnl

My thought was (and still is) if a configuration file needs a configuration file, you're doing it wrong. So yes, I was experiencing some cognitive dissonance with writing a configuriation file for a configuration file.

But on second thought, I'm not configuring a single configuration file, I'm configuring multiple configuration files. And no, it's not one configuration file (with changes for different systems) but several configuration files, all of which need to be changed for a different system. And not only changed, but that the changes are consistent with each other—that component P is configured with the IP address of component W, and that W has the IP address of component P. And in that view, I feel better with having a configuration file for the configuration files.

Another factor to keep in mind is that I'm reading in the sample configuration file (they're in XML so parsers are readily available) from the source repository and then making changes (directly to the in-memory DOM) and saving the results to a new file. That way, I'm sure to get the latest and greatest version of the configuration file (they do change, but it's rare and it can be easy to miss, like what happened in production about two weeks ago)—most of the contents are sensible defaults.

If this sounds like I'm trying to justify an approach, I am. I still dislike “configuration files for a configuration file” and I needed to convince myself that I'm not doing something nasty in this case.

Yes, I might be overthinking this a tad bit. But then again, trying to ensure six different components are consistently configured by using a single configuration file might make this approach A Good Thing™.

Monday, April 01, 2013

Don't Bother Reading The IntarWebs Day

I'm going through the websites I typically read each day, and I'm getting alarmed at all the weirdness I'm seeing. Some sites appear down. Some are behaving oddly. One has even been seized by Homeland Security, but it's only when I see one site with a My Little Pony theme do I realize what's going on—it's the annual “Don't Bother Reading The IntarWebs Day” Day.

Tuesday, April 02, 2013

Unintentional performance art

From

HR <XXXXXXXXXXXXXXXXXXXXXXXXX>

To

Sean Conner <XXXXXXXXXXXXXXXXXXXXXXXXX>

Subject

[HR Happy Fun Time Corner] Now Available to Input Individual 2013 Goals

Date

Mon, 1 Apr 2013 14:27:00 -0500

This might be a phishing message and is potentially unsafe. Links and other functionality have been disabled. Click here to enable active content. This is not recommended. Read more about e-mail safety.

---

All [Corporation] Employees,

We are pleased to announce that the [HR Happy Fun Time Corner] system is now available for you to enter your 2013 Goals. Please work with your manager to enter individual goals that are tied to the 2013 Corporate Goals. As a reminder, the 2013 Corporate Goals are listed below …

While I would love to think that The Corporate HR Department has engaged in a subtle form of performance art satirizing the nature of mission statements as part of a April Fools' joke, the sad thing is—I think the HR department is serious, and it's the Exchange Server that is engaging in a subtle form of performance art.

Wednesday, April 03, 2013

A good example of building parsing expressions on the fly

As part of the regression test, one of the ways I check the results is to scan through the log files, checking the output matches what we expect to happen. The components (and there a quite a few) log KPIs in the format:

stats|F0021: fooreq=123 fooabrts=0 fooerrs=1 foo-qs=2/14 foo-latency=1.2/4.2/15.2ms foo-bar-xlats=3

(Each log line from each component contains a unique identifier which makes it easy to grep for particular lines of interest in the logs.)

In the past week, a major change was made in how the KPI are reported (used to be cumulative over the run of the program—now they're reset as they're logged) required a change in how my testing program processed the logs and I decided this was a good time as any to make a dramatic change in how that's done.

Prior, I used the built in Lua string patterns (think regular expressions) for parsing, but as happens, such code tends to be hard to read, and additional processing is required to massage the resulting values from strings to numbers. So I thought, why not use LPeg for my parsing needs?

-- The rest of the code assumes the following has been declared
local lpeg = require "lpeg"

local Ct = lpeg.Ct -- capture results into a table
local Cc = lpeg.Cc -- matches "", return given value as capture
local Cg = lpeg.Cg -- capture results and assign to given name
local R  = lpeg.R  -- range capture
local P  = lpeg.P  -- literal capture
local S  = lpeg.S  -- character set capture

To parse a number, we look for at least one character in the range of “0” to “9” (inclusive), and pass the match to a function that will convert the string to a numeric value, which is returned as the capture:

local num = R"09"^1 / function(c) return tonumber(c) end

For non-integers, we can use the following bit of code—yes, this won't parse all floating point numbers, but we won't need much more than this for what's logged—an optional minus sign, followed by digits, and an optional decimal point plus more digits, which is also translated t a numeric value:

local fnum = (P"-"^-1 * num * (P"." * num)^-1) / function(c) return tonumber(c) end

We also have a few multi-value KPIs, some with two values, and some with three values. These are parsed with the values returned in a table with the values in conveniently named fields:

local lat  = Ct(                        -- latency
                  P"no-data"      *
                  Cg(Cc(0),'min') *
                  Cg(Cc(0),'avg') *
                  Cg(Cc(0),'max') *
                  Cg(Cc(false),'valid')
               )
             +
             Ct(
                 Cg(fnum,'min') * P'/'  *
                 Cg(fnum,'avg') * P'/'  *
                 Cg(fnum,'max') * P'ms' *
                 Cg(Cc(true),'valid')
               )

local cnt  = Ct(                        -- counts
                 P"no-data" *
                 Cg(Cc(0),'avg') *
                 Cg(Cc(0),'max') *
                 Cg(Cc(false),'valid')
               )
             +
             Ct(
                 Cg(fnum,'avg') * P'/' *
                 Cg(fnum,'max') *
                 Cg(Cc(true),'valid')
               )

The only wrinkle here (and it's not much of a wrinkle) is that

foo-qs=no-data foo-latency=no-data

can be logged and we need to take that into account. If no-data is seen, the code returns the following table (if we're parsing the three value KPI; the two value KPI returns something similar):

{
  min = 0.0,
  avg = 0.0,
  max = 0.0,
  valid = false
}

otherwise, the following will be returned:

{
  min = 1.2,
  avg = 4.2,
  max = 15.2,
  valid = true
}

To parse the actual KPIs, I could have done something like:

F0021 = Ct(
            P": " -- I have a reason for starting the parse at the ':'
          * P"fooreq="        * Cg(num,'foofeq')      * P" "
          * P"fooabrts="      * Cg(num,'fooabrts')    * P" "
          * P"fooerrs="       * Cg(num,'fooerrs')     * P" "
          * P"foo-qs="        * Cg(cnt,'foo-qs')      * P" "
          * P"foo-latency="   * Cg(lat,'foo-latency') * P" "
          * P"foo-bar-xlats=" * Cg(num,'foo-bar-xlats')
	)

But I didn't. Not because it's a lot of typing, but because the probability of errors is high because of the repetitious nature of it—I'm repeating the field names twice (look closely and you can see where I messed up). I'd rather not repeat myself, especially since I have multiple lines of KPIs to parse. Ideally, I'd like to specify the name of the field once, something like:

{
  { "fooreq"		, num } , -- might as well include
  { "fooabrts"		, num } , -- the type of the KPI
  { "fooerrs"		, num } , -- while I'm at it
  { "foo-qs"		, cnt } ,
  { "foo-latency"	, lat } ,
  { "foo-bar-xlats"	, num } 
}

And, because LPeg is composable, I can get away with that:

local function mkmatch(list)
  local pattern = P": "

  for i = 1 , #list do
    pattern = pattern
            * P(list[i][1])		-- look for name of field
            * P"="
            * Cg(list[i][2],list[i][1]) -- capture value as name
            * P" "^0			-- optional space at the end
  end

  return Ct(pattern) -- return all captures in a table
end

F0021 = mkmatch {
  { "fooreq"		, num } ,
  { "fooabrts"		, num } ,
  { "fooerrs"		, num } ,
  { "foo-qs"		, cnt } ,
  { "foo-latency"	, lat } ,
  { "foo-bar-xlats"	, num }
}

And there goes the repetition.

Now, why do I start parsing at the colon? Simple, I use the Lua string patterns to quickly check the log lines—that's fairly easy to see what's going on, and from there, I can call the appropriate LPeg pattern to further parse the KPIs.

Friday, April 05, 2013

You thought programming was hard!

My Lovely and Talented Copy Editor has been complaining that my recents posts have gotten rather technical in nature. Well, to that, I reply: yes, but that's because computers are easier to reason about than English spelling (which is, oddly enough, one of the current topics on a computer related mailing list I'm on). For instance:

Four letters cause me disillusion
OUGH makes phonetic confusion
Four simple letters with four pronunciations
Make learning English tough for Asians.

OUGH has no logic, no rule
Or rhyme or rhythm; it will fool
All who struggle to master expression
English may cause thorough depression.

I pour some water in a trough
I sneeze and splutter, then I cough.
And with a rough hewn bough
My muddy paddy fields I plough.

Loaves of warm bread in a row
Crispy crusts and doughy dough.
Now, my final duty to do
And then my chores will all be through.

My lament is finished, even though
Learning this word game is really slow.
It is so difficult, it's very rough
Learning English is really tough.

If a trough was a truff
And a plough was a pluff
If dough was duff
And though was thuff
If cough was cuff
And through was thruff
I would not pretend, or try to bluff,
But of OUGH I've had enough

Enough Is Enough by Rosemary Chen

And another neat gem from the computer related mailing list I'm on: take this sentence:

Show this bold Prussian that praises slaughter, slaughter brings rout. Teach this slaughter-lover his fall nears.

Remove the first letter of each word, and you get a completely different sentence!

How his old Russian hat raises laughter—laughter rings out! Each, his laughter over, is all ears.

English is weird (and here I thought “I” before “E” except after “C”—I guess that makes “weird” weird).

Of course, there exist other poems about the oddities of the English langauge, like this one:

When the English tongue we speak
Why is break not rhymed with weak?
Won't you tell me why it's true
We say sew, but also few?
And the maker of a verse
Cannot rhyme his horse with worse?
Beard is not the same as heard,
Cord is different from word,
Cow is cow, but low is low,
Shoe is never rhymed with foe.
Think of hose and dose and lose,
And think of goose and yet of choose,
Think of comb and tomb and bomb,
Doll and roll and home and some.
And since pay is rhymed with say,
Why not paid with said I pray?
Think of blood and food and good;
Mould is not pronounced like could.
Why is it done, but gone and lone
Is there any reason known?
To sum it up, it seems to me
That sounds and letters don't agree.

But perhaps the most well know is this poem:

Dearest creature in creation
Studying English pronunciation,
I will teach you in my verse
Sounds like corpse, corps, horse and worse.

I will keep you, Susy, busy,
Make your head with heat grow dizzy;
Tear in eye, your dress you'll tear;
Queer, fair seer, hear my prayer.

Pray, console your loving poet,
Make my coat look new, dear, sew it!
Just compare heart, hear and heard,
Dies and diet, lord and word.

Sword and sward, retain and Britain
(Mind the latter how it's written).
Made has not the sound of bade,
Say—said, pay—paid, laid but plaid.

Now I surely will not plague you
With such words as vague and ague,
But be careful how you speak,
Say: gush, bush, steak, streak, break, bleak,

Previous, precious, fuchsia, via
Recipe, pipe, studding-sail, choir;
Woven, oven, how and low,
Script, receipt, shoe, poem, toe.

Say, expecting fraud and trickery:
Daughter, laughter and Terpsichore,
Branch, ranch, measles, topsails, aisles,
Missiles, similes, reviles.

Wholly, holly, signal, signing,
Same, examining, but mining,
Scholar, vicar, and cigar,
Solar, mica, war and far.

From “desire”: desirable—admirable from “admire”,
Lumber, plumber, bier, but brier,
Topsham, brougham, renown, but known,
Knowledge, done, lone, gone, none, tone,

One, anemone, Balmoral,
Kitchen, lichen, laundry, laurel.
Gertrude, German, wind and wind,
Beau, kind, kindred, queue, mankind,

Tortoise, turquoise, chamois-leather,
Reading, Reading, heathen, heather.
This phonetic labyrinth
Gives moss, gross, brook, brooch, ninth, plinth.

Have you ever yet endeavoured
To pronounce revered and severed,
Demon, lemon, ghoul, foul, soul,
Peter, petrol and patrol?

Billet does not end like ballet;
Bouquet, wallet, mallet, chalet.
Blood and flood are not like food,
Nor is mould like should and would.

Banquet is not nearly parquet,
Which exactly rhymes with khaki.
Discount, viscount, load and broad,
Toward, to forward, to reward,

Ricocheted and crocheting, croquet?
Right! Your pronunciation's OK.
Rounded, wounded, grieve and sieve,
Friend and fiend, alive and live.

Is your R correct in higher?
Keats asserts it rhymes with Thalia.
Hugh, but hug, and hood, but hoot,
Buoyant, minute, but minute.

Say abscission with precision,
Now: position and transition;
Would it tally with my rhyme
If I mentioned paradigm?

Twopence, threepence, tease are easy,
But cease, crease, grease and greasy?
Cornice, nice, valise, revise,
Rabies, but lullabies.

Of such puzzling words as nauseous,
Rhyming well with cautious, tortious,
You'll envelop lists, I hope,
In a linen envelope.

Would you like some more? You'll have it!
Affidavit, David, davit.
To abjure, to perjure. Sheik
Does not sound like Czech but ache.

Liberty, library, heave and heaven,
Rachel, loch, moustache, eleven.
We say hallowed, but allowed,
People, leopard, towed but vowed.

Mark the difference, moreover,
Between mover, plover, Dover.
Leeches, breeches, wise, precise,
Chalice, but police and lice,

Camel, constable, unstable,
Principle, disciple, label.
Petal, penal, and canal,
Wait, surmise, plait, promise, pal,

Suit, suite, ruin. Circuit, conduit
Rhyme with “shirk it” and “beyond it”,
But it is not hard to tell
Why it's pall, mall, but Pall Mall.

Muscle, muscular, gaol, iron,
Timber, climber, bullion, lion,
Worm and storm, chaise, chaos, chair,
Senator, spectator, mayor,

Ivy, privy, famous; clamour
Has the A of drachm and hammer.
Pussy, hussy and possess,
Desert, but desert, address.

Golf, wolf, countenance, lieutenants
Hoist in lieu of flags left pennants.
Courier, courtier, tomb, bomb, comb,
Cow, but Cowper, some and home.

“Solder, soldier! Blood is thicker”,
Quoth he, “than liqueur or liquor”,
Making, it is sad but true,
In bravado, much ado.

Stranger does not rhyme with anger,
Neither does devour with clangour.
Pilot, pivot, gaunt, but aunt,
Font, front, wont, want, grand and grant.

Arsenic, specific, scenic,
Relic, rhetoric, hygienic.
Gooseberry, goose, and close, but close,
Paradise, rise, rose, and dose.

Say inveigh, neigh, but inveigle,
Make the latter rhyme with eagle.
Mind! Meandering but mean,
Valentine and magazine.

And I bet you, dear, a penny,
You say mani-(fold) like many,
Which is wrong. Say rapier, pier,
Tier (one who ties), but tier.

Arch, archangel; pray, does erring
Rhyme with herring or with stirring?
Prison, bison, treasure trove,
Treason, hover, cover, cove,

Perseverance, severance. Ribald
Rhymes (but piebald doesn't) with nibbled.
Phaeton, paean, gnat, ghat, gnaw,
Lien, psychic, shone, bone, pshaw.

Don't be down, my own, but rough it,
And distinguish buffet, buffet;
Brood, stood, roof, rook, school, wool, boon,
Worcester, Boleyn, to impugn.

Say in sounds correct and sterling
Hearse, hear, hearken, year and yearling.
Evil, devil, mezzotint,
Mind the z! (A gentle hint.)

Now you need not pay attention
To such sounds as I don't mention,
Sounds like pores, pause, pours and paws,
Rhyming with the pronoun yours;

Nor are proper names included,
Though I often heard, as you did,
Funny rhymes to unicorn,
Yes, you know them, Vaughan and Strachan.

No, my maiden, coy and comely,
I don't want to speak of Cholmondeley.
No. Yet Froude compared with proud
Is no better than McLeod.

But mind trivial and vial,
Tripod, menial, denial,
Troll and trolley, realm and ream,
Schedule, mischief, schism, and scheme.

Argil, gill, Argyll, gill. Surely
May be made to rhyme with Raleigh,
But you're not supposed to say
Piquet rhymes with sobriquet.

Had this invalid invalid
Worthless documents? How pallid,
How uncouth he, couchant, looked,
When for Portsmouth I had booked!

Zeus, Thebes, Thales, Aphrodite,
Paramour, enamoured, flighty,
Episodes, antipodes,
Acquiesce, and obsequies.

Please don't monkey with the geyser,
Don't peel 'taters with my razor,
Rather say in accents pure:
Nature, stature and mature.

Pious, impious, limb, climb, glumly,
Worsted, worsted, crumbly, dumbly,
Conquer, conquest, vase, phase, fan,
Wan, sedan and artisan.

The TH will surely trouble you
More than R, CH or W.
Say then these phonetic gems:
Thomas, thyme, Theresa, Thames.

Thompson, Chatham, Waltham, Streatham,
There are more but I forget 'em—
Wait! I've got it: Anthony,
Lighten your anxiety.

The archaic word albeit
Does not rhyme with eight—you see it;
With and forthwith, one has voice,
One has not, you make your choice.

Shoes, goes, does. Now first say: finger;
Then say: singer, ginger, linger.
Real, zeal, mauve, gauze and gauge,
Marriage, foliage, mirage, age,

Hero, heron, query, very,
Parry, tarry, fury, bury,
Dost, lost, post, and doth, cloth, loth,
Job, Job, blossom, bosom, oath.

Faugh, oppugnant, keen oppugners,
Bowing, bowing, banjo-tuners
Holm you know, but noes, canoes,
Puisne, truism, use, to use?

Though the difference seems little,
We say actual, but victual,
Seat, sweat, chaste, caste, Leigh, eight, height,
Put, nut, granite, and unite

Reefer does not rhyme with deafer,
Feoffer does, and zephyr, heifer.
Dull, bull, Geoffrey, George, ate, late,
Hint, pint, senate, but sedate.

Gaelic, Arabic, pacific,
Science, conscience, scientific;
Tour, but our, dour, succour, four,
Gas, alas, and Arkansas.

Say manoeuvre, yacht and vomit,
Next omit, which differs from it
Bona fide, alibi
Gyrate, dowry and awry.

Sea, idea, guinea, area,
Psalm, Maria, but malaria.
Youth, south, southern, cleanse and clean,
Doctrine, turpentine, marine.

Compare alien with Italian,
Dandelion with battalion,
Rally with ally; yea, ye,
Eye, I, ay, aye, whey, key, quay!

Say aver, but ever, fever,
Neither, leisure, skein, receiver.
Never guess—it is not safe,
We say calves, valves, half, but Ralf.

Starry, granary, canary,
Crevice, but device, and eyrie,
Face, but preface, then grimace,
Phlegm, phlegmatic, ass, glass, bass.

Bass, large, target, gin, give, verging,
Ought, oust, joust, and scour, but scourging;
Ear, but earn; and ere and tear
Do not rhyme with here but heir.

Mind the O of off and often
Which may be pronounced as orphan,
With the sound of saw and sauce;
Also soft, lost, cloth and cross.

Pudding, puddle, putting. Putting?
Yes: at golf it rhymes with shutting.
Respite, spite, consent, resent.
Liable, but Parliament.

Seven is right, but so is even,
Hyphen, roughen, nephew, Stephen,
Monkey, donkey, clerk and jerk,
Asp, grasp, wasp, demesne, cork, work.

A of valour, vapid, vapour,
S of news (compare newspaper),
G of gibbet, gibbon, gist,
I of antichrist and grist,

Differ like diverse and divers,
Rivers, strivers, shivers, fivers.
Once, but nonce, toll, doll, but roll,
Polish, Polish, poll and poll.

Pronunciation—think of Psyche!—
Is a paling, stout and spiky.
Won't it make you lose your wits
Writing groats and saying 'grits'?

It's a dark abyss or tunnel
Strewn with stones like rowlock, gunwale,
Islington, and Isle of Wight,
Housewife, verdict and indict.

Don't you think so, reader, rather,
Saying lather, bather, father?
Finally, which rhymes with enough,
Though, through, bough, cough, hough, sough, tough??

Hiccough has the sound of sup …
My advice is: GIVE IT UP!

The Chaos by Gerard Nolst Trenité

After that, I think understanding call with current continuation with monads is easier to grok than English spelling.

---

Babies with lightsabers

Whatever you do, LOCK UP YOUR LIGHTSABERS! Kids and lightsabers don't mix.

Tuesday, April 09, 2013

Of course it's a catch-22, it's the 1040!

Well, tis' the season, when the IRS crawls out of its hole and charges a ton of money regardless the state of its shadow. So I'm all set to fill out the 1040-EZ when Bunny holds up a 1099-DIV I received because of some stock I own. “I think you should fill out the normal 1040,” she said. “You don't want to get into trouble for not reporting $26.35 of income.”

“I see you learned how to hyperlink your speech,” I said.

“Ha ha.”

“Sigh. I suppose you are right.”

So I'm reading the instructions when I see the following:

Use Form 6251 to figure the amount, if any, of your alternative minimum tax (ATM). Also see the Instructions for Form 6251 to see if you must file the form.

1040 Instructions

Okay, I think, let me check the Form 6251 instructions to see if I need to bother filling this out …

Who Must File

Attach Form 6251 to your return if any of the following statements is true.

1. Form 6251, line 31, is greater than line 34.
2. …

Instructions for Form 6251

Let me get this straight … I have to fill out Form 6251 before I can know if need to fill out Form 6251 …

I always did find it amusing to fill out the full 1040 (but I'm outside the Alternative Minimum Tax income range—that is, if Wikipedia is to be trusted).

It also turns out that the extra $26.35 of extra income did not bump my income to the following line in the 2012 Tax Table, although if it did, I would have owed an additional $14.

Woot‽

Wednesday, April 10, 2013

Three issues with profiling code

I had three issues with profiling code at The Corporation. I manged to solve all three:

1. gprof has issues dealing with multithreaded code—namely, it doesn't handle it very well. I did find a fix for the issue, and while it's a small bit of code, explaining why it works is more than I care to get into at this point in time (it comes down to—if you know why you need this, then you'll understand what it does).

2. gprof always uses the same file for output, gmon.out, regardless of the program name. This is okay when you are working with one program. It's not okay if you are working with multiple programs (right now, I'm testing two programs, running one instance of one, and up to 200 instances of another one).

   But thankfully, there is a fix for this issue as well. You just need to set an environment variable to a unique name, and the output file will be that name with the process-id. Since I run all these programs from a Lua script, it's easy enough to ensure that this environment variable is created.

3. Compiling all these programs to be “profile-enabled.” Yes, I could modify the Makefiles, but some third-party libraries use their own build system (and depending on the programs, libraries and platforms, I have to edit one of sixteen or so files). But I don't always want a “profile-enabled” build (which would mean re-editing a bunch of files). What I want is a simple way to get a “profile-enabled” build, or not.

   Yes, I could take the time to extend the Makefiles and custom build systems to include a “profile-enable” option, but frankly, breaking the build is scaring me off that route. So I decided that for me, the simplest thing that could possibly work would be to write a wrapper around gcc that would add the proper option (and remove conflicting options) to do a “profile-enabled” build (based on an environment variable).

   The code is basically, add the (and remove) the appropriate option(s) to the command line, then run the compiler.

   It beats editing a ton of files.

Tuesday, April 16, 2013

45 years old and can still pack a wallop

The blast, when it came, was loud without being overwhelming. We were close enough that there wasn't more than a quarter second's delay between the flash and the sound, and I felt the warmth of burning kerosene exhaust roll over me. The gas generator spoke with a deep rumbling, topped with a rocket's crackle-crackle-crackle—a sound I'd always thought was just the microphone clipping when listening to recordings of rocket launches. The overall noise of the thing was impressive—probably about as loud as a loud rock concert—but we were far enough away not to need hearing protection. The gas generator produced a long horizontal column of flame, which held steady for the entire test. It was impressive, but it was even more impressive when I reminded myself that in a real F-1 all this fire and noise and smoke was merely used to drive the machinery that fed fuel into the engine for the real fireworks.

Via Instapundit, How NASA brought the monstrous F-1 “moon rocket” engine back to life

I did know know that the F-1 rocket of the Saturn V (of which there are five as part of the first stage) were actually two rockets—the smaller, producing 55,000hp just to drive the pumps of the main rocket (producing approximately 32,000,000hp—remember, that's just one F-1 rocket).

I also did not know that there existed quite a number of F-1 rockets, and as the article states, there's a group working to test fire an F-1 rocket. It's pretty amazing to think that these are still the most powerful rockets every produced and they're over 40 years old.

Simply amazing.

Friday, April 19, 2013

Notes from a service station at mile marker 144 along the Florida Turnpike

Bunny and I are going out of town for the weekend. We're on the Florida Turnpike, and had just turned into the Port St. Lucie Service Plaza only to find a total mob scene.

It wasn't just crowded. It was packed with barely enough room to squeeze inside the main service building. There were literally over a thousand kids milling about the place, over a dozen long distance touring buses, and State Troopers stationed liberally throughout the parking lot.

The kids were all from Broward and Palm Beach high schools, headed north for some large powwow in central Florida and by pure coincidence, all the buses had decided to stop at the same plaza just minutes before we arrived.

The State Troopers had nothing to do with the students—they were there due to a “heightened security level from the Department of Homeland Security” (which to me, is something I would have expected from the Soviet Union, not the United States). Nothing more was said about the threat.

---

First impressions of Chalet Suzanne

So Bunny and I arrive at Chalet Suzanne in lovely Lake Wales, Florida. Bunny made the plans a few months ago, wanting to stay at the historic hotel and eat the six course dinner at its renowned dining room. How renowned? It has its own airstrip!

We arrive and I am … underwhelmed. I heard “Chalet Suzanne,” “4-star dining,” “private airstrip” and I'm picturing something along the lines of Biltmore, a stately building reminiscent of classical French or Swiss architecture, and yet, here we are, at … um … eclectic Disneyesque.

It certainly has character.

But the staff is proving to be very nice and helpful. We're lead to our room, which is above the dining lounge, towards the back of the building.

As we were lead to the room, I couldn't help but feel like I was on the set of Popeye.

I have to keep reminding myself that this has character, although I doubt that the Popeye set was across the water from a modern housing development.

The Governor North Room itself is quite nice, if you don't mind its non-Euclidean geometry. There's the low and uneven ceiling; the slight north-sloping floor of the entry hall and sitting room; the bedroom is about six inches lower than the rest of the suite, and the color, which some would call “pastel green” I would characterize as “institutional mint green” (in other words—it's not a color I would use).

Character, in other words.

As we were getting settled in, one of the staff hand delivered (did I mention how nice the staff is?) a plate of two chocolate truffles. If these were anything to go by, the food should be quite good here.

Our plans for tomorrow include dinner at this renowned 4-star restaurant, plus a possible visit to Isengard, weather permitting.

---

It has “character”

I wouldn't say I hate Chalet Suzanne; it's just that I'm having some cognative dissonance between my expectations, Bunny's enthusiam, and the reality of the place.

In fact, out entire dinner conversation this evening was spent talking about the right word to describe Chalet Suzanne. I offered my description, but Bunny felt the connotations were too negative, and offered a few alternatives. But I felt the connotations of the words she suggested were more negative than the word I used, and countered with some alternatives. Bunny countered that those were too negative and suggested some other ones, which I felt were more negative than what I was feeling.

Yes, our entire dinner conversation was spent trying to compromise on a word that describes Chalet Suzanne, and about the best we can agree on is “character.”

It has “character.”

Saturday, April 20, 2013

Some clarifications about our trip and our room

Bunny is concerned that I've given the impression that she forced me on this trip to Chalet Suzanne, but that is far from the truth. While she did come up with the idea, she did ask me if I wanted to go and I said yes. No coersion involved.

There is one other thing I need to clarify—I said that the bedroom was six inches lower than the rest of the suite—and it is. It's just that there's two steps leading down into the bedroom—it isn't a sheer drop or anything bad like that.

---

A good sign

Bunny and I hit the Chalet Suzanne Dining Room Lounge for breakfast.

The Dining Room Lounge is a multi-level facility. Really, each room is practically its own level, usually a few steps up or down as you work your way through the place. The decor reeks of a high-class restaurant and the staff is, as always, exremely nice. We're lead through several rooms (and levels) to our table, overlooking the lake (and incidentally, right below our room—yes, we didn't have that far to walk); jazz music barely audible in the background. Our waitress handed us the breakfast menu, a piece of slate just slightly smaller than an iPad with that day's selection.

Bunny and I both ordered the pancakes and bacon.

The plates came with three stacks of pancakes. The pancakes were roughly dollar coin sized, perhaps three, four inches across, and less “pancake” and more “crêpe”—very thin, nearly translucent. And excellent, some of the best pancakes I've had.

And the bacon? Well, it's bacon. Bacon is always good. If this is the level of food we can expect, then dinner should be well worth the trip.

---

Isengard, a spooky place, and a gang of turtles

The weather permitted.

We saw Isengard!

Okay, it's really Bok Tower, a rather large and imposing musical instrument which is acoustically engineered such that the sound only carries to the immediate vicinity around the tower itself.

We learned that there is an elevator inside, so the carillonneurs don't have to climb twenty flights of stairs to get to the keyboard.

We also learned that there are very few pictures from inside Bok Tower, and the few that do exist (of the first floor) are mostly in black-and-white.

After leaving Isengard, we then found ourselves at a nearby attraction, Spook Hill.

The illusion is pretty good. Stop the car at the white line, place it in neutral and let off the break, and yes, you do start rolling backwards. The perspective makes it appear as if you are moving up the hill, but that's all it is—an illusion. A darned clever one, but only an illusion.

Once back at Chalet Suzanne, Bunny asked if I could take pictures of the turtles in the lake. I cheerfully obliged and started taking pictures of the happily swiming turtles, but then, things turned ugly—they started bum rushing me!

Well, it wan't much of a bum rush—they are turtles after all, and I could easily outpace them back to the safely of the room. But they did rush towards me, sticking their little heads up at me, daring me to defy them and their territory.

Well … something like that.

By then, it was time to dress for dinner anyway …

---

So, how good is the Chalet Suzanne Dining Room Lounge?

Bunny and I arrived at 5:00 pm for our five course dinner. We really didn't have far to go, just outside our room, down some stairs and around the building.

First up, the appetizer. I'm not fond of grapefruit, and a am thoroughly not a fan of chicken livers, so I opted for the Maryland Crab Cakes; Bunny kept with tradition for this course and had the Caramelized Grapefruit and Organic Chicken Liver. The crab cakes were very good, but not quite as excellent as the crab cakes at Cap's Place (which I'm surprised I haven't written about, seeing how getting there involves a boat ride). Bunny loved the Caramelized Grapefruit and Organic Chicken Liver.

Next course, soup, and both of us had the traditional Romaine Soup, which doesn't have any romaine in it (and hasn't had romaine in it since 1957, when the recipe changed so it could be canned and sold). It does however, have spinach and mushrooms. Bunny loved the soup. I didn't care for it. I found it to be a bit heavy in pepper, with a subtle other flavor that I couldn't quite place, but I didn't like. It wasn't repulsive and I could eat it, but for me, the soup that went to the moon could have stayed there with Apollo 15.

Next course, salad, and here, I kept with tradition with the unique Chalet Cæsar Salad, while Bunny bucked tradition and went with Baby Blend Salad. Bunny loved the salad, whereas I … well …

Okay. I have to explain something about my food dislikes. Generally speaking I have a thing about sweet dishes, and I tend to dislike the mixing of savory and sweet. I will admit to not being at all consistent about this (I love bacon and syrup for instance) and I have a larger problem with sweet dishes than I do savory. For instance, don't bother serving me rice pudding as that triggers my gag reflex. I just can't eat rice pudding. But risotto? I love risotto. The thing is, there's no difference in texture (another thing I have—I find some food textures revolting) as both are soupy, but rice pudding is sweet and risotto is savory. Same deal with bread pudding. Instant gag reflex, yet I love the bread in French Onion soup (and Bunny loves pointing out that French Toast is a type of bread pudding, but not in my universe). Again, it's a sweet/savory thing.

Cream cheese—I hate the flavor of it. And the idea of cheese anything in dessert is another thing I find repulsive. Don't get me wrong, I generally like cheese (except for cream cheese), and because I associate “cheese” with “savory,” I don't care for it in desserts.

And finally we get to gelatin. Me, I associate gelatin with dessert. You know, Jell-o. The idea of a sweet dessert-type item in a savory dish I find revolting. And that means aspics. Which reminds me of horrible things from the 70s and …

Yeah.

So the Cæsar Salad sitting before me. The unique Chalet Cæsar Salad. The artichoke in the middle of the salad is unexpected, but okay, I can deal with that. What I couldn't deal with were the two bits of slime on either side of the dish—orange aspic and tomato aspic.

IN A XXXXXXX SAVORY DISH!

Okay, yes, the non-contaminated portions of the Cæsar Salad were very good; perhaps some of the best Cæsar Salad I've had. But I could not wrap my head around the slimy bits of aspic in the dish. For me, they marred an otherwise excellent salad.

Yes, I'm still emotionally scared over the incident.

Anyway, onto the next course, easily the most problematic portion of our dinner—the entrée.

We both ordered the Grilled Buffalo Ribeye, medium for Bunny, medium rare for me. And not being a fan of winter vegetables, I was able to substitute mushrooms for the zucchini.

The entrées arrived. Mashed sweet potatoes. See above about sweet and savory. My buffalo ribeye was cooked fine, but Bunny's was still a bit too raw for her liking. So they took both dishes back. I to get regular mashed potatoes, and Bunny to have her buffalo ribeye placed on the grill for another couple of minutes. The dishes came back and we started to eat.

“Is your steak tough?” asked Bunny. “Because mine is very tough.”

“It's buffalo,” I said. “It's a game animal.”

“Buffalo?”

“Yes,” I said. “Buffalo.”

“This isn't beef?”

“Nope. Buffalo.”

“I thought ‘buffalo’ meant a type of cut!”

“Oh.”

Suffice to say, Bunny did not enjoy the Buffalo Ribeye.

I, on the other hand, found it to be very good.

The mashed potatoes weren't that great (Bunny tried them, liked them, but realized they had cheese. I'm not a fan of cheese in mashed potatoes, prefering a more pure approach of a dash of milk and a bit heavy on the butter).

And the mushrooms … well, let's just say I found the taste I didn't enjoy in the non-romaine Romaine Soup.

Finally, dessert. Bunny had the Crème Brûlée and I the Orange Aspic Pound Cake.

Hey, just because I found the orange aspic on my Cæsar Salad doesn't mean I don't like orange gelatin. It just has to be in the right context—in this case, dessert (yes, I'm weird that way).

And the desserts where very good. But at this point we really couldn't finish them, as we were both stuffed at the end of a five course meal, and passed out in a food coma as soon as we waddled back to our room.

Overall, the food was good; the only major failings were our own expectations on the food (mushrooms and aspic for me; buffalo for Bunny). Was it worth the price we paid? Even had we enjoyed the full meal, I still feel it was too expensive.

---

Notes from an overheard dinner conversation at a 4-star restaurant

“Look, there's an aligator in the lake.”

“I don't see it.”

“Over there.”

“Hmm … looks like a log to me.”

“I've never seen a log move like that before.”

“The wind is making it bob like that.”

“I'll bet you that's an alligator.”

“How about loser pays for dinner.”

“Oh is that another 'gator in the lake again? I swear we had one removed just last week.”

“Ha!”

“I was paying for dinner anyway.”

“I wonder what they eat?”

“The turtles.”

“But it can't eat through the shell.”

“Alligators pretty much swallow their food whole.”

“But what happens to the shell?”

“I guess it comes painfully out the other end. Oh,sorry. Waitress, could we get an extra napkin?”

Sunday, April 21, 2013

Final thoughts on Chalet Suzanne

Aaaaaaannnnnnnnnnnnnnd we're back!

Final thoughts on our stay at Chalet Suzanne—I don't regret staying there. Aside from the non-Euclidean geometry of the buildings and the (in my opinion) odd color schemes, the place was clean, the bed was quite comfortable, and the staff were extemely friendly. The food, overall, was good.

Was it worth the price? I, personally, felt it was more expensive than it should be (and yes, if you expect to stay and eat there, have a fat wallet on hand), but that's me.

Thursday, May 02, 2013

A fit place for an Englifh Factory, along the Hogohegee R., the Head whereof very little known

Via Flutterby is this wonderful site of historical maps. I was checking out A Map of the British Empire in America with the French and Spanish Settlements adjacent thereto (1746) when I came across a Ft. Barnwell in North Carolina, built in 1712.

Ft. Barnwell was named for one John “Tuscarora Jack” Barnwell who fought the Tuscarora in that area in 1711-12. And John “Tuscarora Jack” Barnwell is an ancestor of mine (on my mom's side of the family—she was a Barnwell). How cool is that?

Monday, May 06, 2013

The only people that get rich with “get-rich-quick” schemes are those that are selling the “get-rich-quick” scheme.

From

XXXXXXXXXXXX <XXXXXXXXXXXXXXXXXXXX>

To

sean@conman.org

Subject

Chuck Stebbins

Date

Mon, 6 May 2013 12:57:39 -0400

Sean,

Read your "Tampogo" article while researching Chuck Stebbins. Chuck is now with a company called XXXXXX XXXXX. Actually the name of the company is changing to XXXXXXXXXXX. XXXXXXXXXXXXXXXXXXX is the website.

I've invested some money with this company and awaiting the next step. If you have any insight or interest in Chuck's new business, please let me know.

He's writing in reference to one of four entries about Tampogo. I haven't really given it much thought since I wrote the articles four years ago, and it's interesting to note that the company no longer appears to be around (fancy that).

I can't find any connection between Chuck Stebbins and this “new” company, which is selling an opportunity to sell some diet-aid product. It's not coming across completely as a “multi-level marketing” scam, but some of the numbers the introductory video is showing are misleading.

Okay, the thrust appears to be you “invest” $5,000, and in return you get 10 tablet computers (wouldn't surprise me if they wholesale around $60/piece) for an “in-store interactive display”, 100 units of the diet product, and some guides about shilling selling the product. You place the “in-store interactive displays” in stores (nail salons, doctor offices, car washes(?!)) and split the profits 50/50 with the store.

Okay, sounds straightforward to me. But here are some numbers the introductory video is currently showing (not the full table, but enough to show what's going on with the numbers):

Small POP Reinvestment Model, $5,000 Investment, 50/50 with retail partner

		Month 1	Month 2	Month 3
Reinvestment	12%		900	1,238
POP Units		10	14	19
Sales		15,000	20,625	28,359
Cost of POPs	240	2,400	900	1,238
Gross Income		7,500	10,313	14,180
Network Fee	50	500	688	945
Retail Partner		2,300	4,363	5,998
Your Gross		2,300	4,363	5,998
Labor	Route Help	0	0	1,500
Your Net		2,300	4,363	4,498

Now, the “product” is $50 (okay, $49.95). Month 1 assumes the “starting package,” that you place them all out on day 1, and 30 (or 28, 29, or 31) days later. The numbers are largely consistent, although it helps to look at things slightly differently:

Month 1 revenue on 300 units sold (10 per POP)

Gross Income	15,000
Inventory	-7,500
Network Fee	-500
Cost of POPs	-2,400
Retail Partner	-2,300
Net Income	2,300

That makes it easier to see what is going where.

One issue already—the “starter pack” only comes with 100 units; that's still 200 units short of this projection, and so you need to spend (from what I can determine using these numbers) another … um … $5,000 just to top of the inventory (ouch) [this company also claims that the “starter pack” is worth $11,495, but given the figures from this, it's actually worth around $7,500 unless they really expect the POP units to retail at around $500 a piece—in any case, just looking at the numbers presented in their introductory video just … yeah … not good].

But the real issue I have is with that 12% reinvestment. $900 is not 12% of $2,300 (it's more like 40%!). No, that 12% is based off the gross income minus the inventory (or $7,500). That's before all other expenses! Okay, let's roll that in:

Revised Month 1 revenue on 300 units sold (10 per POP)

Gross Income	15,000
Inventory	-7,500
Network Fee	-500
Cost of POPs	-2,400
Retail Partner	-2,300
“Reinvestment”	-900
Net Income	1,400

Ouch. Okay, now let's look at month 2:

Month 2 revenue on

Gross Income	20,625
Inventory	-10,312
Network Fee	-688
Cost of POPs	-960
Retail Partner	-4,333
“Reinvestment”	-1,238
Net Income	3,095

And already the numbers are in trouble. Not only is that 12% “reinvestment” pre-net, but it's not even enough to support the additional POPs—it's actually $60 short! Also, the sales figure is bogus because I can't make it come out to whole number of units and the number. In fact, the numbers for months two and three are close but not quite right (averaging a bit under 30 units per POP per month).

And that 12% reinvestment figure is criminal, given how they've defined it, and the cost of the POPs across the months is inconsistent (some months what is listed is actually less that what it would really be; other months it's a bit more).

But the biggest problem I see is the end-game. At the end of year 2, they “show” you earning over $1,000,000 a year. But in order to get that, you need to have 332 POPs. That's quite a feat, but maybe doable. But when you start having two, three, ten, people all doing the same thing in the same area?

Um … good luck?

Really, all I'm doing here is applying basic math to the figures given. That, and some common sense (which does seem to be in short supply these days) and personally—this is a business I would give a pass on.

Sunday, May 26, 2013

“Star Trek Into Darkness”

Bunny and I went to see the latest Star Trek movie, “Star Trek Into Darkness,” the latest film in the rebooted Star Trek universe. And overall, we both enjoyed the film. While not a perfect film (unlike the expertly plotted “Star Trek II: The Wrath of Khan”) it is a fun film.

Visually the film was stunning with the alien worlds looking, well, alien, instead of a sound stage filled with foam rocks or the California desert and the use of CGI not completely obvious (due to either a lot of time and effort into the effects, or the frenetic pace of editing, a complaint I had about the previous film). Also, to my relief, less lens flares in this film.

But that's not to say this film was flawless—far from it. The first major problem I had with the film was the tribble. Yes, it wasn't gratuitous—it did serve the plot of the film—but … tribbles? In the original series, they were creatures that multiplied faster than rabbits and infested the entire ship. Here … well, to me, it just seemed like the wrong choice for that particular plot point (yes, this is a spoiler-free review).

Also, this film could have used more Dr. McCoy. Karl Urban is wonderful as Dr. McCoy and it's a shame he doesn't get as much screen time as Captain Kirk or Mr. Spock (on the plus side, they toned down the physical comedy of Scotty and made his alien sidekick tolerable).

Another major complaint—they made too many references to previous Star Trek material, as if the studio just doesn't trust the audience to enjoy the film without using certain Star Trekkian tropes. Besides the obvious tribble, you have McCoy parodying himself when he goes “Damn it man, I'm a doctor, not a XXXXXXX XXXXXXXXXXX!”; violating the Prime Directive (in the opening action sequence); explicitly referencing red shirts and the worst moment in the film that I won't mention (because it totally spoils the film) but you will certainly know it when you see it (and man, you see it coming a parsec away).

But to the film makers' credit, the ending isn't quite the deus ex machina it could have been—it was foreshadowed, so props for that.

Overall, I can recommend it. Just don't think too hard about it (why is the Enterprise underwater? Okay, it's because the plot needs for it to be underwater, but it still wasn't explained in universe) and enjoy it for what it is, a science fiction action movie set in the Star Trek Universe.

Thursday, May 30, 2013

The Escherian Stairwell at RIT

Are you prepared to have your mind blown? Ready? Okay, check out the Escherian Stairwell. And while you are puzzling over that, remember the words of Sherlock Holmes: “that when you have eliminated the impossible, whatever remains, however improbable, must be the truth” (links via GoogleMyFacePlusSpaceBook).

Wednesday, June 12, 2013

RIP Stupid Twitter Trick

One month shy of six years.

That's how long my Stupid Twitter Trick™ has been running.

Then, my Stupid Twitter Trick™ received the following from Twitter:

HTTP/1.0 410 Gone
content-length: 167
content-type: application/json; charset=utf-8
date: Wed, 12 Jun 2013 06:57:00 UTC
server: tfe
set-cookie: guest_id=v1%3A137102022081399817; Domain=.twitter.com; Path=/; Expires=Fri, 12-Jun-2015 06:57:00 UTC
strict-transport-security: max-age=631138519

{"errors": [{"message": "The Twitter REST API v1 will soon stop functioning. Please migrate to API v1.1. https://dev.twitter.com/docs/api/1.1/overview.", "code": 68}]}

Sigh.

It isn't like this is the first time Twitter broke things and I was all set to fix it when I suddenly realized—six years! I've been pushing the same quotes file for six years! I've repeated all the quotes π times (no, seriously—7,743 tweets, 2,464 quotes, means each quote was repeated π times).

So, I've decided to put Silicon Wisdom on hiatus, until I figure out what stupid Twitter thing I want to do next.

---

It was a cheap joke 31 years ago …

As Bunny and I were watching “Airplane II: The Sequel,” I remarked that I couldn't figure out if I was dismayed or thrilled at just how topical the jokes were in a 31 year old film. Was it the Middle Eastern terrorists wandering around the airport, the “naked full-body scanners” or the priest reading “Altar Boy” magazine (all within the first twenty minutes)?

I just don't know …

Thursday, July 04, 2013

Iron Sky

Sure, you can barbeque various critters and watch the fireworks to celebrate the Fourth of July, and yes, Bunny and I did all that. But we also watched the film “Iron Sky.”

What's the film about? Two words: Space Nazis!

Yup, that's right. Nazis from space!

The backstory is that the Nazis escaped from Earth in 1945 and are hiding out on the dark side of the moon. And due to a chance encounter with an American astronaut, they decide it's time to head on back to conquer the Earth.

It's one of those films where you just have to turn off your brain and roll with it. Sure, there are plot holes, but frankly, the entire premise is a plot hole (“How did the Nazis get into space? How do they generate their atmosphere? Where's their industrial base to build all the flying saucers? How can they get to the Earth in less than a day?”) but that's okay, because this is a fun film (Space Nazis! Space dirigibles! Albino African-Americans! Mad scientists! Beautiful daughters of said mad scientists! Flying saucers vs. fighter jets! Fighting the in war room! What's not to love?)

Wednesday, July 10, 2013

Don't think of it as “slow,” think of it as providing you time to experience that Zen moment …

In English: the philosophy of JavaScript (to the extent that it has any philosophy) is that you should not be able to observe what is going on in system memory, full stop. This is so unbelievably out of touch with how real people write mobile applications, I can't even find the words to express it to you. I mean, in iOS world, we don't believe in garbage collectors, and we think the Android guys are nuts. I suspect that the Android guys think the iOS guys are nuts for manual memory management. But you know what the two, cutthroat opposition camps can agree about? The JavaScript folks are really nuts.

Via Hacker News, Why mobile web apps are slow | Sealed Abstract

It's a long article, but well worth the read (it has extensive references to back up its claims) if you do any type of development on smartphones (like we do here at The Corporation). It's mostly about how bad garbage collection is on embedded systems yet how everybody is trying to use a garbage collected langauge on embedded systems (said embedded systems being smart phones).

And the comments at Hacker News are not bad, but this comment has a few sharp observations that some people I know would greatly enjoy:

Some of the platforms are themselves economically untractable for smaller teams. Android is a prime example of that; even if we skip over the overengineering of the native API, the documentation is simply useless. The odds of having a hundred monkeys produce something on the same level of coherence and legibility as the Android API documentation by typing random letters on a hundred keyboards are so high, Google should consider hiring their local zoo for tech writing consultance.

(untitled)

(Although from some of the comments heard at The Ft. Lauderdale Office of The Corporation, the term “overengineering” in reference to Android is an oxymoron.)

Sunday, August 04, 2013

Programs from the past, Part II

I could have sworn I wrote a bit more about a program I wrote while at FAU; specifically, just how long it took to run back then vs now (or rather, a few years ago), but having failed to find any using Google or locally searching the entries on the server, I ended up spending all day yesterday reading through ten years of archives and still not finding anything, I guess I haven't.

Well. [Deep subject. —Editor] [Shut up you! —Sean]

So I wrote a series of programs to search through the domain of the following pair of equations:

x_i+1 = (Ay_i + B) x_i (1 - x_i)
y_i+1 = (Cx_i + D) y_i (1 - y_i)

The intent was to generate a large number of (x y) pairs and plot the results for a given set of values for A, B, C and D. Something like:

The top window shows the resulting chaotic attractor when A is 2.4375, B is 1.5624, C is -0.8659 and D is 4.0 as you loop through the above two equations 15,000 times to generate 15,000 points (all between 0 and 1 for both the x and y axis). And the thing about a chaotic attractor is, the initial starting point (x0 and y0) is largely immaterial (but not quite, as the starting values must be between somewhere between 0 and 1). Any given value for x0 and y0 will result in the same figure being drawn for the given values of A, B, C and D.

The program shown allows you to change the settings of the four controlling variables A, B, C and D by sliding the named boxes horizontally (the botton one allows you to slide multiple variables at the same time—in this case, A and C will slide to the left if you slide the bottom box to the left).

That was fine, but my bosses at the time wanted another view generated—a “map” of the chaotic attractors, if you will. To generate this “map,” first you calculate how many points are generated before you settle into an attractor:

int mainloop(const double A,const double B,const double C,const double D)
{
  double xn,xn1;
  double yn,yn1;
  int    ix;
  int    iy;
  int    count;
  bool   pix[DIM][DIM];	/* have we seen this (xy) pair yet? */
  
  memset(pix,0,sizeof(pix));
  
  for (count = 0 , xn = yn = 0.5 ; count < MAXLOOP ; count++)
  {
    xn1 = ((A * yn) + B) * xn * (1.0 - xn);
    yn1 = ((C * xn) + D) * yn * (1.0 - yn);    
    xn  = xn1;
    yn  = yn1;
    
    /* exit if we're outside the bounds */
    
    if (xn <  0.0) return MAXLOOP;
    if (xn >= 1.0) return MAXLOOP;
    if (yn <  0.0) return MAXLOOP;
    if (yn >= 1.0) return MAXLOOP;
    
    ix  = (int)(xn * DIM);
    iy  = (int)(yn * DIM);
    
    if (pix[ix][iy])
      break;
    pix[ix][iy] = true;
  }
  
  return(count);
}

The assumption—if this returns MAXLOOP, there is no attractor for the given set of values for A, B, C and D; othersise it's an indication of how quickly (or how “showey”) we settle into the pattern.

Then it's a matter of going through a range of values and for the map images, since they're two dimensional, I only sweep through two of the four governing variables over their range:

for (A = -4.0 ; A <= 4.0 ; A += 0.016)
  for (B = -4.0 ; B <= 4.0 ; B += 0.016)
    plot(A,B,count_as_color(mainloop(A,B,-0.8659,4.0));

(count_as_color() just maps a count to a color)

And we end up with an image something like:

In this image, A is the horizontal axis, going left to right from -4.0 to 4.0, and B is the veritcal axis, going, bottom to top, from -4.0 to 4.0; C and D are fixed, at -0.8659 and 4.0 respectively. The red plus in the upper right hand quadrant is the location of the “French horn” attractor above. The black area represents areas that have no chaotic attractor.

But my bosses didn't just want one map—no, they wanted a larger set. So, I generated a series of images as this, each image having a different value for C. And while the map may not change drastically—for instance, if we bump C up a notch to -0.8499:

The resulting chaotic attractor is completely different:

And I generated a ton of maps by looping through values of C from -4.0 to 4.0—500 images in total.

Now, when I did this the first time, twenty years (or a bit more—maybe up to twenty-two years ago) I was doing this on a state of the art Unix workstation, an SGI workstation (perhaps you've seen their commerical?). It cost $30,000 and it took a year to generate the 500 images. Each individual image took some ten hours to generate.

Fast forward to today. On a laptop that is maybe two years old now. I reran the code and was able to generate an entire image (in fact, both of the map images above) in only 8 seconds.

With four cores, that means, on a two year old laptop that might have cost $1,500 (tops) would take 17 minutes what it took me a year to generate twenty years ago.

Oh, and about those eight seconds …

Update on August 5^th, 2013

Oops, I made a slight mistake …

---

About those eight seconds

In my previous post, I mentioned a program that took a year to run twenty-some years ago on a then state-of-the-art workstation could now be done in 17 minutes on a two year old laptop.

What I didn't mention is that the program twenty-some years ago was written in C and the program I ran today was written in Lua.

Yes, computers are so fast these days that a scripting language can out-perform computers from two decades ago.

“Okay,” you say. “But I won't want to wait 17 minutes for my data.”

Okay, fine. I see two options, and let's try the first option and one that most people would do—drop down to C. And yes, that does give us an improvement, an impressive improvement—only 2.5 seconds per frame, and across four cores that means you'll have the results in a little over five minutes. Not that bad.

The other option, and hear me out—is to take our Lua code and run it via LuaJIT, a (pretty much) drop in replacement for Lua that compiles down to native code. Even if it's a bit slower than C, it should still be faster than Lua with no code changes.

So how does LuaJIT fare?

Personally, I was expecting the C version (which I actually wrote first) to be faster, if only buy a little bit, but …

So, here's the C version (which generates a single image):

[spc]saltmine:~/source/play>time ./a.out >/dev/null

real    0m2.483s
user    0m2.470s
sys     0m0.000s

And now the LuaJIT version:

[spc]saltmine:~/source/play>time luajit amap.lua >/dev/null

real    0m0.849s
user    0m0.840s
sys     0m0.000s

And no, that's not a mistake (belive me, I checked and rechecked)—here:

[spc]saltmine:~/source/play>time lua amap.lua >/dev/null

real    0m8.091s
user    0m8.060s
sys     0m0.000s
[spc]saltmine:~/source/play>time luajit amap.lua >/dev/null

real    0m0.856s
user    0m0.850s
sys     0m0.000s

So … um … I can have that data to you in two minutes? Is that fast enough?

On reflection, it makes sense that LuaJIT will outperform C in this case. It's heavily CPU bound and the fact that the main function:

function mainloop(A,B,C,D)
  local pix = {}
  local xn  = 0.5
  local yn  = 0.5
  
  for count = 1 , MAX do
    local xn1 = ((A * yn) + B) * xn * (1.0 - xn)
    local yn1 = ((C * xn) + D) * yn * (1.0 - yn)
    
    xn = xn1
    yn = yn1
    
    if xn <  0 then return MAX-1 end
    if xn >= 1 then return MAX-1 end
    if yn <  0 then return MAX-1 end
    if yn >= 1 then return MAX-1 end
    
    local ix = math.floor(xn * DIM)
    local iy = math.floor(yn * DIM)
    local f  = iy * DIM + ix -- Lua doesn't really do N-dimensional arrays
    
    if pix[f] then return count-1 end
    pix[f] = true
  end
end

can be recompiled per call to take advantage of the paramters. For instance, when A and B are both 0, the first expression then becomes:

local xn1 = xn * (1.0 - xn)

and given that I'm doing 32,768 interations of this (oh, did I fail to mention that? Yes, I'm doing 27,768 more interations than the code did twenty-some years ago) this does save quite a bit of time.

Update on August 5^th, 2013

Oops, I made a slight mistake …

Monday, August 05, 2013

About those eight seconds, part II: mea culpa

The images presented in the previous two posts have been mostly reconstructions. The only piece of software I have from that project twenty-some years ago is the piece that drew the chaotic attractors, and that was specific to the SGI and had to be rewritten (extensively) for a modern system.

The other program—the one that generated the maps? I have no clue as to where that code went. Or rather, why I neglected to keep a copy of it. So I had to basically recreate that program from scratch. The original reconstruction started out as:

int mainloop(const double A,const double B,const double C,const double D)
{
  double xn,xn1;
  double yn,yn1;
  int    ix;
  int    iy;
  int    count;
  bool   pix[DIM][DIM];
  
  memset(pix,0,sizeof(pix));
  
  for (count = 0 , xn = yn = 0.5 ; count < MAXLOOP ; count++)
  {
    xn1 = ((A * yn) + B) * xn * (1.0 - xn);
    yn1 = ((C * xn) + D) * yn * (1.0 - yn);    
    xn  = xn1;
    yn  = yn1;

But since I was setting memory and not plotting points, I needed to be very careful about the values of xn and yn. I remembered that the values needed to be between 0 and 1, so attempt one I had:

assert(xn >= 0.0);
assert(xn <  1.0);
assert(yn >= 0.0);
assert(yn <  1.0);

But those assumptions were proven false rather quickly. Attempt two I decided to print the values of xn and yn and as it happened, very quickly (like on the first pass) the values hit +inf. I interpreted that to mean I could bail early, and thus finished the reconstruction with attempt three:

    if (xn <  0.0) return MAXLOOP;
    if (xn >= 1.0) return MAXLOOP;
    if (yn <  0.0) return MAXLOOP;
    if (yn >= 1.0) return MAXLOOP;
    
    ix  = (int)(xn * DIM);
    iy  = (int)(yn * DIM);
    
    if (pix[ix][iy])
      break;
    pix[ix][iy] = true;
  }
  
  return(count);
}

But the sharp edges in the maps were bothering me. So much so that I rewrote the code to continue the loop if the values of xn and yn were out of range, thinking that maybe they might come back into range.

And yes, they did:

The downside—this revised code now takes a bit longer to run. The LuaJIT version is still faster than the C version, but the difference (on average 1 minute for the LuaJIT version; 1 minute, 30 seconds for the C version) isn't as jaw-droppingly great.

Sigh—it's not quite as fast as I made it out to be.

Tuesday, August 06, 2013

Programs from the past, Part III: It's a numbers game

I'm still not done yet. When last we left off, the recreation of a project written twenty-some years ago wasn't as fast as I thought. But as I shall reveal, there's a bit more lurking here than I realized.

For a fair comparison, I actually have two versions of a program that generates those map files:

One version in C and one in Lua (previously, the recreated programs just spit out numbers that were then converted to an image with a second program). And here's how long it took to generate the results:

Timings to generate a map file

Version	Timings
Original program	~36,000s
Lua	1,096s
C	79s
LuaJIT	69s

(remember, the “Original program” was written twenty-some years ago and ran on a 32-bit 33MHz machine; the programs I just wrote are running on a 64-bit 2.8GHz machine)

My initial mistake in recreating this program was to toss out values that exceeded a range and from that, I got fantastic timings, but I was throwing out too much, as can be seen here:

The “too-trimmed” version came about because I looked at some of the intermediate results and saw infinities (well, the IEEE 754's concept of infinity) and at that point, any operations done on “infinity” is “infinity.”

But my mistake was thinking that all values that fell out of range would end up being “infinite.” Not all did, and some even fell back into range.

But I was still troubled by the infinite results. So, why not explicitely check for “infinity?” In Lua/LuaJIT:

if xn == math.huge or xn == -math.huge then return MAX end
if yn == math.huge or yn == -math.huge then return MAX end

And in C:

if (xn ==  HUGE_VAL) return MAX;
if (xn == -HUGE_VAL) return MAX;
if (yn ==  HUGE_VAL) return MAX;
if (yn == -HUGE_VAL) return MAx;

(for those curious, HUGE_VAL is defined in math.h.)

Adding those lines makes all the difference:

Updated timings to generate a map file

Version	Timings
Original program	N/A
Lua	16.25s
C	2.60s
LuaJIT	1.25s

It's nice to see LuaJIT still beating the snot out of C, and yes, I was able to generate a full set of maps, just like I did twenty-some years ago, in under three minutes (remember, I'm running the code across four CPUs).

But now I'm upset, because checking for “infinity” was something I didn't do twenty-some years ago, and now I'm thinking, what if I had? Could that simple check, had I known about it, cut the run time of a year down to a month?

I can't blame the university for not offering a class on floating point arithmetic, because it did! And worse, I took the class! (when I entered FAU as a freshman, I knew that the first class for the Computer Science degree involved Fortran. I found the first class that had “Fortran” as part of the title and signed up for it, only to spend find the lectures spending more time on the horrors of floating point arithmetic and very little time on programming. It turned out I took the wrong class; what I signed up for was a Fortran class taught out of the Math Department (the very one I worked for when I wrote these programs twenty-some years ago) for mathematicians. When I discovered the mistake, I was able to get out of the class without issue, but only becuase I was actually doing quite well. In my defense, I was a freshman and didn't have my act together; but FAU didn't exactly have a Computer Science Department at the time, so it didn't have its act together either!). It never dawned on me to even check the intermediate results and bail early.

Sigh.

Thursday, August 08, 2013

World War II according to Hollywood

A wonderful explanation of World War II as done by Hollywood. Indiana Jones keeping the Ark of the Covenant away from the Nazis, or the Japanese forcing Obi-Wan to build a bridge—it's all there.

---

Programs from the past, Part IV: There Ain't No Such Thing As the Fastest Code

The point I want to make, though, is that the biggest optimization barrier that Terje faced was that he thought he had the fastest code possible. Once he opened up the possibility that there were faster approaches, and looked beyond the specific approach that he had so carefully optimized, he was able to come up with code that was a lot faster. Consider the incongruity of Terje's willingness to consider a 5 percent speedup significant in the light of his later near-doubling of performance.

Michael Abrash, Zen of Code Optimization

I'm probably belaboring the inanimate equus pleonastically, but this isn't the first time I've done so, so why stop now?

As was pointed out to me, I missed an obvious optimization in the map-generation program. That small change drastically changed the runtime of the C version of the program:

Latest timings to generate a map file

Version	New Time	Previous Time
Lua	15.50s	16.25s
LuaJIT	0.65s	1.25s
C	0.22s	2.60s

Some other testing revealed that I can now generate in 37 seconds (remember, four cores running a stupidly parallel problem) that twenty-some years ago took a year to do—generate 500 map files.

And speaking of 37 seconds … thanks to the Classic Computers Talk Mailing List, I was able to run this latest code on a nearly identical SGI machine that I used twenty-some years ago—a 33MHz R3000 running IRIX 4.0.5.

Back then, the generation of one map took around ten hours, but I had missed a few optimizations that could have help speed up the program. And yes, it's true—the optimizations I've since added, plus using a high optimization setting on the C compiler, resulted in a map file being generated in 37 seconds.

On a twenty-some year old 33MHz machine.

Head. Desk.

The entire program run, which took a year, could have been done in less than 5¼ hours!

I think it's time to stop belaboring this inanimate equus.

Thursday, August 15, 2013

Nobody expects the Sleep Paralysis Inquisition!

I had fallen asleep at the hotel (oh yes, Bunny and I had to abandon Chez Boca for the night because of plumbing issues—no damage, but no usage of either bathroom) when it felt like Bunny slid into bed behind me. I cracked open an eye, which was tough, as I was very tired, and yet, there she was, in the other bed!

But I swore I could have felt someone—

Nay! What Devilry is this? That was when I felt the bed moving unnaturally behind me, and lo, above my head, jutting out just far enough for me to see, was the metal leg of the bed looming. Further more, said Devilry had paralyzed me.

I attempted to scream. “BUNNY! BUNNY!” But all that was coming out was “Bunny. Bunny.”. But she was unmoving, still consumed by her book. From what I could see, all she was hearing was “Zzzzzzzzzzz.”

I just knew the bed was gyring and gimbling behind me, waiting, plotting, to suck me into the Abyss. I needed to flee. Struggling against paralysis, I was able to stand up, but the bed covers were smothering me, entangling me as I attempted to move. I fell flat, face first onto the floor, which surprisingly, did not hurt. But it did knock the breath out of me and I passed out.

When I came to, I was still in the bed. Bunny was still reading her book, having known not of my plight with the Hellmouth forming behind me, nor of my failed attempt to escape it.

It was then I realized the Devilry for what it was—sleep paralysis. I may have to rethink what triggers my sleep paralysis attacks, since I was not napping when this happened.

---

Apparently, I have products!

From

"Dr. Glidden Kennedy" <cucdtk.tp@hoabinh.gov.vn>

To

(Obviously not me) cucdtk.tp@hoabinh.gov.vn

Subject

Having gone through your listed products

Date

Thu, 15 Aug 2013 20:48:31 +0700 (ICT)

Good Day ,

Having gone through your listed products, we offer great interest to do a purchase agreement with your company. Please get back to us with the following information Please Quote.

1. Prices FOB
2. Payment terms
3. Delivery Period
[There is no 4. Spoons! –Sean]
5. Specified delivery date assuming from the Date of Order.

Please your quick reply will be highly appreciated to our private Email (ptamex412@yahoo.com)

Regards,
Mr. Woody Allen
Purchasing Operations Manager.
PT AMEX Corporation

I have listed products to offer?

Really? [Yes, your Amazon affiliate links. –Editor]

Oh, yeah.

But that aside, what's the angle here? I mean, random spamming of companies looking to buy unspecified items? Um … perhaps inserting themselves as middle men, advertising your products as theirs, passing on the orders they get to you to fulfill, and charging their “customers” more money?

Perhaps?

Either that, or Mr. Allen is really desperate for money.

And assuming he is, Mr. Allen, here are my answers:

1. FOB EXW Boca Raton (Incoterms 2010).
2. Cash up front, no credit. Payable as US dollars or an equivalent amount in gold or silver.
3. Please allow four to six weeks for delivery.
4. We don't talk about Flight Club (think happy thoughts—I'm expecting only four people to get this).
5. Um, what part of “Please allow four to six weeks for delivery” did you not understand?

That should work.

Wednesday, September 11, 2013

Please, Mr. Watterson, take this idea and run with it

There are a few strips that show it being a viable idea …

Please?

Friday, Debtember 20, 2013

The silent chatter box

I'm sitting in a waiting room, waiting for the mechanics to finish changing the oil and rorate the tires. The TV is on, but not the sound [Oops. The sound was actually on. My mistake for letting this mistake get through. —Editor]. After five minutes of the inane chatter of The Twenty-Four Hours of Talking Heads That Masquerade as News Chanel, I had enough, and turned the sound down entirely.

At the time, I was the only one here.

Now there are several other people here and it's amazing. They're intently watching the TV, but not one has complained about the sound being off. No one has made a move to up the sound. They're just consuming the TV as-is.

People are strange.

Monday, Debtember 30, 2013

It's Apple! Upgrading to an iPhone 5 should be easy, right? Right?

I've had my iPhone 4 for several years now. I like it, and aside from the power button being a bit wonky after that zombie clown attack in Atlanta a few months ago [Zombie clown attack? You never mentioned a zombie clown attack! And when were you in Atlanta? —Editor] [Back in early November. Say, I never did mention the whole zombocalypse in Atlanta, did I? —Sean] [No, you never did. —Editor] [Okay, here—have a picture of a zombie clown. —Sean]

[There! Happy now? —Sean] [No. I want to know more. —Editor] [Well, tough! It'll have to wait! —Sean] [Bastard! —Editor], but when it (“it” being the wonky power button, not the zombie clown attack) came to the attention of Bunny, she felt I should get a new iPhone. And lo, it was time for an upgrade anyway.

So she planned this for a combo-Christmas/birthday gift and therefore we found ourselves at the mall, in a Certified Oligarchist Cell Phone Company Reseller (no names, but at one point it's rumored the Certified Oligarchist Cell Phone Company Reseller once sold radios, but I can't seem to verify that) to get an upgrade to the iPhone 5. The iPhone 5c seemed too cheap to me—I mean, this is Apple—you are already paying a premium and for said premium, I want something made of premium materials. Sure, in the 60s plastic might have been a premium material, but this is the 21^st century—-plastic is sooooo 20^th century. The 5s premium was a bit too premium, so that left the plain iPhone 5.

It's here that the entire operation goes pear shaped.

The salescritter helping us, is … how shall I put this … um … in over his head. It's clear he has no idea what he's doing, much less an idea of what store he's actually in. I made my decision, and it takes him something on the order of ten minutes to actually find the item in stock. Then it comes time to actually make the transaction.

First, another salescritter has has to walk him through ringing up the item. Then he has to be walked through transferring the phone number from the old phone to the new phone. Then he has to be walked through printing out the documents. Then he has to be walked through walking us through in signing the documents. Only then, nearly twenty minutes later, does he allow me to actually handle the new iPhone 5.

Only I don't get to handle it. He has to be walked through in cutting open the cellophane around the box, then walked through taking it off and opening the box, then walked through pulling the iPhone 5 out of the box and turning it on, only to realize it isn't charged at all.

Half an hour later, it's charged just enough to turn it on. Another few minutes to explain that I would like for him to transfer all the data from my old phone to the new phone, and then he has to be walked through transfering only the contacts and the two photos I had on my old phone to the new phone.

Apps?

Not transferred.

Notes?

Not transferred.

Bookmarks?

Not transferred.

Contact pictures?

Not transferred.

Sigh.

Yet another half an hour to transfer my existing phone numbers and two pictures from my old phone to the new phone (I actually expected much worse though, and had prepared by backing up my iPhone 4 prior to leaving the house).

And only then did I get a chance to actually see and hold my new iPhone 5.

During the long periods of waiting, I had checked the Certified Oligarchist Cell Phone Company Reseller for new iPhone 5 cases, given that Apple saw fit to make the iPhone 5 about a centimeter longer than the iPhone 4. I would have to give up on the rubberesque case I normally use (and really like, along with the Hipstamatic—sigh) unless I could find a suitable replacement.

And no, I could not find a suitable replacement. But I was asked, multiple times, if I also wanted to get a new case. The first time I informed the salescritter that I wanted to conclude this transaction (“this” being the actual purchanse of the iPhone 5) before worrying about a case. The six subsequent times I had to keep reminding him that I already said "not now." (granted, having worked at the Certified Oligarchist Cell Phone Company Reseller many many years ago before it became the Certified Oligarchist Cell Phone Company Reseller, I know that salescritters get paid by commission, hence the hard sell, but come on! Three times should be enough of a clue—sheesh!).

I do know we were at the Certified Oligarchist Cell Phone Company Reseller for over an hour, maybe an hour and a half, before we walked out, vowing never again to return to said Certified Oligarchist Cell Phone Company Reseller (that may or may not have sold radios at one point in history).

I did find a new case for the iPhone 5, but that was after visiting two other stores and paying a nice premium for a leather case.

We spent nearly 2½ hours at the mall, getting an iPhone 5 and a case. Once I got home, the real fun began.

First, an hour to upgrade iOS from 6 to 7 that was mandatory before I could even think of restoring the backup. Then another hour trying to get the proper backup restored on the phone. It took that long to realize that my new iPhone had not only a different “name” than my old iPhone, but that iTunes keeps the backups from different devices in different locations and would default to offer backups only from the plugged in device. The UI is a bit unclear on that (also, iTunes is now under the impression that I now have six iPhones, two iPads and an iPod in a pear tree. This just makes the situation all the more merrier. Ho ho ho).

So, nearly five hours to upgrade from an iPhone 4 to an iPhone 5. Boy, I sure am glad that Apple makes this easy to do.

• Current
• Next
• Previous
• First
• Last
• Top
• Home
• About
• Archive
• Search
• Glossary
• Copyright
• Help
• Accessibility

Obligatory Picture

Obligatory Contact Info

• Comments? sean@conman.org

Obligatory Feeds

• RSS Feed
• Atom Feed
• JSON Feed

Obligatory Links

• Flutterby!
• KIRK.is

Obligatory Miscellaneous

• About the Source Code
• Source Code

Obligatory AI Disclaimer

No AI was used in the making of this site, unless otherwise noted.

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.