The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Tuesday, January 01, 2013

Happy New Year

It's a war zone out there, what with the rockets' red glare, the explosions, the crazy neighbor attempting to scare off the old year and in the process, scaring in the new year as well as the neighbors. No way I'm going out there in these conditions. Even Murrow would give pause before entering the mæstrom of fire emanating from across the street. No fool, I (or Murrow).

So, from the relative safely of Chez Boca, I bid you Happy New Year.

At least this year we don't have to worry about the world ending in an Emmerichesque orgy of disaster porn or New Age woo-woo mystical nonsense. Thank <insert deity of your choice> for that!

Wednesday, January 02, 2013

Notes from the Bible Belt

For Xmas Christmas, Bunny gave me a subscription to the Transylvania Times, the newspaper of Brevard, NC (and not a taboid following the exploits of Vlad Ţepeş).

I like Brevard; it has that small town, “Mayberry” feeling. The downtown is easily walkable and has some excellent restaurants despite its small size. And every time I'm there I take the time to buy the local newspaper (published twice weekly) as it's more entertaining than any local newspaper here in Lower Sheol.

For instance, this letter to the Editor:

Who are they? Are you one of them?

Why are they taking God out of our country, out of our schools, off our highways and anywhere else they can?

They took prayer out of our schools. Now our children are being killed.

They stopped parents from correcting our children. They call it abuse.

Yes, I have to remind myself that North Carolina is part of the Bible Belt, so there's some level of craziness to contend with. And except for the highway bit (odd—I don't recall God ever being involved with the US road system) this type of letter is something one would expect from the Bible Belt.

Then we get to the real craziness—

They took Christ out of Christmas by putting Xmas and crossing him out.

On our highways, I see signs, “Deer Xing.” It's taking the cross out of crossing, the cross where Jesus shed his blood to forgive our sins.

The star that shone brightly in the sky, showing where Jesus was born, is not on the top of our Christmas trees. In its place is a ribbon or something else.

…

Um … what?

Wow. Just wow.

First off, the use of “Xmas” for “Christmas” isn't a recent phenomenon, nor is it part of a conspiracy to secularize Christmas. Xmas has a long history going back to 1755; further if you accept “Xp̄es mæsse” as a precedent (circa 1100). And using “X” as shorthand for “Christ” goes back to 1485.

And that's exactly what it was used for—shorthand. “They” took the “Christ” out of Christ a long time ago.

And that bit about deer crossings—funny, I don't recall Jesus dying on the cross for the sins of ungulates. As far as I can tell, the FHWA requires a picture of a deer for a “Deer Xing” sign, but North Carolina might use alternative signage, so blame North Carolina for Jesus' inability to save ungulates.

And then we have the next letter to the editor …

More notes from the Bible Belt

Not all people who live in the Bible Belt are Fundamentalist fruit baskets; no, sometimes they're clueless progressives who slept through civics class.

Why, in violation of the Constitution, has Rosman [Rosman being the only other town in Transylvania County, and is even smaller than Brevard. —Editor] allowed a local church to present a live Nativity scene on public property?

What local official is responsible for enforcing the law of the land in this situation?

The author might be thinking of the “separation of church and state,” but that phrase nevers appears in the Declaration of Independence nor in the Constitution of the United States of America (it comes from a letter written by Thomas Jefferson).

The only strictions placed upon the Federal Government is the First Amendment:

Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof …

and this bit in Article VI:

… no religious test shall ever be required as a qualification to any office or public trust under the United States.

It says nothing about banning religious displays. In fact, until the Fourteenth Amendment, it might have passed Constitutional muster for a state (for example, Utah had it been admitted as a state prior to 1868) to legislate an establishment of religion, under the Tenth Amendment (as the First Amendment only applies to Congress, not a state). The Fourteenth Amendment pushes the restrictions the Constitution placed on the Federal government to the State governments (so there goes any State-level mandated religion) but even then, it still says nothing about a government displaying Nativity scenes.

Knowing nothing more about the “Rosman Nativity Scandal of 2012” than what's presented in the opinion letter, it wouldn't surprise me if the said local church didn't fill out the appropriate forms and pay the appropriate fees to host a public display on public property, allowing the church members to exercise the other part of the First Amendment:

… or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the government for a redress of grievances.

I'm sure that had a group wanted to present a Kwanzaa celebration on public property they could fill out the proper forms and paid the proper fees just like the church members probably did.

Sheesh! It's not like the Constitution is hard to read.

Thursday, January 03, 2013

The Average Human, Part II

Disney asking its female employees to to help design an animated male lead reminded me of the average human, a post I made a few years ago.

Amazingly enough, the Face Research Make an Average page still exists, only with a new crop of pictures.

I thought it might be interesting to compare how averages made with a fresh set of pictures would hold up to the previous set of pictures and remarkedly, they are very similar in nature.

The Average Woman

[The Average Woman in 2007] [The Average Woman in 2013]

The Average Man

[The Average Man in 2007] [The Average Man in 2013]

The 2007 pictures appear first, then then current, 2013 pictures. And then, the average of both men and women to create—

The Average Human

[The Average Human in 2007] [The Average Human in 2013]

I wonder how closer we are to my predictions of digital actors?

Friday, January 04, 2013

Rebooting commercial jetliners

I think my friend Gregory will like these: cold start of an Airbus 320 and a cold start of a Boeing 737 (links via Flutterby).

As I was viewing the Airbus 320 video, I was struck that landing the plane appeared to be vastly easier than starting booting it up! And watching how complicated it was for the planes makes the episode where Picard & Co. reboot the Enterprise seem all the more deus ex machina than it already was.

Saturday, January 05, 2013

It's rough being The Computer Guy

Reason #9—Every Conversation You Have Is Roughly The Same

When the computer guy dares to mention what he does for a living, the typical response is, “I have a question about my home computer”

Or when the computer guy first hears about a widespread problem within the computer network he's responsible for, he can barely begin to assess the problem before a dozen other people call to report the same problem.

Or when the computer guy explains a certain process on a computer to a user who is incapable of retaining the process, he will inevitably need to reinstruct the user of this same process—indefinitely.

Via The Endeavour, 10 Reasons It Doesn't Pay To Be “The Computer Guy”

When I read this, I was taken back to Gregory's rant about the user community (which was more of a rant about people who ask the same questions over and over again).

What really struck home, though, was this comment at The Endeavour:

No thank you. I dropped residential support and told all my businesses we put file servers in their business and we re-image workstations at the first hint of trouble. I also emphasize to businesses that work machines should be work machines.

Those jobs began to actually be worth the time.

It doesn't pay to be the computer guy

Fortunately, my exposure to residential support was brief (back in the mid 90s I worked at a local ISP and I was stuck on the front line of support) and since then, I've been able to obtain jobs where talking to residential customers isn't a concern—it pays to be a programmer.

Sunday, January 06, 2013

The lengths I go to for your hypertext environnment for reading pleasure

One of the many reasons I stopped blogging for so long (a post in May, a few in August, then nothing til January 1st) is that it felt too much like work. And in looking back, I see I've been bitching about this topic for at least ten years now, and sadly, the process I go through making an entry today is the same as it was ten years ago. It's the process of writing raw HTML to be posted on the blog.

At least the actual act of submitting a post to be published by mod_blog (the engine that runs my blog) is automated.

Another issue is that I can't quite type fast enough to get my thoughts out. Couple that with my desire to actually use HTML as it was intended (you know, to provide a hypertext environment for reading) causes me to stumble. It also doesn't help that I'm also thinking about several different things at once [okay, don't forget to link back to the 2002 bitch entry] [Done. —Editor] [also, don't forget to talk about throwww and geeze, there's already thirteen tabs just for this entry alone] [Also done. —Editor]. No wonder my Lovely and Talented Copy Editor has her work cut out for her.

Sure, I could use a simpler markup langauge, like say, Markdown [make sure I get back and make a link here] [Done. —Editor] but I know HTML; why bother learning a half-baked markup language that isn't really designed for a hypertext environment for reading (in my opinion).

Throwww [oh, almost forgot to mention that I found the link at Hacker News] appears to be a system where you can edit the page directly (much like the original idea Tim Berners-Lee had when he wrote the first web browser [and here I would go off and try to find pages to link to about Tim Berners-Lee's ideas about editing webpages directly in the browser, but I don't want to digress and … oh, lost the thought … oh, wait, got it back.] [four, count them, four, tabs to find a suitable link. —Editor] but there doesn't appear to be a way see the rendered results and it appears to use its own half-baked markup language.

Blech.

Distraction Free Writing (also via Hacker News) also uses Markdown (what? Is HTML too damn hard for people to use that we must use a watered down half-baked ill-specified [don't forget to link to the author of Markdown refusing to standardize it] [Done. —Editor] formatting language that isn't nearly as expressive as HTML?) and does have a way of previewing the results (that's nice) but it's hardly distraction free what with the exceedingly loud typing noises it makes (at the time—it seems the author of Distrction Free Writing received enough negative feedback to have fixed that) exceeds the loud typing noises my keyboard [don't forget a link there either] [Done. —Editor] makes.

To compound the problems, a solution I had [link to 2008/12/03.1] [Done. —Editor] no longer works, as sometime over the past year the version of Firefox [yet another link] [Nah, forget the link to Firefox. It caused us enough pain. —Editor] autoupdated itself into oblivion (that is—the last update broke on my Linux system so bad I can't even run it) so now I'm forced to use Firefox 1.5 that came installed on the system (yes yes, I'm running a version of Firefox that existed when the dinosaurs roamed the earth, but then again, anything software related that is older than twenty minutes is as old as the dinosaurs and I'll stop here before I digress on the obessiveness of constant upgrades our industry forces upon us) and thus, the program I used to construct <BLOCKQUOTE>s doesn't like Firefox 1.5.

Crap.

So now it's time to go back through some twenty tabs, adding links to make that oh so nice hypertext environment for reading.

[A total of twenty-three webpages were referenced (not necessarily linked to) in the making of this post for your hypertext environment for reading pleasure. —Editor]

[The Editor here is not to be confused with the Lovely and Talented Copy Editor. –Editor]

Monday, January 07, 2013

I think a more serious question is—why do I know about Zeta Reticuli and the Reptilians from Draco?

As I was taking out the garbage (and yes, this happens very early in the morning) I noticed an unusually bright light in the western sky changing colors, and not moving. The rest of the stars weren't twinkling that much, and besides, I've never seen a twinkling star change colors quite like the light I was observing.

I went back inside, grabbed my old-school telescope (based off a design that Galileo used) but it proved to be a bit difficult to use. Bunny then came outside, wondering what I was looking at with the telescope. I showed her, and we both came back inside for some binoculars.

Well, that didn't resolve the issue any. It still looked like a point of light rapidly changing colors. A few minutes later, it winked out of existence.

Okay, so technically what I saw was an unidentified flying object—an object, in the sky, that was unidentifiable. Was it an actual alien spacecraft from the Zeta Reticuli system? No, I doubt it; everybody knows it's the Reptilians from the Draco constellation that rule us here).

Okay, seriously, it was probably an airplane, headed due west (thus it didn't appear to be moving relative to our position at Chez Boca) that we saw.

Tuesday, January 08, 2013

This day in history

Interesting! Not only is today the birthday of the Supreme Commander of the cleanest race on the planet, the Sagacious Leader of the Democratic People's Republic of Korea, Kim Jong Un, but it is also the feast day of Our Lady of Prompt Succor, the day a UFO landing left phystical evidence (France, 1981), and the only time in the history of the United States our debt was fully paid off (back in 1835!).

I'm enjoying a lovely cup of tea

[Loose-leaf tea, baby!]

For some strange reason (I doubt it's to celebrate Kim Jong Un's birthday) I received a beautiful tea mug with infuser, and several ounces of Golden Monkey tea, which is quite good.

Wednesday, January 09, 2013

Stupid Twitter Tricks III

The lack of activity here has been so severe, I didn't notice that the Twitter feed stopped showing up in the Obligatory Sidebar™. It took some time to track down the issue, and it appears that the Twitter API changed locations (and I find it weird that APIs can now refer to URLs, but perhaps that's my old-school mentality showing). At least it was an easier fix than the last time Twitter puked.

Sigh.

Thursday, January 10, 2013

I reject your reality and substitute my own!

REMOVING her ex-husband from more than a decade of memories may take a lifetime for Laura Horn, a police emergency dispatcher in Rochester. But removing him from a dozen years of vacation photographs took only hours, with some deft mouse work from a willing friend who was proficient in Photoshop, the popular digital-image editing program.

Like a Stalin-era technician in the Kremlin removing all traces of an out-of-favor official from state photos, the friend erased the husband from numerous cherished pictures taken on cruises and at Caribbean cottages, where he had been standing alongside Ms. Horn, now 50, and other traveling companions.

“In my own reality, I know that these things did happen,” Ms. Horn said. But “without him in them, I can display them. I can look at those pictures and think of the laughter we were sharing, the places we went to.”

“This new reality,” she added, “is a lot more pleasant.”

…

After her father died several years ago, Theresa Newman Rolley, an accountant in Williamsport, Pa., hired Wayne Palmer, a photographic retoucher, to create a composite portrait of the two of them because she had no actual one of them together.

That photograph—of a moment that never happened—now hangs in her living room. It still brings tears to her eyes, she said.

“It's the only picture of my dad and me together,” Ms. Rolley said, adding, “If the only reason I can get one is cropping it in, it still means the same to me.”

I Was There. Just Ask Photoshop.

I remember back in FAU the drawing instructor told us to draw what we saw half the time, and the other half, told us to ignore any distracting details that didn't pertain to the subject at hand. So in a sense, we were manipulating reality as we saw it.

Also back at FAU, the photography instructor (and this was back before everything turned digital so we were using 35mm cameras, and even then we weren't photographing reality. If we were, we wouldn't be using black-and-white film (real life is in color these days) has us manipulating reality. We had the power to adjust the shutter speed (a fast shutter to freeze the action; a slow shutter to convey speed of motion), the f-stop (a low setting to blur the foreground and background around the subject; a high setting to keep everything in focus) and even the sensitivity of the film (a slow film speed would produce super crisp pictures but required tons of light; a fast film speed could do wonders in low light but the results are grainy); all “adjustments” we could do, in camera, to modify “reality” (and when you get to color—then you have the ability to manipulate the color temperature; make the scene cool and impersonal, warm and inviting, or totally alien in nature).

And once the film was developed (less development, less contrast, a softer picture; more development, more contrast, a harsher look), you could futher manipulate the resulting images; dodging and burning, cropping, even the paper used to expose the image (matt finish, high gloss), as well as the minor touch-ups with a fine brush and ink to remove any white spots due to dust on the enlarging lens or dark lines due to scratches on the film.

And the stuff mentioned in the article? Just a more modern version of scissors removing that ex-person from your life.

Or a gross violation of reality.

Take your pick.

Because visual representations of reality have always been m anipulated in one way or another.

Friday, January 11, 2013

Did I repeat myself? Did I repeat myself? Did I repeat myself? Let me fix that.

“Did you mean to post four copies of your post?” asked Bunny.

“No, why?”

“Because there are four copies of your post, that's why,” said Bunny.

“How do you link to the post in question like that when you speak?” I asked. She shook her head and walked away. “Bunny?”

So yes, there was a glitch last night. The only way for four copies to show up like that would be for four copies to be sent via email (which is my preferred interface). And in checking the logs, I did indeed see four copies show up:

Jan 11 05:03:58 brevard postfix/local: 084412EB72E4: to=<XXXXXXXXXXXXXXXXXXXXX>, relay=local, delay=3, status=deferred (Command died with signal 6: "/home/spc/web/sites/boston.conman.org/htdocs/boston.cgi --config /home/spc/web/sites/boston.conman.org/journal/boston.cnf --email --cmd new". Command output: *** glibc detected *** double free or corruption (fasttop): 0x09cf8d38 *** )
Jan 11 05:27:10 brevard postfix/local: 084412EB72E4: to=<XXXXXXXXXXXXXXXXXXXXX>, relay=local, delay=1395, status=deferred (Command died with signal 6: "/home/spc/web/sites/boston.conman.org/htdocs/boston.cgi --config /home/spc/web/sites/boston.conman.org/journal/boston.cnf --email --cmd new". Command output: *** glibc detected *** double free or corruption (fasttop): 0x08646d38 *** )
Jan 11 06:00:29 brevard postfix/local: 084412EB72E4: to=<XXXXXXXXXXXXXXXXXXXXX>, relay=local, delay=3394, status=deferred (Command died with signal 6: "/home/spc/web/sites/boston.conman.org/htdocs/boston.cgi --config /home/spc/web/sites/boston.conman.org/journal/boston.cnf --email --cmd new". Command output: *** glibc detected *** double free or corruption (fasttop): 0x0934bd38 *** )
Jan 11 07:07:09 brevard postfix/local: 084412EB72E4: to=<XXXXXXXXXXXXXXXXXXXXX>, relay=local, delay=7394, status=sent (delivered to command: /home/spc/web/sites/boston.conman.org/htdocs/boston.cgi --config /home/spc/web/sites/boston.conman.org/journal/boston.cnf --email --cmd new)

The fact that four showed up showed that mod_blog crashed when exiting. I find it disturbing that I've been running the latest version since the 1st and this is the first time it crashed. Not only that, but it crashed three times in a row then worked.

Such bugs are terrifying.

So, inspired by the crash reports in Redis, I decided to roll my own and it was surprisingly easy given the output (from a test of the code):

CRASH: pid=6103 signal='Aborted'
CRASH: reason='Unspecified/untranslated error'
CRASH: CS=0073 DS=C02E007B ES=007B FS=0000 GS=0033
CRASH: EIP=00B767A2 EFL=00000246 ESP=BFE162F8 EBP=BFE1630C ESI=000017D7 EDI=00CBBFF4
CRASH: EAX=00000000 EBX=000017D7 ECX=000017D7 EDX=00000006
CRASH: UESP=BFE162F8 TRAPNO=00000000 ERR=00000000
CRASH: STACK DUMP
CRASH: BFE162F8:                         C5 78 BB 00 F4 BF CB 00 
CRASH: BFE16300: F4 BF CB 00 00 00 00 00 C0 C6 F0 B7 3C 64 E1 BF 
CRASH: BFE16310: 29 93 BB 00 06 00 00 00 20 63 E1 BF 00 00 00 00 
CRASH: BFE16320: 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16330: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16350: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16360: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16370: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16380: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE16390: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
CRASH: BFE163A0: A7 C4 CB 00 A8 C4 CB 00 F4 BF CB 00 A7 C4 CB 00 
CRASH: BFE163B0: 60 C4 CB 00 D8 63 E1 BF 73 D0 C4 00 F4 BF CB 00 
CRASH: BFE163C0: 4F E7 BE 00 02 00 00 00 B8 C6 54 08 42 00 00 00 
CRASH: BFE163D0: B8 C6 54 08 42 00 00 00 14 64 E1 BF 9C E9 BE 00 
CRASH: BFE163E0: 60 C4 CB 00 B8 C6 54 08 42 00 00 00 32 30 31 34 
CRASH: BFE163F0: A7 C4 CB 00 00 00 00 00                         
CRASH: STACK TRACE
CRASH:        /home/spc/source/boston/build/boston[0x805d3c4]
CRASH:        /home/spc/source/boston/build/boston[0x805daa2]
CRASH:        /lib/tls/libc.so.6[0xbb79b0]
CRASH:        /lib/tls/libc.so.6(abort+0xe9)[0xbb9329]
CRASH:        /lib/tls/libc.so.6(__assert_fail+0x101)[0xbb0e41]
CRASH:        /home/spc/source/boston/build/boston[0x8053f30]
CRASH:        /home/spc/source/boston/build/boston[0x805d0b9]
CRASH:        /home/spc/source/boston/build/boston(ChunkProcessStream+0xc8)[0x805d32e]
CRASH:        /home/spc/source/boston/build/boston(ChunkProcess+0xcb)[0x805d240]
CRASH:        /home/spc/source/boston/build/boston(generic_cb+0x74)[0x8053a68]
CRASH:        /home/spc/source/boston/build/boston(pagegen_days+0x1c4)[0x8057f5c]
CRASH:        /home/spc/source/boston/build/boston(generate_pages+0xd0)[0x8057bfc]
CRASH:        /home/spc/source/boston/build/boston[0x80501bb]
CRASH:        /home/spc/source/boston/build/boston(main_cli+0x1e5)[0x80500cd]
CRASH:        /home/spc/source/boston/build/boston(main+0x133)[0x8051eb3]
CRASH:        /lib/tls/libc.so.6(__libc_start_main+0xd3)[0xba4e93]
CRASH:        /home/spc/source/boston/build/boston[0x804cdbd]

POSIX allows you to register a signal handler with additional contextual information. This contextual information includes the contents of the CPU registers and it's just a matter of finding the structure definition in a header file. Once I have the registers, it's not hard to get the stack dump.

The stack trace was even eaiser than that. It seems that the Linux C library comes with a few functions to do just that—backtrace() collects the return addresses off the stack; backtrace_symbols() will resolve the addresses back to function names and backtrace_symbols_fd() does the same, only it writes the informtaion to a file. I ended up using backtrace_symbols_fd() since I'm executing in the context of a signal handler, and I know that open(), read() and close() are marked as “async-signal-safe” (that is, safe to call in a signal handler) by POSIX, but malloc() and free() are not, and backtrace_symbols() is known to call malloc() (and thus, I would need to call free()) and I don't want to risk a crash in the crash report code. I'm taking a risk because backtrace() and backtrace_symbols_fd() could call non-async-signal-safe functions, but I'm doing as much as I can to get as much context as I can as safely as I can.

The utility of a crash report

Well, that was fast. I installed the new version of mod_blog, went out to dinner with Bunny, and came home to over 80 crash reports, reason: “Address not mapped for object.” Or in other words, “the program tried to access memory that didn't belong to it.”

Bad news—it wasn't every request, just certain ones. Even worse news—the logging server and the web server have a 3′50″ difference in time (that took me about fifteen minutes to realize). I also realized that even with stack dump and trace, there still wasn't enough contextual information to figure out what was going on.

I decided to dump the command line arguments (a hack—I have to store the arguments in a global variable and make crashreport(), a library function, depend upon these global variables, which is bad form) and the environment variables (less of a hack—the system already has these in a global variable). With that, I was able to track down the issue.

The issue—some script kiddies are trying to hit the webpage that creates webpages. This isn't normally an issue, since the POST method requires authentication, but these script kiddies were trying to create entries via GET, and the error that was generating wasn't being properly checked (since that shouldn't happen—oops). And because of that, some unititialized variables were dereferenced and boom! A crash report.

The surprising thing about all this is that Apache not once reported the CGI program crashing. To me, that seems fairly obvious, but apparently not. So there's no telling how long this has been happening (yeah, you can tell I check the logs often, can't you?).

I do need to think about how to handle command line arguments in crashreport(). It's library code and thus, shouldn't rely on global variables. I have to think about this …

Saturday, January 12, 2013

A few notes about yesterday's crashes

I wrote crashreport() in an attempt to find out why glibc was reporting a double free (or memory corruption), so imagine my surprise when I found other crashes happening. I did find the root causes for the crashes yesterday, but I have yet to figure out why the memory corruption happened.

First off, no points to Apache for failing to report the unexpected termination of a child process. I can certainly understand that the Apache developers don't expect anyone to use CGI anymore, and if people do, to use a CGI developed in a scripting language that probably won't core dump. But still, they make the CGI module, and that the program the CGI module executes can be written in anything and hiding the fact that a program crashed due to SIGSEGV or SIGABRT is, to me, inexcusable.

Had Apache logged the crash, I probably would have found the error a few years ago (seriously). The actual crash only happened after the output was generated and sent to the browser, so I never saw anything unusual. And because Apache never said anything about a crash and well … everything is okay, right?

Second, the code path with the crash was in a seldom used code path—specifically, when the addentry.html page was requested. I normally use email to create entries, not the web interface. But it's not like I never use the web interface, but I can safely count on two hands the number of times I've used it over the past thirteen years.

So to say it doesn't get a lot of use is an understatement.

Now, are there features I don't use? Yes. And such code is currently commented out. That code was written at a time when I expected other people might use the codebase, but alas, only one other person ever used mod_blog (only to stop blogging due to personal reasons) and now, as far as I know, I'm the only one who uses this codebase. That doesn't bother me, but it does indicate that I should probably remove the code that I don't use.

But the web interface? I use it just enough to justify its existence in the codebase.

Third, the addtion of command line and evironment variables to the output of crashreport() (and I solved the global variable issues I had) certainly helped with the diagnosis. It revealed a request that would reliably crash the program (the aforementioned addentry.html page) and with a reliable way to crash the program, it's easy to isolate the buggy code (if a bit tedious).

And to tell the truth, the bug has existed since May 26th, 2009, when I made the following commit:

Basically, I rewrote the core blogging engine over the past twelve hours. I still have yet to support adding new entries via the engine, but until I get that fixed, I can add them manually.

only I didn't quite update all the code properly. And since the code path in question isn't executed except when called as a CGI program (I should note that mod_blog can be run from the command line as well), and Apache never logs CGI programs that crash, no wonder I never saw this bug.

Sunday, January 13, 2013

One more final note about the crashes

So far so good—no more crashes since yesterday.

It's also interesting to know that there's a Reddit post on obtaining stack traces on excpetions under Windows, Linux and Mac OS-X. The approach taken is similar to my approach (which I “borrowed” from Redis) with the exception of using a separate signal stack—apparently, Mac OS-X won't point to the right stack when running a signal handler—nice to know.

So, hopefully ,there won't be any more cra—

ha ha, only ki

Monday, January 14, 2013

The significance of this is that you can build parsing expressions on the fly …

I found Meta II to be an interesting approach to parsing, and the closest modern equivilent to that are parsing expression grammars (PEGs), and the easiest one to use I've found is the Lua implementation LPeg.

What's interesting about LPeg is that it isn't compiled into Lua, but into a specialized parsing VM, which makes it quite fast. Maybe not as fast as lex and yacc but certain easier to understand and vastly easier to use.

Let me amend that: I find the re module to be easier to use (which is build on LPeg), as I find this:

local re = require "re"

parser = re.compile [[
	expr		<- term (termop term)*
	term		<- factor (factorop factor)*
	factor		<- number
			/  open expr close

	number		<- space '-'? [0-9]+ space	
	termop		<- space [+-] space
	factorop	<- space [*/] space
	open		<- space '(' space
	close		<- space ')' space
	space		<- ' '?
]]

to be way easier to read and understand than

local lpeg = require "lpeg"

local space    = lpeg.P" "^0
local close    = space * lpeg.P")" * space
local open     = space * lpeg.P"(" * space
local factorop = space * lpeg.S"*/" * space
local termop   = space * lpeg.S"+-" * space
local number   = space * lpeg.P"-"^-1 * lpeg.R"09"^1 * space

local factor , term , expr = lpeg.V"factor" , lpeg.V"term" , lpeg.V"expr"

parser = lpeg.P {
  "expr",
  factor = number
         + open * expr * close,
  term   = factor * (factorop * factor)^0,
  expr   = term   * (termop   * term)^0
}

As such, I've been concentrating on using the re module to brush up on my parsing skills to the point that I've been ignoring a key compent of LPeg—expressions!

Sure, raw LPeg isn't pretty, but as you can see from the above example, it is built up out of expressions. And that's a powerful abstraction right there.

For instance, in mod_blog, I have code that will parse text, converting certain sequences of characters like --- (three dashes) into an HTML entity &mcode;. So, I type the following:


``The name of our act is---The Aristocrats! ... Um ... hello?''

which is turned into

&ldquo;The name of our act is&mdash;The Aristocrats! &hellip; Um &hellip;
hello?&rdquo;

to be rendered on your screen as:

“The name of our act is—The Aristocrats! … Um … hello?”

Now, I only support a few character sequences (six) and that takes 160 lines of C code. Adding support for more is a daunting task, and one that I've been reluctant to take on. But in LPeg, the code looks like:

local lpeg  = require "lpeg"

local base =
{
  [ [[``]] ] = "&ldquo;" ,
  [ [['']] ] = "&rdquo;" ,
  [ "---"  ] = "&mdash;" ,
  [ "--"   ] = "&ndash;" ,
  [ "..."  ] = "&hellip;",
  [ ".."   ] = "&#8229;" ,
}

function mktranslate(tab)
  local tab   = tab or {}
  local chars = lpeg.C(lpeg.P(1))
  
  for target,replacement in pairs(tab) do
    chars = lpeg.P(target) / replacement + chars
  end

  for target,replacement in pairs(base) do
    chars = lpeg.P(target) / replacement + chars
  end
  
  return lpeg.Ct(chars^0) / function(c) return table.concat(c) end
end

Now, I could do this with the re module:

local re   = require "re"
local R    = { concat = table.concat }
local G    = --[[ lpeg/re ]] [[

text    <- chars* -> {} -> concat

chars   <- '`'   -> '&ldquo;'
        /  "''"   -> '&rdquo;'
        /  '---'  -> '&mdash;'
        /  '--'   -> '&ndash;'
        /  '...'  -> '&helip;'
        /  '..'   -> '&#8229;'
        /  { . }

]]

filter = re.compile(G,R)

But the former allows me to pass in an additional table of translations to do in addition to the “standard set” programmed in, for example:

translate = mktranslate {
  ["RAM"]  = '<abbr title="Random Access Memory">RAM</abbr>',
  ["CPU"]  = '<abbr title="Central Processing Unit">CPU</abbr>',
  ["(tm)"] = '&trade;'
}

And I would want this why? Well, I have Lua embedded in mod_blog, so using Lua to do the translations is straightforward. But, now when I make an entry, I could include a table of custom translations for that entry. Doing it this way solves a problem I saw nearly a decade ago.

Tuesday, January 15, 2013

Linux vs. Windows. Two operating systems go in, only one comes out!

It quickly become apparent that Windows has no package management to speak of. I had to actually go directly to software vendors' websites and manually download, unpack, and install software. In some cases, the packages would come as a RAR file, and there was no unrar to extract the files with. The worst thing though, is that Widows wanted me to reboot it every time I installed something new.

I finally got my hardware to work after four hours of research, downloading, and rebooting. By that time, I was already feeling like a slave to the OS.

Via Hacker News, Branko's Thought Dump: On the state of Windows on the desktop

My friend Gregory is a Windows person who has had his share of problems with Linux. Well Gregory, here's the story of a Linux person who's had his share of problems with installing Windows.

Now, to avoid accusasions of being biased, I will also include the views of a Windows user on installing Linux:

When I installed Windows the process worked properly: The installer looked at my computer for a long time, then copied some files and rebooted. Then it had another long ponder before rebooting again. Then it booted into Windows, announced there were critical updates, and asked to boot again. Then once I installed my graphics drivers it wanted another boot. I guess my only complaint is that it stopped to ask about that fourth reboot. That was kind of odd. I mean, why would I say no?

On the other hand, the Linux installer feels like it?s missing parts or something. I ran the installer and it rebooted just once, right into Linux. After that I installed some updates, but the machine still didn?t ask to reboot itself. It just sat there like it was ready for me to start using it. I waited ten minutes just to be sure, but it never did ask to restart.

Eventually I had to bring in the laptop and reboot that, just so I could feel like the job was done properly.

Linux vs. Windows—Twenty Sided

See, I'm being fair here …

Leviathan

Before you could get a building permit, however, you had to be approved by the Zoning Authority. And Zoning—citing FEMA regulations—would force you to bring the house "up to code," which in many cases meant elevating the house by several feet. Now, elevating your house is very expensive and time consuming—not because of the actual raising, which takes just a day or two, but because of the required permits.

Kafka would have liked the zoning folks. There also is a limit on how high in the sky your house can be. That calculation seems to be a state secret, but it can easily happen that raising your house violates the height requirement. Which means that you can't raise the house that you must raise if you want to repair it. Got that?

Roger Kimball: This Metamorphosis Will Require a Permit

Somehow, I get the feeling that we've reached peak government.

Wednesday, January 16, 2013

Life at The Corporation

I've certainly peered deeper into the Abyss of XXX XXX's implementation of SS7 and come to a deeper understanding of the Lovecraftian nature of it.

A comment about my “goal” on my self-evaluation for The Corporation.

I'm finding myself in an odd position at The Corporation.

And it's not a bad thing at all.

I was hired to do performance measurements and testing of “Project: Wolowizard” and my first manager was R, who started (and still runs) the Ft. Lauderdale Office of The Corporation (the main office being in Seattle). My role fell into the “Quality Assurance” department and in 2011, I was transferred to the QA department, and my manager became E, who worked at Corporate Headquarters in Seattle (while I still work in the Ft. Lauderdale Office).

I ended up being the only QA employee in the Ft. Lauderdale Office. But more crucially, I'm the only QA employee doing QA on call processing (the part that happens between cell phones) in the entire company. So my weekly meeting (read: phone call) with the QA team would go something like:

E
So, random QA person, what have you been working on this week?
Random QA Person #1
[Long rambling technobabble about setting up unit tests for a smart phone application.]
E
Very good. Now, other random QA person, what have you been working on this week?
Random QA Person #2
[Even longer rambling technobabble about helping Random QA Person #1 set up unit tests, with a long digression about Ruby programming.]
E
Very good, very good. What about you, yet another random QA person?
Random QA Person #3
[Exposition about the difficulties in testing Random Java Technology for yet another smart phone application.]
Random QA Person #1
[Interjects into what Random QA Person #3 was saying about a possible fix for the difficulties in testing Random Java Technology.]
Random QA Person #3
Okay, I think I'll try that.
E
Wonderful. What about you, some other random QA person?
Random QA Person #4
[SEAN strains to hear Random QA Person #4 over the phone, but Random QA Person #4 mumbles so softly, SEAN cannot make out what Random QA Person #4 is saying, so SEAN spaces out for a few minutes.]
E
…Sean? Are you there?
Sean
[Realizes it's his turn to speak.] Oh yeah … um … this week I've [technobabble about running the latest “Project: Wolowizard” regression test and some of the difficulties in testing the Protocol Stack From Hell™.]
E
Okay … well then … see you all next week.

Nothing against E—E is a very nice person, but no one else on the QA team deals with call processing. Nor do they program in C, C++ or Lua. Java, Javascript and Ruby, yes. But the rest of the QA team in the Corporation deal with applications for cell phones and smart phones; I'm the only one that deals with call processing.

And because of that, it was felt by The Powers That Be™ that I would be better served under a manager in the Ft. Lauderdale Office (this was a few months ago). So now I'm working under J. J's a great guy, but he too, knows nothing about call processing (he's actually managing other, non-call processing cell phone related projects). Except for a meeting every week or two (or like today, for a performance review) I've just been plugging away at testing and staring into the Abyss of The Protocol Stack From Hell™.

It could be worse. I could have daily meetings like my fellow office mates.

(But it's not like I'm the only one there who understands call processing. There are three others, all at the Ft. Lauderdale Office. R, who is a VP in The Corporation and runs the Ft. Lauderdale Office; M, the lead developer who is now enmeshed in writing a smart phone client for “Project: Wolowizard,” and S, who does for Ops (call processing stuff) what I do for QA (call processing stuff) and is probably in a similar situation as me.)

Thursday, January 17, 2013

Texting from the other side of the table

[Development of Wireless Telegraphy. Scene in Hyde Park. 1906]

(Via Violet Impudence)

My dad has often lamented the scene he sees whenever he goes out to a restaurant. It is inevitable that a nearby table of two or more diners will all be sitting in rapt silence, staring into small lit screens of smart phones and not engaging in conversation at all. Or rather, conversation with the fellow diners—for all he knows, they could be communicating with others, maybe at other restaurants.

It's amusing to note that this isn't a modern phenomenon.

Friday, January 18, 2013

Notes from a conversation outside a restaurant

As Bunny is getting out of the car, a kid walks up with a handful of leaflets. “Here,” he says, handing one to Bunny.

Bunny takes it and glances at it. From my vantage point, I can see that it's a leaflet for a psychic reading, presumably at some local establishment. Bunny takes one look at it, and hands it back. “No thank you. Not today,” she said. The kid takes it back, and wanders off.

“You know,” I said, “he should have seen that coming.”

Saturday, January 19, 2013

Trees are freaking awesome!

I did not know that trees create a negative pressure to force water up through their trunks (link via Hacker News commentary on a NewScientist blurb about the limits to tree growth.)

So, what date did October 23, 4004 BC fall on?

Who needs machine readable dates? As far as I can see there are two target audiences for this operation. The first is obviously social applications that have to work with dates, and where it can be useful to compare dates of two different events. An app must be able to see if two events fall on the same day and warn you if they do.

However, as a target audience social applications are immediately followed by historians (or historical, chronological applications). After all, historians are (dare I say it?) historically the most prolific users of dates, until they were upstaged by social applications.

…

Let’s go another eight hundred years back and land just in time to see Hannibal victorious against the Romans at Cannae. This historical battle, sources assure us, took place on 2 August 216 BC. We don’t have a prayer of re-mapping this date to a proleptic Gregorian or a Julian one.

The ancient Roman year had 355 days, and in theory every second year ought to have a so-called intercalary month of 22 or 23 days. The problem was that these months were inserted irregularly, and no chronologist ancient or modern has ever taken the trouble to track down the exact use of the intercalary month. (Besides, the sources are just not there.)

This means that we will never know exactly on which proleptic Gregorian date the battle of Cannae took place. The best we can say is that it took place in high summer; probably in July or August.

…

Before Dionysius introduced his reform, people used the old Roman system, in which every year was named after its two consuls.

After the Romans had discarded their monarchy in 509 BC they were forced to stop using regnal years. They needed a new naming system, and they decided to allow their two chief magistrates, the consuls, to give their names to the year.

Thus, “in the consulate of Cn. Pompeius Magnus and M. Licinius Crassus Dives” is a historically valid alternative to “70 BC.” In fact, BC or AD years may be considered a convenient shorthand for the “semantically” more correct consular years.

Although the consuls lost all political power after Augustus founded the Empire in 27 BC, the title was still given out to aristocrats who’d deserved a plum, as wel as to the Emperor himself, until the office was abolished in 541 AD. The consuls continued to give their names to the year. (In return they were graciously allowed to squander their fortunes on organising circus games.)

Via Hacker News, M aking <time> safe for historians

This is an amazing article (long, but well worth reading) about the difficulties historians have with time. Unfortunately, not only are calendars complicated, but even the concept of time is non-intuitive.

Sunday, January 20, 2013

Leviathan II

Finally, you have to guess what the law is. There is so much “discretion” [3,4] afforded to regulatory agencies that the threat of fines and seizures over bizarre interpretations of the law by a Carmen Ortiz- style ambitious regulator is never far from your mind. Example [5]:

[Newsweek:] What exactly would constitute a “medical claim?” Would pointing people to medical research papers [qualify]?

[FDA]: It depends. There are rules as to how one can do that … Those rules are actually worked out pretty well, and they just would need to make sure they’re staying within the rules.

[Newsweek:] Are those rules on the Web?

[FDA]: I don’t know where the policy is. I would have to get it for you. It’s an agencywide policy. I would have to find it for you. And it won’t be that easy for people to follow it …

…

Look, finally, how they claim in an official court filing against family farms producing raw milk that you have "No Generalized Right to Bodily and Physical Health" [12], where they approvingly cite the case of Cowan vs. US, where a terminal cancer patient was denied access to experimental medication, denied the right to opt-out of the FDA:

There is No Generalized Right to Bodily and Physical Health.

Plaintiffs' assertion of a “fundamental right to their own bodily and physical health, which includes what foods they do and do not choose to consume for themselves and their families” is similarly unavailing because plaintiffs do not have a fundamental right to obtain any food they wish. In addition, courts have consistently refused to extrapolate a generalized right to “bodily and physical health” from the Supreme Court's narrow substantive due process precedents regarding abortion, intimate relations, and the refusal of lifesaving medical treatment.

…

I know it sounds surreal, but they are arguing here that you only control your own body with respect to abortion, intimate relations, and euthanasia. Everything else is controlled by the FDA, yea even unto your death from cancer.

Comment at Hacker News

The whole comment should be read, if only to read the extensive footnotes provided in the comment (those numbers in brackets). But it does show just how nanny-esque (and not to mention risk-averse) our government has become.

I'm telling you—Leviathan, peak government.

Monday, January 21, 2013

One peculiar router …

I decided to poke around a bit with the home router, a Cisco WRVS4400N Wireless-N Gigabit Security Router and I must say, it's an odd router from a management point of view.

So I get a list of interfaces (via SNMP) on the device:

Interface Dump
Status Interface Bytes In Bytes Out
uplo207454207454
downipsec100
downipsec200
downipsec300
upbr0304475952467833506
upppp010052555380535424
upsit100
upeth0246917723273606969
upeth111939148788802941
upeth2109882441191904431
downteql000
upsit000
downtunl000
downgre000
upipsec000

Okay, lo is the local loopback device, I have no idea what br0 is, but I see eth0, eth1 and eth2 which are obviously the Ethernet ports in use. So, what do I have plugged in where? Okay, that's easy to determine—unplug a device, see which interface is marked as “down”. This is typical Cicso behavior, right?

Okay, I unplug the Mac from the network and I see:

Interface Dump
Status Interface Bytes In Bytes Out
uplo207454207454
downipsec100
downipsec200
downipsec300
upbr0304488023467841666
upppp010052605980536312
upsit100
upeth0246926120273617295
upeth111939309188804157
upeth2109888649191905881
downteql000
upsit000
downtunl000
downgre000
upipsec000

So. It's not going to mark one of the Ethernet ports as being down. Lovely. Looks like I'm going to have to do this the old fasioned way:

[Physically tracing the wires]

Physically trace each Ethernet cable. Okay, in the above image, the left-most cable goes out to the Intarwebs. The right-most cable (the blue one, in the port labeled “4”) goes to the Mac. The one to the left of the blue cable (in port “3”) goes to my Linux system. The one in port “2” goes to a computer that is currently turned off. And the one in port “1” (the second on the left) wraps around and is hanging in front of the shelves (I use that one for the laptop).

Wait a minute … let me look at the front again …

[Cisco WRVS4400N]

We have the Internet; Mac is at gigabit speed; my Linux system, and … the turned off system? Really?

Okay, it appears that the Ethernet card in the currently off system receives just enough power to maintain a connection status; there's probably a “wake-on-LAN” feature on its Ethernet card.

Okay, now that's that's straightened out … um …

eth0, eth1 and eth2 …

Um …

The external ports don't match up with the internal ports. And given that there's a computer that is turned off, shouldn't one of the Ethernet ports return no data? And where's eth3?

What exactly is going on?

Okay, put that aside for now. What's the routing table look like?

Routing Table
Dest Mask NextHop Proto Metric Age Interface
0.0.0.00.0.0.070.XXXXXXXXXXlocal 00ppp0
127.0.0.0255.255.0.00.0.0.0local 00sit1
192.168.1.0255.255.255.00.0.0.0local 00br0
239.0.0.0255.0.0.00.0.0.0local 00br0
70.XXXXXXXXXX255.255.255.2550.0.0.0local 00ppp0

Um, 70.XXXXXXXXXX isn't my IP address; it's 74.XXXXXXXXX (I pay extra for a static IP address because of work issues). Okay, it appears that 70.XXXXXXXXXX is the remote side of my connection, but that routing (while it works) just looks odd to me. So, what IP addresses are assigned to which interfaces?

Interfaces
Destination Mask Interface
127.0.0.1255.0.0.0lo
192.168.1.1255.255.255.0br0
74.XXXXXXXXX255.255.255.255ipsec0

Okay, there's my IP address, but it's … ipsec0? Weird. And it seems that br0 is a grouping of all the Ethernet ports.

But really, the mislableled Ethernet ports, the turned off computer sending and receiving traffic, it just has me skeeved out a bit. And the whole mess makes it difficult to monitor the network (not that I need to monitor my network, but Cisco is selling this as a “Small Business” device and a “Small Business” might want to monitor its network).

Oh, I just thought of something … the wireless interface—it's missing! I mean, it's missing in the interface list; physically it's there or Bunny wouldn't be able to use her laptop on the Intarwebs.

This is one strange router …

Update on Tuesday, January 22nd, 2013

My friend Mark fills me in on what might be happening …

Tuesday, January 22, 2013

More on that peculiar router of mine …

From
Mark Grosberg <XXXXXXXXXXXXXXXXX>
To
spc@conman.org
Subject
Router
Date
Tue, 22 Jan 2013 11:10:49 -0500 (EST)

To explain your router: You have three actual hardware Ethernet MACs:

The br0 interface is a software bridging between eth1 and eth2. This way your wireless computer is on the same network as your wired computers. But it's not a real physical device but represents packets from either eth1 or eth2 to the IP stack. The multiplexing is done in the driver for that interface.

As for the IP address discrepancy: eth0 does not actually act as your Internet. That's not how DSL from XXXXXXXX works. eth0 uses managed IP's to bring up a PPPoE session. So ppp0 will likely have your public IP address. Those packets get tunnled within the other set of IP's. That's a guess since I don't have XXXXXXXXX DSL though.

The thing with software, even embedded software these days is that there are so many layers of existing crap (that don't need to be there but skittish managers are loathe to remove things they don't grok) it's amazingly difficult to grasp the whole picture since there is often little rationale behind it.

That actually explains the behavior I'm seeing on the router. So I can monitor the Intarweb traffic on eth0, overall LAN traffic on eth1 and the WiFi network via eth2.

Okay then.

Cellphone guts for the curious …

M thought I might find the following email he sent to some of his fellow cow-orkers interesting (and I did). He also gave me permission to post it here.

So, if you were ever interested in the internal processing guts of a cell phone, well, here you go …

From
MXXXXXXXXXXXX <XXXXXXXXXXXXXXXXXXXXX>
To
XXXXXXXX XXXX XXXXXXXXXXXXXXXXXXXXXX
Subject
XXXXXXXXXX firmware guts for the curious …
Date
Sun, 20 Jan 2013 19:07:00 -0500

So I found myself poking around inside the firmware of my personal cell phone, an HTC EVO 3D. This phone is based on the XXXXXXXXXXXXXXXXXXX chipset. In investigating something about how my phone worked I decided to disassemble the cellular firmware on it and see how some of the EVDO stuff was done. I found some interesting things for those of you who are curious as to how some of these things work beyond the Linux side of Android. If you aren't then feel free to skip this e-mail.

A cell phone running something like Android had a complete separation between the application processor (the CPU running Linux, your apps, the GUI, all that stuff) and the cell phone processor (runs the mobile data stack, voice codecs, speakerphone echo cancellation and all other manner of hard real-time stuff). The (somewhat incorrect) terms of “application processor” and “baseband processor” were assigned to these two CPUs, respectively. So why the separation in the first place? Well Linux isn't very good at things like voice or real-time. Running a cellular radio is quite a bit of real time code. And all the speech pathways of the actual “phone” part are also a bit much for Linux to deal with. Linux, as with much “desktop” style software out there gives little thought to things like interrupt latency—making it a terrible OS to do anything hardware. Plus it makes sense that when smartphones first started popping up everywhere that you had a bunch of grizzly EE guys doing the RF stuff who didn't want to mess with Linux and all that “high level crap.” (And yes, I fall squarely in that camp). So the dual-CPU approach keeps the different types of engineers from getting in each others' way.

I used to think that the baseband firmware was pretty traditional, monolithic piece of embedded software. Turns out I was dead wrong. I can only guess at the reasons for their architecture since I have no real contacts at XXXXXXXX. But the firmware is actually quite a hodgepodge of things. Those of you who fancy yourselves software archaeologists may find this informative.

The software behind the baseband CPU is quite interesting and totally undocumented. But I dug in and found a fascinating world. First everything in the XXXXXXXXXX baseband runs under a hypervisor—which they call a “microvisor.” For those of you unfamiliar with what this means it's a bit like running VMware on a bare computer and then running Linux and Windows simultaneously. Only in this case the hypervisor is a commercial port of the L4 microkernel. The hypervisor divides the processor into “cells” that each run their own distinctive world. Most of the cellular modem software runs under an OS called Iguana. Iguana wraps an older RTOS used by XXXXXXXX called REX that now runs inside an L4 task and makes it a message-based server. Those of you who worked with me at XXXXXXX may remember how we ran [one operating system] on top of [another operating system] on our XXXXXXXXXX XXXXX PBX—this is kind of the same thing. Only the ARM architecture is far saner to deal with than x86 when it comes to virtualization.

The cellular software running under these layers is called XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX—and it's a quite typical monolithic embedded blob of software. But it's had a few interfaces shoved in it and now communicates with the rest of the firmware through a message based interface.

So why did they go to all the trouble to run their embedded firmware under L4? Well there is more running on the baseband processor than you may think. It's grown in complexity and now contains several large pieces of disjoint software, so the microkernel approach starts to make sense when you consider how much legacy code they have. Poking around I found out the XXXXXXXXXX firmware also uses pForth. Mostly it looks likes for remote diagnostics the cellular carrier can send blobs of FORTH code right to the radio. The radio firmware also seems to have an IP stack (with TCP) so it can do its own interesting things (both bad and good). This TCP stack is what talks to the cellular network. The rmnet0 interface actually goes over shared memory and talks to this IP stack (which is where PPP is actually running).

The GPS functionality of your phone is also quite a huge pile of code and again the Linux side of things sees a very simple interface where coordinates are fed into the application processor. The huge amount of code necessary to do the actual positioning (and the associated WAAS & aGPS stuff) also runs as a task under L4. There are also a pile of DSP's running their own kernel that communicate with the L4 kernel for management purposes.

I think XXXXXXXX is planning on eventually removing the distinct baseband processor—maybe in their low-cost chipsets. You can run Linux under L4. This means the entire Android application processor could in theory be virtualized and run alongside the baseband software on a single core. I could see some advantages to this and I wonder if XXXXXXXX is working on (or implemented) this. A quick peek at the various device drivers in the open source kernels for the XXXXXXXXXX shows the drivers just fill in buffers and queue messages in a very high level manner compared to a typical device driver. Since the hardware is actually managed by stuff running under Iguana. Since this is just a message interface it wouldn't be terribly difficult to move everything to a single core design.

Wednesday, January 23, 2013

dew umopapisdn

I pop open Google Maps, hit the arrow and I find myself contemplating a new outlook on Lower Sheol:

[An upside down map of South Florida]

At first, I thought Google decided to put an end to the Eurocentralism of world maps and force a new perspective on us, but then I realized that earlier, I had simply rotated the map to face the direction I was travelling.

Ah well …

Thursday, January 24, 2013

But if anyone could have faked it, Stanley Kubrick is a good choice

S. G. Collins' take on the conspriacy that the Apollo moon landing was faked (link via Jason Kottke) is an interesting perspective on it. No, Mr. Collins doesn't think the moon landings were fake, but his argument for it being real is one I've never seen presented.

I think it's worth watching.

Update on Saturday, January 26th, 2013

Sigh. I borked the link to the video.

Friday, January 25, 2013

So charactures are drawn with peripheral vision

You have to try this (link from my friend Hoade). I've done it multiple times and each time, it just amazes me.

Human vision is weird sometimes.

Even more optical illusions

And speaking of optical illusions, Wlofie sent along a link to Impossible World, a collection of optical illusions and impossible images.

Saturday, January 26, 2013

Playing around with QR codes

QR codes have been around for quite some time and I've been meaning to play around with them, but lacked the software to generate them. Just recently though, I came across QArt Coder, a web based application to generate QR codes.

I slapped in http://boston.conman.org/ and the result was amazing:

[QR code of http://boston.conman.org/]

Who'd have thought the QR code for my own site would look that way?

Sunday, January 27, 2013

In the old days, this was probably much harder

From a link on FaceGoogleBookPlus is this video of a mind reader revealing his tricks and let me tell you, it's quite revealing, not only in how it was done, but just how much information was available.

I have to wonder how my college friend Rebel One deals with this crap?

Over the past week, a number of users of the popular photo sharing app Instagram and parent company Facebook have been locked out of their accounts and prompted by both services to upload images of their government issued photo IDs to regain access, as CNET first reported on Tuesday.

Via Hacker News, Instagram Asking For Your Government Issued Photo IDs Now, Too.

I'm not sure how I feel about this. I mean, there was the Google Plus real name controversy a year and a half ago and that didn't go over so well and now we have FaceBook (who owns Instagram) possibly, maybe, pushing rather silently, a “real name” policy for its site.

On the one hand, it's a company, and a company can run its site as it sees fit, and we, as potential users, can use or not use that company as we see fit. But the whole FaceGoogleBookPlus social space does seem to be taking over the Internet these past few years and it's my fear that the only way to use the Intarwebs will be through one of these large social networking sites.

I think overall, I find it spooky.

(Oh, and Rebel One? That's his real name. Just look on his driver's license. Okay, okay, it's not really Rebel One—that's just a nickname for his real name, The Rebellious One.)

Monday, January 28, 2013

Notes from a weekly gathering of food trucks in Hollywood, Florida

Bunny, Gregory and Amy have wandered off to get some more food at the Food Trucks @ Young Circle; I'm standing guard (literally—there are no chairs) at the table (no chairs) we've been using.

[Crab cake sliders! Bacon wrapped hot dogs with crumbled Gorgonzola!]

A gentleman, alone, walks up and stops at the table. In one hand he's holding a small empty cup, in his other hand he sets down a large chocolate shake. He then grabs the straw sticking out of his shake, covers the top with a finger, and proceeds to transfer a sample of his shake to the small cup.

At no point does he acknowledge my existence (and to be fair, I didn't acknowledge back). He just intently transfers some of the chocolate shake from one container to the other via the straw.

After transfering some three or four strawsful, he picks up both containers, and walks off.

Antagonistic phrases

I titled the previous post as “Notes from a bimonthly gathering of food trucks in Hollywood, Florida.” I noted that the Food Trucks @ Young Circle were bimonthly from this sentence:

Starting tonight and every other Monday …

Food Trucks at Young Circle Every Monday, Starting Tonight (and yes, I didn't notice the huge headline at the top of the article)

To me, the phrase “every other Monday” translates to “this Monday, not this Monday, this Monday, not this Monday”—in other words, you skip a Monday.

Bunny informed me that no, it does not mean that; as phrased it meant “this Monday, and all the Mondays after that”—all other Mondays.

Aaaaaaah!

Biweekly—is that “twice a week” or “every two weeks?”

Even the word I used, “bimonthly”, can appear to mean two different things.

Man, I hate antagonistic phrases …

Tuesday, January 29, 2013

Cellphone guts for the curious, Part II

My friend Brian forwarded this comment about cellphone guts to me (the lateness is due to getting permission to post this email from a third party):

From
Brian Yoder <XXXXXXXXXXXXXXXXXXXXXXX>
To
Sean Conner <sean@conman.org>
Subject
Fwd: Cellphone guts for the curious … - The Boston Diaries - Captain Napalm
Date
Wed, 23 Jan 2013 17:49:23 -0500

Sean,

I passed along your blog post to my friend Chip, and here is his response.

Regards,
Brian

Begin forwarded message:

From
XXXXX XXXXX
To
Brian Yoder <XXXXXXXXXXXXXXXXXXXXXXX>
Subject
Re: Cellphone guts for the curious … - The Boston Diaries - Captain Napalm
Date
January 23, 2013, 1:50:52 AM EST

They must have done a very good virtualization work, because FCC regulations specify that the computer that runs the transmit recieve be separate from the rest of the phone and with a ROM.

It drove cell phone manufactures nuts because of cost, but the FCC did want any one to hack in to phone and change the radio software.

Ah … politics trumps technology …

I'm terribly upset that GCC didn't start NetHack

Ah yes, undefined behavior of C. It's easy to see in retrospect, but it's still a bit surprising. The code:

#include <limits.h>

int foo(int a,int b)
{
  return a / b;
}

int main(void)
{
  return foo(INT_MIN,-1);
}

And when you compile with the right options … 

[spc]lucy:/tmp>gcc -std=c99 -O0 crash.c
[spc]lucy:/tmp>./a.out
Floating point exception (core dumped)
[spc]lucy:/tmp>

What's actually going on here? Well, I compiled the code with crashreport(), so I could capture the crash:

CRASH(9372/000): pid=9372 signal='Floating point exception'
CRASH(9372/001): reason='Integer divide-by-zero'
CRASH(9372/002): pc=0x804883d
CRASH(9372/003): CS=0073 DS=007B ES=007B FS=0000 GS=0033
CRASH(9372/004): EIP=0804883D EFL=00010296 ESP=BFFC370C EBP=BFFC3710 ESI=BFFC37C4 EDI=BFFC3750
CRASH(9372/005): EAX=80000000 EBX=00CBBFF4 ECX=BFFC371C EDX=FFFFFFFF
CRASH(9372/006): UESP=BFFC370C TRAPNO=00000000 ERR=00000000
CRASH(9372/007): STACK DUMP
CRASH(9372/008):        BFFC370C:                                     1C 37 FC BF 
CRASH(9372/009):        BFFC3710: 38 37 FC BF 7C 88 04 08 00 00 00 80 FF FF FF FF 
CRASH(9372/010):        BFFC3720: F4 BF CB 00 F4 BF CB 00 F8 A9 04 08 F4 BF CB 00 
CRASH(9372/011):        BFFC3730: 00 00 00 00 A0 CC B8 00 98 37 FC BF 93 4E BA 00 
CRASH(9372/012):        BFFC3740: 01 00 00 00 C4 37 FC BF CC 37 FC BF 26 22 B8 00 
CRASH(9372/013):        BFFC3750: F4 BF CB 00 00 00 00 00 50 37 FC BF 98 37 FC BF 
CRASH(9372/014):        BFFC3760: 40 37 FC BF 55 4E BA 00 00 00 00 00 00 00 00 00 
CRASH(9372/015):        BFFC3770: 00 00 00 00 D4 CF B8 00 01 00 00 00 80 87 04 08 
CRASH(9372/016):        BFFC3780: 00 00 00 00 60 21 B8 00 B0 2C B8 00 D4 CF B8 00 
CRASH(9372/017):        BFFC3790: 01 00 00 00 80 87 04 08 00 00 00 00 A1 87 04 08 
CRASH(9372/018):        BFFC37A0: 47 88 04 08 01 00 00 00 C4 37 FC BF CC 92 04 08 
CRASH(9372/019):        BFFC37B0: 20 93 04 08 B0 2C B8 00 BC 37 FC BF 92 9A B8 00 
CRASH(9372/020):        BFFC37C0: 01 00 00 00 BB A9 FF BF 00 00 00 00 C3 A9 FF BF 
CRASH(9372/021):        BFFC37D0: E4 A9 FF BF F4 A9 FF BF FF A9 FF BF 0D AA FF BF 
CRASH(9372/022):        BFFC37E0: 34 AA FF BF 51 AA FF BF 79 AA FF BF 95 AA FF BF 
CRASH(9372/023):        BFFC37F0: A7 AA FF BF B8 AA FF BF CE AA FF BF EC AA FF BF 
CRASH(9372/024):        BFFC3800: F5 AA FF BF 04 AB FF BF C7 AC FF BF             
CRASH(9372/025): STACK TRACE
CRASH(9372/026):        ./a.out[0x804889c]
CRASH(9372/027):        ./a.out[0x8049078]
CRASH(9372/028):        /lib/tls/libc.so.6[0xbb79b0]
CRASH(9372/029):        ./a.out[0x804887c]
CRASH(9372/030):        /lib/tls/libc.so.6(__libc_start_main+0xd3)[0xba4e93]
CRASH(9372/031):        ./a.out[0x80487a1]
CRASH(9372/032): DONE

And from there, we can load up the program and do some disassembly:

[spc]lucy:/tmp>gdb a.out
GNU gdb Red Hat Linux (6.3.0.0-1.132.EL4rh)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-redhat-linux-gnu"…Using host libthread_db
library "/lib/tls/libthread_db.so.1".

(gdb) disassemble 0x804883d
Dump of assembler code for function foo:
0x08048828 <foo+0>:     push   %ebp
0x08048829 <foo+1>:     mov    %esp,%ebp
0x0804882b <foo+3>:     sub    $0x4,%esp
0x0804882e <foo+6>:     mov    0x8(%ebp),%edx
0x08048831 <foo+9>:     lea    0xc(%ebp),%eax
0x08048834 <foo+12>:    mov    %eax,0xfffffffc(%ebp)
0x08048837 <foo+15>:    mov    %edx,%eax
0x08048839 <foo+17>:    mov    0xfffffffc(%ebp),%ecx
0x0804883c <foo+20>:    cltd   
0x0804883d <foo+21>:    idivl  (%ecx)
0x0804883f <foo+23>:    mov    %eax,0xfffffffc(%ebp)
0x08048842 <foo+26>:    mov    0xfffffffc(%ebp),%eax
0x08048845 <foo+29>:    leave  
0x08048846 <foo+30>:    ret    
End of assembler dump.
(gdb)

It faulted on the IDIV instruction, but it wasn't technically an “integer division-by-zero.” The Intel 80386 (and the Pentium™ in my computer is little more than a glorified Intel 80386) book I have describes IDIV as:

An 80386 interrupt zero (0) [which is reported as an “Integer division-by-zero”] is taken if a zero divisor or a quotient too large for the destination register is generated. [emphasis added]

Now, EAX is -2,147,483,648 (80000000 in hexadecimal notation, which can be represented in 32-bits (we're running 32-bit code here—the issue still happens on 64-bit systems but the value will be vastly larger), but -2,147,483,648 divided by -1 should be 2,147,483,648, but 2,147,483,648 cannot be respresented in 32-bits [Technically, the value can be represented in 32 bits, but the instruction in question, IDIV is a signed instruction, and because of the way Intel does signed integer math, the signed quantity 2,147,483,648 cannot be represented as a 32-bit signed quantity in 32-bits. —Editor] and thus, because the quotient is then considered “too large” we get the fault which ends the program.

This is fine as far as C goes, because C says such behavior is “undefined” and thus, anything goes.

Simple once you know what's going on.

(And for the 99% of my readership who don't get the NetHack reference in the title … )

Wednesday, January 30, 2013

… and a scientist does science!

It is no longer enough to simply ask someone if they are a programmer. Saying a programmer writes programs is like saying a scientist does science. The difference is that botanists don't design nuclear reactors.

Via Reddit, Erik McClure: "Programmer" is an Overgeneralization

It took me some time to convince my dad that, while I am a programmer, I do not use, nor understand, Microsoft Windows; there is more to programming than just The Redmond Monopolist Operating System. The above article goes into just a few different areas that all fall under “programming.”

Even system administration (the job of keeping computers fed and happy) has disciplines—you have those that run Microsoft Windows (and even there, you have several different flavors of Windows to manage) and those that run Unix (and here you can even specialize even further—Solaris, BSD, any number of Linux distributions).

And even if you know how to run Linux, there's a quantitative difference between running a small network (say, less than 100 machines) and something on the scale of Google (100,000 machines? 500,000 machines? A million?).

(If you are curious, I'm clearing out the backlog here … )

Thursday, January 31, 2013

Parsers vs. regular expressions? No contest

I'm finding that where once in Lua to its regular expressions for parsing, I am now turning to LPeg—or rather, the re module, as I find it easier to understand the code once written.

For instance, the regression test program I wrote for work outputs the results of each test:

1.a.8 0-0 16-17 scp:  ASREQ (1) ASRESP (1) LNPHIT (1) SS7COMP (1) SS7XACT (1) tps:  ack-cmpl (1) cache-searches (1) cache-updates (1) termreq (1)

Briefly, the first three fields are the test case ID, and indications if certain data files changed. The scp field indicates which variables of the SCP (you can think of this as a service on a phone switch) were modified (these just happen to be in uppercase) and then the tps field indicates which TPS (our lead developer does have a sense of humor) were modified. But if a variable is added (or removed—it happens), the order can change and it makes checking the results against the expected results a bit of a challenge.

The result is some code to parse the output and check against the expected results. And for that, I find using the re module for parsing:

local re   = require "re"

G = [[
line		<- entry -> {}
entry		<- {:id: id :} 			 %s
		   {:seriala: serial :}	         %s
		   {:serialb: serial :}	         %s
		   'scp:' {:scp: items* -> {} :} %s
		   'tps:' {:tps: items* -> {} :}
id		<- %d+ '.' [a-z] '.' %d+
serial		<- %d+ '-' %d+
items		<- %s* { ([0-9A-Za-z] / '-')+ %s '(' %d+ ')' }

]]

parser = re.compile(G)

to be more understandable than using Lua-based regular expressions:

function parse(line)
  local res = {}
  
  local id,seriala,serialb,tscp,ttps = line:match("^(%S+)%s+(%S+)%s+(%S+)%s+scp%:%s+(.*)tps%:%s+(.*)")
  
  res.id      = id
  res.seriala = seriala
  res.serialb = serialb
  
  res.scp = {}
  res.tps = {}
  
  for item in tscp:gmatch("%s*(%S+%s%(%d+%))%s*") do
    res.scp[#res.scp + 1] = item
  end
  
  for item in ttps:gmatch("%s*(%S+%s%(%d+%))%s*") do
    res.tps[#res.tps + 1] = item
  end
  return res
end

with both returning the same results:

{
  scp =
  {
    [1] = "ASREQ (1)",
    [2] = "ASRESP (1)",
    [3] = "LNPHIT (1)",
    [4] = "SS7COMP (1)",
    [5] = "SS7XACT (1)",
  },
  id = "1.a.8",
  tps =
  {
    [1] = "ack-cmpl (1)",
    [2] = "cache-searches (1)",
    [3] = "cache-updates (1)",
    [4] = "termreq (1)",
  },
  serialb = "16-17",
  seriala = "0-0",
}

Personally, I find regular expressions to be an incomprehensible mess of random punctuation and letters, whereas the re module at least lets me label the parts of the text I'm parsing. I also find it easier to see what is happening six months later if I have to revisit the code.

Even more importantly, this is a real parser. Would you ranther debug a regular expression for just validating an email address or a grammar that validates all defined email headers (email address validation starts at line 464)?

Obligatory Picture

An abstract representation of where you're coming from]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

Obligatory AI Disclaimer

No AI was used in the making of this site, unless otherwise noted.

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: https://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

https://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.