The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Monday, November 09, 2009

“But it's about the dancing!”

For months now Bunny and I have been going back and forth on things like Microsoft songsmith, Auto-Tune which has been, basically, an argument about tools vs. crutches. [1] That topic surfaced once more today when Bunny mentioned that some Aussies are rather upset at Britney for lip syncing (and here I thought that was common knowledge—guess not down under).

Here is one Australian's take on it:

That being so, it makes sense that when it comes to touring she should mime. After all, it allows the projection of voice perfection, or at least as perfect as it sounds on CD. What sort of a show would it be if she were heard to miss a note or sing offkey. If you had spent a couple of hundred dollars or more, you'd probably walk out and/or demand your money back.

Still, if this is a comeback tour (with the somewhat rapidly turning ironic title of Circus) then surely wouldn't a singer want to reacquaint her voice to her fans? Then again maybe her singing voice ain't what it used to be.

The truth, for the punters, is you'd never know anyway unless you heard her singing in the shower. This is the age of musical perfection where every singer can have perfect pitch, where any imperfection can be erased, where the human can be deleted from the process of recording. It's in the technology. In the digital age, sound can be manipulated. Voices can be modulated, on stage and in the studio. You would never know unless you had something with which to compare it.

Miming, however, is not new. Bands and solo singers were doing it on television pop shows decades ago. Indeed, wasn't it strange that those guitars sounded just like they did on the record, yet they weren't plugged in, and wait a minute where are the amps?

Concerts, however, are another thing. There's a transaction involved between fan and artist. Which leads to one of the most notorious cases in popular music of things not being as they seem. Milli Vanilli won a Grammy Award. Yet the duo of Fabrice Morvan and Rob Pilatus, whom everyone thought were Milli Vanilli were, in fact, just the pretty faces, the image music entrepreneur Frank Farian wanted to project to the public. The voices were actually those of studio singers. The whole thing came crashing down when Morvan and Pilatus were found to be lipsynching at one of their concerts.

Britney Spears | lip synching | Warwick McFadyen

Hmm … tool … crutch … this is appearing to be the Month of the Tool …

  1. I view such things as tools, Bunny views them as crutches, but she's had musical training and I haven't. I view such things as IDEs and PHP as crutches, whereas I think Bunny would view them as tools, but I have a background in programming. I'll have to ponder on this some more. [Back]

Perhaps an 80M script is a bit excessive …

Every so often I'll do a bit of work on an unimportant project, just to keep myself sane from working in PHP and Drupal.

About a month ago I decided to save the data from email indexer program as a Lua program, something like:

emails = 
  filelist = 
      file = "/home/spc/Mail/sent" ,
      size = 902273,
      time = "Tue, 10 Nov 2009 09:35:10 GMT",
      file = "/home/spc/LINUS/Archive.mail/20060607/cctalk",
      size = 230140,
      time = "Tue, 02 May 2006 18:22:28 GMT",
    -- and so on ... 

  mbox = 
      info = { mboxfile = 1, oh={45, 322}, ob={368, 15}},
      ['Message-ID'] = "<20081021051331.GA30804@lucy.localdomain>",
      ['From'] = { "Sean Conner <>",},
      ['To'] = { "",},
      ['Subject'] = "This is a test",
      ['Date'] = "Tue, 21 Oct 2008 01:13:31 -0400",
      ['MIME-Version'] = "1.0",
      ['Content-Type'] = 
      mimeheaders = 
        ['Content-Disposition'] = "inline",
      ['Lines'] =  1,
      extraheaders =  
        ['User-Agent'] = "Mutt/1.4.1i",
        ['Status'] = "RO",
    -- and so on ... 

That way, I could load it into the Lua interpreter and work with the data in Lua, instead of writing a bunch of C code. I debugged the output to make sure it was valid Lua and everything was fine.

Until I threw 80,919 messages from 2,360 email files I had lying around (going back to 1991). Then all I got from Lua was:

lua: constant table overflow

Hmmm … okay, maybe throwing a 80MB into the Lua interpreter wasn't such a good idea.

But then tonight I decided to give it one more try. The source code to Lua didn't reveal any immediate settings to tweak, so I did a bit of searching. And yes, I'm not the only one with that problem. Reading further, I learned that while there isn't a limit to the size a Lua table can get, there is a limit to the number of constants in a single Lua function.

But the code isn't in a Lua function.

Or is it?

It is. When you load Lua code from an external source, it gets compiled into an anonymous function that needs to be run. So, the solution is to break the initialization into several functions, and from some experimenting, I found that things would work (with this particular data set) if I only initialized 16,384 items per function.

But there's a difference between “it worked” and “this is a usable solution.”

Generating the Lua code? 30 seconds.

Loading the Lua code into the interpreter? Six minutes and an overheated CPU

Interesting …

Update Tuesday, November 10th, 2009

I managed to hit the worst case run-time with the code. Change the order of things, and it runs in about 15 seconds. Go figure …

Update Wednesday, February 3rd, 2010

It was a bug in Lua that has since been fixed.

Obligatory Picture

[The future's so bright, I gotta wear shades]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site:, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.