The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Friday, July 04, 2003

The Price They Paid

Spring, the Kidlets and I went to a friend's house for the 4^th. Throughout the house my friend had the following taped up on the walls:

THE PRICE THEY PAID

Have you ever wondered what happened to the 56 men who signed the Declaration of Independence?

Five signers were captured by the British as traitors and tortured before they died.

Twelve had their homes ransacked and burned.

Two lost their sons serving in the Revolutionary Army, another had two sons captured.

Nine of the 56 fought and died from wounds or hardships of the Revolutionary War.

They signed and they pledged their lives, their fortunes, and their sacred honor.

What kind of men were they?

Twenty-four were lawyers and jurists. Eleven were merchants, nine were farmers and large plantation owners; men of means, well educated. But they signed the Declaration of Independence knowing full well that the penalty would be death if they were captured.

Carter Braxton of Virginia, a wealthy planter and trader, saw his ships swept from the seas by the British Navy. He sold his home and properties to pay his debts, and died in rags.

Thomas McKeam was so hounded by the British that he was forced to move his family almost constantly. He served in the Congress without pay, and his family was kept in hiding. His possessions were taken from him, and poverty was his reward.

Vandals or soldiers looted the properties of Dillery, Hall, Clymer, Walton, Gwinnett, Heyward, Ruttledge, and Middleton.

At the battle of Yorktown, Thomas Nelson, Jr., noted that the British General Cornwallis had taken over the Nelson home for his headquarters. He quietly urged General George Washington to open fire. The home was destroyed, and Nelson died bankrupt.

Francis Lewis had his home and properties destroyed. The enemy jailed his wife, and she died within a few months.

John Hart was driven from his wife's bedside as she was dying. Their 13 children fled for their lives. His fields and his gristmill were laid to waste. For more than a year, he lived in forests and caves, returning home to find his wife dead and his children vanished. A few weeks later, he died from exhaustion and a broken heart.

Norris and Livingston suffered similar fates.

Such were the stories and sacrifices of the American Revolution. These were not wild-eyed, rabble-rousing ruffians. They were soft-spoken men of means and education. They had security, but they valued liberty more.

Standing talk straight, and unwavering, they pledged: “For the support of this declaration, with firm reliance on the protection of the divine providence, we mutually pledge to each other, our lives, our fortunes, and our sacred honor.”

They gave you and me a free and independent America. The history books never told you a lot about what happened in the Revolutionary War. We didn't fight just the British. We were British subjects at that time and we fought our own government!

Some of us take these liberties so much for granted, but we shouldn't.

So, take a few minutes while enjoying your 4th of July Holiday and silently thank these patriots. It's not much to ask for the price they paid. Remember: Freedom is never free!

I hope you will show your support by please sending this to as many people as you can. It's time we get the word out that patriotism is NOT a sin, and the Fourth of July has more to it than beer, picnics, and baseball games.

Quite a stirring tale there. But like all things found on the Internet (where my friend found this) I wondered if there wasn't more to this. And the one place I know that regularly discredits discusses such Internet tales is Snopes. Poked around some and:

The main point of this glurge is to impress upon us that the men who signed the Declaration of Independence were relatively well-educated and wealthy men who were also well aware they had much to lose by putting their names to that document, yet after much careful consideration and thought they signed it anyway, “knowing full well that the penalty would be death if they were captured” (although the article omits mentioning that support for independence was far from unanimous, that some of the colonies voted against adopting the Declaration of Independence, and some of the delegates didn't affix their signatures to the document until several years later). The signers were courageous men who risked everything in the service of what they perceived to be a common good, and for that they are genuinely worthy of honor, respect, and admiration. Unfortunately, this article attempts to commemorate them with a train of glurge that jumps the track of truth at the very beginning and finally pulls into station bearing a simplified version of history in which all the incongruities that get in the way of a good story are glossed over. (We're still puzzling over exactly which history books “never told us a lot about what happened in the Revolutionary War,” and if any history books failed to stress the obvious point that “we were British subjects at that time and we fought our own government,” it was probably because they reasonably assumed their readers could infer as much from the constant repetition of words such as “revolution” and “independence.”)

Glurge Gallery—Would July to Me?

---

Blow Stuff Up

Spring, the Kidlets and I were invited to my friend C's house for Fourth of July celebrations. The Kidlets spent the time swimming, Spring spent the time attempting to catch a nap, and I spent the time hanging with friends, some of whom I haven't seen in several years.

The highlight of the day's festivities were the fireworks! C had obtained several boxes of the “slightly questionable” mortar-type fireworks—the type that go up several hundred feet and explode in a shower of colors. Once it got dark, it was time to set up. Mark (also a friend of C's and who caught a ride there with us) helped set up the lanching platform for the mortars.

Things started out calmly enough—one mortar at a time. Pretty soon confidence was gained to attempt the simultaneous launch of two, three, four mortars at a time, showering the area with fireworks. But as Spring said, “I've never done fireworks that there hasn't been at least one dud.”

And boy, was the dud spectacular.

No one is quite sure what exactly happened, but at attempt was made to light six mortars at the same time (ah, gotta love blowing stuff up!). All six fuses were lighted. A couple of seconds of anticipation go by. Hilarity ensues as one of the mortars is either stuck in the tube, was put in upsidedown by mistake, or didn't have enough ooomph for launch. Once it became apparent that it wasn't going to explode a few hundred feet above our heads but right in the middle of the street, people start scrambling for cover as we are caught within the blast radius of a live, “slightly questionable” firework.

It wasn't quite as bad as the picture makes it out to be (digression: I had set the digital camera for long exposures, on the order of a second or more, so having a live firework go off aproximately 50′ away basically overexposed the image—still, it's darned impressive) and luckily, given the circumstances, no one was hurt. And since the Kidlets where there, this is something we can bring up the next time they get “fireworks fevor” (the Younger is especially vulnerable to this infliction).

The show continued on, and afterwards, we cleaned up the spent fireworks, tore down the launch pad, said our goodbyes and went home happy that we were able to celebrate our freedom by blowing stuff up!

Sunday, July 04, 2004

Blow Stuff Up Day

Have a safe and wonderful Fourth (and hopefully, this won't happen to you).

Monday, July 04, 2005

Light fuse, go boom

Just like two years ago, we were again invited to Smirk's house for Forth of July fireworks. And just like two years ago, Smirk had again obtained a few boxes of the “slightly questionable” mortar-type fireworks.

Unlike two years ago though, there were a bit more pre-show safety lectures about the correct method of loading and lighting the fireworks—as exciting as it was last time, no one really wanted to be within 50′ of a live firework mortar as it explodes.

And this year, there were no ground-based explosions.

Granted, there were two that only made it (at max) 100′ into the air before exploding—-spectacular yes! Picture, no (out of frame). But no ground-based explosions.

And we were all happy that we were able to yet again celebrate our freedom by blowing stuff up!

Wednesday, July 04, 2007

Blowing things up with alien technology

Ah yes, the Fourth of July. The time of year when all Americans go outside and blow stuff up. It looked unlikely that anything would be blown up this year, as it was storming all day, but apparently, it cleared up enough for a few fireworks shows I can now hear going off in the distance.

Besides, there are more things one can do to celebrate the Fourth of July than simply to blow things up, such as indulging in a perennial favorite topic of mine—government conspiracies about aliens and their technology!

PACL was located in Palo Alto, but unlike XPARC, it wasn't at the end of a long road in the middle of a big complex surrounded by rolling hills and trees. PACL was hidden in an office complex owned entirely by the military but made to look like an unassuming tech company. From the street, all you could see was what appeared to be a normal parking lot with a gate and a guard booth, and a 1-story building inside with a fictitious name and logo. What wasn't visible from the street was that behind the very first set of doors was enough armed guards to invade Poland, and 5 additional underground stories. They wanted to be as close as possible to the kinds of people they were looking to hire and be able to bring them in with a minimum of fuss.

One downside to CARET was that it wasn't as well-connected as other operations undoubtedly were. I never got to see any actual extra- terrestrials (not even photos), and in fact never even saw one of their compete vehicles. 99% of what I saw was related to the work at hand, all of which was conducted within a very narrow context on individual artifacts only. The remaining 1% came from people I met through the program, many of which working more closely with “the good stuff” or had in the past.

My Experience with the CARET Program and Extra-terrestrial Technology

Of course, it's well known that AT&T developed the transistor based upon alien technology and that Intel has managed to reverse engineer alien technology to build their latest chips.

But like all great conspiracy theories (and I love these because they're so entertaining) they leave a few things unanswered. In the case of AT&T (or more specifically, Bell Labs), did they even have the technology to reverse something as complex as a microchip? Send back even an old Pentium to Bell Labs in even 1940 (a full seven years before they invented the transistor)—could they even figure out what it was? (I'm assuming they could get the actual chip out of the ceramic casing) Could they even detect a single transistor?

Besides, a transistor is nothing more than two back-to-back diodes in a single package, and diodes have been around since the 1880s.

I received a link to the CARET page from a mailing list I'm on, and as one member of the list commented:

How do you build a “secret” 5 story underground facility in Palo Alto with and

1. have the right number of cars in the parking lot and
2. not hit the water table. The industrial parts of PA are either in the flats next to 101 and the SF Bay or are near Foothill Expressway where almost all of the industrial parks are owned by Stanford.

Besides, how do you advertise jobs for such a place?

Oh, and the Intel Pentium being based upon alien technology? That, I can actually buy. I mean, have you ever tried programming that thing? Something that convoluted can only be the result of humans partially reverse engineering alien technology …

---

“Houston, we have a problem … ”

London, 4th July 2007. Steorn, an Irish technology development company, will publicly demonstrate a real-life application of its Orbo free energy technology for the first time. The demonstration will take place in the Kinetica Museum gallery, London UK on Wednesday 4th July. People around the world will be able to watch the exhibit via a live web stream.

Update 4/7/07 23:30

Due to slight technical difficulties we will now be publishing the live stream as of Thursday 5th July.

Via flutterby, Steorn demonstrates free energy technology in public forum

In keeping with the woo-woo technology, I present you with Steorn, a company premoting free energy, only they seem to be having difficulties …

Hmmmmm ….

---

“I'm proud to be an American, where at least I know I'm free, living under a system of Law, without systemic corruption and I can blow stuff up for fun, not for survival.”

Such a uniquely backfiring setup was responsible for the world's worst library. A few days after I arrived in Cameroon, I visited one of the country's most prestigious private schools—Cameroon's equivalent of Eton. The school boasted two separate library buildings, but the librarian was very unhappy. I soon understood why.

At first glance the new library was impressive. With the exception of the principal's palatial house, it was the only two-story structure on campus. Its design was adventurous: a poor man's Sydney Opera House. The sloped roof, rather than running down from a ridge, soared up in a V from a central valley like the pages of an open book on a stand.

When you're standing in the blazing sunlight of the Cameroonian dry season, it's hard to see at first what the problem is with a roof that looks like a giant open book. But that's only if you forget, as the architect apparently did, that Cameroon also has a rainy season. When it rains in Cameroon, it rains for five solid months. It rains so hard that even the most massive storm ditches quickly overflow. When that kind of rain meets a roof that is, essentially, a gutter that drains onto a flat- roofed entrance hall, you know it's time to laminate the books. The only reason the school's books still existed was that they'd never been near the new building; the librarian had refused repeated requests from the principal to transfer them from the old library.

Via Flares Into Darkness, Why Poor Countries Are Poor

This is a long article, going into detail about the systemic corruption in your typical African nation, and how difficult it will be for Cameroon to pull itself out of its death spiral. It's well worth reading, and a reminder of just how good we have it here in the west, where we can blow stuff up without worry …

Friday, July 04, 2008

The Fourth Of July

Ah, National Blow Stuff Up Day today, although given both the weather (the Miami fireworks show was cancelled) and how under the weather I felt, I didn't do much else but sleep.

I think it was something I ate.

Saturday, July 04, 2009

Dandelions and Palm trees

Bunny and I headed up to Indiantown (and while a small town, it was large enough to attract about a dozen Tea Party protestors) to visit her mom, attend a local barbeque and to catch the Indiantown fireworks show.

And for such a small town, it was quite the fireworks show.

I also heard that the local group of citizens of Indiantown actually collect money and pay for the show, not the general tax payers. How about that?

Sunday, July 04, 2010

Just an FYI for those of you who might think of doing something silly like setting off illegal fireworks you got by driving to the border of South Carolina and totally said you would use them legally, wink wink nudge nudge say no more say no more …

---

Scaring 'da birds

It's been seven years since last I attended a Fourth of July party with my friend C, and once again, I'm at his house to help scare the local bird population. Fortunately this time, there was no big boom, but as a precaution, C moved the festivities to the back yard, next to the pool.

Monday, July 04, 2011

The Inadvertent Michael Jackson Impersonation

Our neighbor tends to go all out on the Fourth of July, so there's no real need to go anywhere for a fireworks show. So when he and his friends started firing off their works, Bunny and I went outside to enjoy the show.

Then this one is lit off:

My reaction was pretty much: Hmm … wow that's loud, and large, and low, and oh XXXX XXXX that's going to XXXXXXX land on me get it off get it off get it off. “Aaaaaaaaaaaaaaaaaaaaaaaaaaah!”

Wow.

I've been in an explosion.

Now my hair caught on fire.

Okay, okay, technically it wasn't on fire, but it was singed, and man, singed hair smells horrible.

Blech.

Fortunately, I suffered no burns, but I was lucky. It could have much much worse, which is a nice time for this little PSA:

I think next year, I'll watch the fireworks on TV from the comfort of an underground bunker.

Thursday, July 04, 2013

Iron Sky

Sure, you can barbeque various critters and watch the fireworks to celebrate the Fourth of July, and yes, Bunny and I did all that. But we also watched the film “Iron Sky.”

What's the film about? Two words: Space Nazis!

Yup, that's right. Nazis from space!

The backstory is that the Nazis escaped from Earth in 1945 and are hiding out on the dark side of the moon. And due to a chance encounter with an American astronaut, they decide it's time to head on back to conquer the Earth.

It's one of those films where you just have to turn off your brain and roll with it. Sure, there are plot holes, but frankly, the entire premise is a plot hole (“How did the Nazis get into space? How do they generate their atmosphere? Where's their industrial base to build all the flying saucers? How can they get to the Earth in less than a day?”) but that's okay, because this is a fun film (Space Nazis! Space dirigibles! Albino African-Americans! Mad scientists! Beautiful daughters of said mad scientists! Flying saucers vs. fighter jets! Fighting the in war room! What's not to love?)

Friday, July 04, 2014

Notes on a conversation held during a fireworks show

FWOOM!

POW!

“Oooooooooh!”

FWOOM!

BOOM POP POP POP POP POP POP!

”Aaaaaaaahhhh!”

THWOM!

BABOOM!

“Oooooooooooooooooooooooh!”

“Hey, watch this!”

Fizzle. Fizzle. FWOOOSH THWACK-OOOM!

“Ow ow ow ow ow ow ow ow ow ow ow!”

“Eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeek!”

“Stop! Drop! Roll!”

“Extinguisher!”

“Medic!”

Fwooooooooooooooooooooossssssssssshhhhhhhhhhhhhhhhhhhhhh.

“Are you okay?”

“Yeah … tis but a minor burn. Man, that was fun! Let's do it again!”

Saturday, July 04, 2015

Moving from reservation to reservation

With our reservation up, Bunny and I bid our adieu to The Sunset Motel.

The only real complaint we had about the place was the plumbing—straight out of the 50s. The inflow of water during our showers did not match the outflow of water down the drain, so we were continuously up to our ankles in water. Also, there was a fine line (about 2mm) between “boiling alive” and “flash frozen.” And the less said about the toilet, the better.

The other complaint we had (and I'll admit this is totally a First-World 21^st Century problem) was the rather lackluster wi-fi access point (more like a yo-yo wi-fi access point). Most of the time I used the “Personal Hotspot” function of my iPhone due to the poor performance of the wi-fi (and managed to burn through half our “Trust Us! One gigabyte is more than you need” data plan in a few days—damn you overly large webpages!—but I admit, I'm a special case here).

But with our reservation sunsetting on The Sunset Motel, there was a reservation rising at The Red House Inn.

Again, due to recurrent procrastination, I couldn't reserve the room we wanted, but that's not to say the room we got was bad. It's not.

And like the previous room, the bathroom has a shower with dual-shower heads.

Now I'm wondering if all the rooms here at The Red House Inn have dual-shower heads.

Also in the room is a beautiful writing desk that I've taken over.

And so far, the wi-fi access here has been fantastic (no First-World 21^st Century problems here)!

---

Bada-BOOM!

To celebrate the Fourth of July, Brevard closes Main Street (Main Street! And a block north and south on Broad Street, the main road through town) to vehicular traffic for an all-day festival.

There were plenty of artists, food stalls, games and rides to keep the thousands that showed up busy. Even the rain didn't discourage anyone from slowly meandering about (the rain just made the meandering more frantic at first, then slowed it down a bit).

By the time of the fireworks show, 9:30 PM at Brevard College, Main Street had been opened, but a quarter mile segment of Broad Street, from the Transylvania Courthouse north towards the campus (the Courthouse is at the top of a hill) had been closed off to allow people a good view of the fireworks show.

After the show, it was quieter in Brevard than our own neighborhood. I don't know if that's because of stricter enforcement of fireworks restrictions, or those setting off The Good Stuff™ just live further away from town.

I'm guessing the latter.

---

Extreme revolutionary acts, Brevard edition

At 3:00 PM, right in front of the Transylvania County Courthouse, a bunch of rabble rousers stood in the middle of the street and right then and there, started ranting about the government.

Stuff about intolerable taxes, a government that doesn't listen to its citizens, usurping the will of the Governors on a whim, giving foreign governments power over us, for waging unjust wars and other things that are outright trea—oh wait a second! Bunny just corrected me—they were reading The Declaration of Independence.

Oh.

How revolutionary!

Tuesday, July 04, 2017

Be careful, for tomorrow could be Sorrowing Day—the 5th of July

My friend Hoade sent me several stereoscopic pictures (and a viewer) for Christmas. One struck a chord with me:

(and yes, I'm trying to give you, the reader, an impression of viewing a stereoscopic picture on the Intarwebs)

On the back is written:

No 1128 (b). SORROWING—JULY 5.

This is the same little boy who started out to make so much noise and smoke yesterday celebrating the Fourth. In the United States in one year, according to the figures gathered by experts, there were 168 boys and men and a few girls, too, who, like this boy, started out celebrate [sic], and wound up by leaving sorrowing friends, for that many were killed by explosions or died from lockjaw as the result of celebrating just one July 4th; and besides that, there were hundreds upon hundreds—something like 7,000 boys and men and girls maimed and crippled in that same celebration. Most of those who died were killed by lockjaw, one of the most terrible of diseases.

Perhaps five whole regiments of young men and boys are killed or injured every Fourth of July by fireworks and revolvers and mostly by toy pistols that shoot caps and torpedoes—for these cause most of the lockjaw.

Is it worth while to celebrate that way, and kill and maim in a celebration so many whose lives are useful to their country? Would it not be better to leave the fireworks to those skilled in handling them, and instead of devoting July 4th to fun, to make it a day of patriotism, to teach to all Americans what the Declaration of Independence means?

No year is given on that particular picture, but some of the others have “1925” so I guess they are all from about the same time period. And given the number of times “lockjaw was mentioned, it appears it was a grave concern at the time.

Nowadays, you have to concern yourself with burnt hair from overzealous neighbors.

Be safe, and remember, there's a reason professionals exist.

---

Oh make a little smoke, make a little fire, get BBQ tonight! Get BBQ tonight!

Last week I decided I wanted to do barbecue ribs for dinner tonight. And by “barbecue” I mean “barbecue,” with real smoke and everthing. Bunny got two racks of ribs, and yesterday, I spent some time doing some prep work.

First, we need some chunks of hardwood to produce the smoke. We have a bag firewood Bunny brought some time ago, so I grabbed half a log, and on the table saw, split it in half, and then cut it into four chunks.

I wanted to use the bandsaw, but unfortunately, the tire on the lower wheel disintegrated, rendering the bandsaw temporarily out of commission.

I can't say I enjoyed using the table saw for this, but using the The Gripper for splitting the log, and the cross-cut sled for the chunks certainly helped to keep my fingers safe. But for wood sold to be used in a fire, it was very nice wood.

Then, just before bed, I spread some yellow mustard over the ribs—this will help the rub to stick. Then, the rub (brown sugar, salt, many, many spices—there are plenty of recipies out on the Intarwebs), then wrapped the ribs in foil and placed into the refrigerator overnight.

Around noon today, I started the barbecue. I don't have a smoker, but I do have a classic Weber Kettle Grill, which can do the job in a pinch. The method I'm using is the “snake method.” You arrange charcoal around the edge of the grill:

I placed one layer two bricks deep, then another layer on top that was two bricks deep. On top of that, the wood chunks are placed to provide the smoke. And yes, three chunks was more than enough to impart a smokey flavor to the ribs. You only need a few hot coals placed at one end to start things off. Over time, the coals will slowly catch fire down the snake and if you get it just right, you won't have to add any coals at all (the above lasted 4½ hours, which was enough to cook the ribs—more on that below).

I do recommend using a starter chimney:

All you do is spread some vegetable oil on some newspaper (makes it burn longer), wad it up, place it in the bottom of the chimney and light it on fire. Then walk away. Just walk away. It'll light the coals and when they're starting to ash over, pour them out at one end of the snake.

Once that's done, add a foil pan with hot water (I placed the pan on the level with the coals, then used an electric kettle to boil the water and pour it into the pan). This not only helps with the smoking, but it also helps to regulate the heat during cooking. Place the grill grate on the grill, and slap those ribs on it, as far away from the heat as possible.

Place the lid on, with the hood vents placed as far away from the heat as possible. This forces the smoke across the ribs. Also, you'll want to adjust the vents on the top and bottom to keep the tempurature as close to 250°F as possible (220° would be ideal). This may require some fiddling until you get it right. I checked on the them about every hour for the first three hours, then every 30 minutes thereafter. You can tell the ribs are done by sliding some tongs along the length of the ribs about half-way, and lifing them up. If the meat “cracks” then they're done. Slather any barbecue sauce you want on them, place the lid back on for another 5 to 10 minutes, then pull them off.

All that's left was to carve them up and eat.

As a first-time experiment, they were good, but there are a few things I will change for next time. First, the ribs were tender, but not “fall-off-the-ribs” tender, and I think that's because I left the rub on just too long. I think I would go no shorter than an hour, no longer than two, before cooking. Second, our grill was barely large enough to handle two racks of ribs (and I suspect these were larger racks—between the two of us, we only ate about 2/3 of a single rack), so next time, one rack.

Other than that, I think this was a success.

Wednesday, July 04, 2018

All's quiet on the holiday front

Right now it's all quiet outside, but I suspect that won't last the day.

So today, after you have read The Constitution, but before you hand over your beer and say, “Watch this!” please remember to be careful. I would hate for this to happen to you:

---

Notes on an overheard conversation

“So the weather report states thunderstorms for the Fourth.”

“Oh! At night?”

“Mostly the afternoon and ending around 8:00 pm.”

“Darn!”

“Oh, expecting to get out of walking to the Boca Raton fireworks show?”

“No.”

“No?”

“No. I was hoping our neighbor wouldn't blow us all up!”

---

Neither heat nor rain nor threat of thunderstorms stays the neighbor from the swift lighting of firework rounds

It sounds like Desert Storm outside.

I guess the threat of thunderstorms wasn't enough to deter the neighbor.

Sigh.

Thursday, July 04, 2019

“T'was the night after fireworks, and all through the land, I can only hope, that no one lost a hand.”

It's that time of the year again when people spend vast amounts of time and money shooting off fireworks. As of now, it no longer sounds like a war zone and the smell of black powder has drifted onward. So I hope everyone had a safe Fourth of July and that this:

never happened to you.

Saturday, July 04, 2020

Adventures in Formatting

If you are reading this via Gopher and it looks a bit different, that's because I spent the past few hours (months?) working on a new method to render HTML into plain text. When I first set this up I used Lynx because it was easy and I didn't feel like writing the code to do so at the time. But I've never been fully satisfied at the results [Yeah, I was never a fan of that either –Editor]. So I finally took the time to tackle the issue (and is one of the reasons I was timing LPEG expressions the other day [Nope. –Editor] … um … the other week [Still nope. –Editor] … um … a few years ago? [Last month. –Editor] [Last month? –Sean] [Last month. –Editor] [XXXX this timeless time of COVID-19 –Sean] last month).

The first attempt sank in the swamp. I wrote some code to parse the next bit of HTML (it would return either a string, or a Lua table containing the tag information). And that was fine for recent posts where I bother to close all the tags (taking into account only the tags that can appear in the body of the document, <P>, <DT>, <DD>, <LI>, <THEAD>, <TFOOT>, <TBODY>, <TR>. <TH>, and <TD> do not require a closing tag), but in earlier posts, say, 1999 through 2002, don't follow that convention. So I was faced with two choices—fix the code to recognize when an optional closing tag was missing, or fixing over a thousand posts.

It says something about the code that I started fixing the posts first …

I then decided to change my approach and try rewriting the HTML parser over. Starting from the DTD for HTML 4.01 strict I used the re module to write the parser, but I hit some form of internal limit I'm guessing, because that one burned down, fell over, and then sank into the swamp.

I decided to go back to straight LPEG, again following the DTD to write the parser, and this time, it stayed up.

It ended up being a bit under 500 lines of LPEG code, but it does a wonderful job of being correct (for the most part—there are three posts I've made that aren't HTML 4.01 strict, so I made some allowances for those). It not only handles optional ending tags, but the one optional opening tag I have to deal with—<TBODY> (yup—both the opening and closing tag are optional). And <PRE> tags cannot contain <IMG> tags while preserving whitespace (it's not in other tags). And check for the proper attributes for each tag.

Great! I can now parse something like this:

<p>This is my <a href="http://boston.conman.org/">blog</a>.
Is this not <em>nifty?</em>

<p>Yeah, I thought so.

into this:

tag =
{
  [1] =
  {
    tag = "p",
    attributes =
    {
    },
    block = true,
    [1] = "This is my ",
    [2] =
    {
      tag = "a",
      attributes =
      {
        href = "http://boston.conman.org/",
      },
      inline = true,
      [1] = "blog",
    },
    [3] = ". Is it not ",
    [4] =
    {
      tag = "em",
      attributes =
      {
      },
      inline = true,
      [1] = "nifty?",
    },
  },

  [2] =
  {
    tag = "p",
    attributes =
    {
    },
    block = true,
    [1] = "Yeah, I thought so.",
  },
}

I then began the process of writing the code to render the resulting data into plain text. I took the classifications that the HTML 4.01 strict DTD uses for each tag (you can see the <P> tag above is of type block and the <EM> and <A> tags are type inline) and used those to write functions to handle the approriate type of content—<P> can only have inline tags, <BLOCKQUOTE> only allows block type tags, and <LI> can have both; the rendering for inline and block types are a bit different, and handling both types is a bit more complex yet.

The hard part here is ensuring that the leading characters of <BLOCKQUOTE> (wherein the rendered text each line starts with a “| ”) and of the various types of lists (dictionary, unordered and ordered lists) are handled correctly—I think there are still a few spots where it isn't quite correct.

But overall, I'm happy with the text rendering I did, but I was left with one big surprise …

---

Spending cache like its going out of style

I wrote an HTML parser. It works (for me—I tested it on all 5,082 posts I've made so far). But it came with one large surprise—it bloated up my gopher server something fierce—something like eight times larger than it used to be.

Yikes!

At first I thought it might be due the huge list of HTML entities (required to convert them to UTF-8).

A quick test revealed that not to be the case.

The rest of the code didn't seem all that outrageous, so fearing the worst, I commented out the HTML parser.

It was the HTML parser that bloated the code.

Sigh.

Now, there is a lot of code there to do case-insensitive matching of tags and attributes, so thinking that was the culpret, I converted the code to not do that (instead of looking for <P> and <p>, just check for <p>). And while that did show a measurable decrease, it wasn't enough to lose the case-insentive nature of the parser.

I didn't check to see if doing a generic parse of the attributes (accept anything) would help, because again, it did find some typos in some older posts (mostly TILTE instead of TITLE).

I did try loading the parsing module only when required, instead of upfront, but:

1. it caused a massive spike in memory utilization when a post was requested;
2. it also caused a noticible delay in generating the output as the HTML parser had to be compiled per request.

So, the question came down to—increase latency to lower overall memory usage, or consume memory to decrease a noticible latency?

Wait? I could just pre-render all entries as text and skip the rendering phase entirely, at the cost of yet more disk space …

So, increase time, increase memory usage, or increase disk usage?

As much as it pains me, I think I'll take the memory hit. I'm not even close to hitting swap on the server and what I have now works. If I fix the rendering (and there are still some corner cases I want to look into) I would have to remember to re-render all the entries if I do the pre-render strategy.

---

A relatively quiet Fourth of July

This has been one of the quietest Fourth of July I've experienced. It's been completely overcast with the roll of thunder off in the distance, the city of Boca Raton cancelled their fireworks show, and our neighbor across the street decided to celebrate with his backyard neighbor on the next street over.

Yes, there's the occasional burst of fireworks here and there, but it sounds nowhere near the levels of a war zone as it has in the past.

Happy Fourth of July! And keep safe out there!

• Current
• First
• Last
• Top
• Home
• About
• Archive
• Search
• Glossary
• Copyright
• Help
• Accessibility

Obligatory Picture

Obligatory Contact Info

• Comments? sean@conman.org

Obligatory Feeds

• RSS Feed
• Atom Feed
• JSON Feed

Obligatory Links

• Flutterby!
• KIRK.is

Obligatory Miscellaneous

• About the Source Code
• Source Code

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.