The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Saturday, May 11, 2024

How to measure ⅚ cup of oil, part III

I just received a nice email from Muffintree14 thanking me for helping them make a recipe where they needed to meaure out ⅚ of a cup! They were trying to measure out 200ml of something (they didn't specify what) and it turns out that 200ml is about ⅚ of a cup. I suspect they could have just used a regular cup, as that's 237ml. As long as you aren't baking bread (or other pastry-like food item) then it probably doesn't matter that much. Roughly speaking, 200ml is close enough to 1 cup that you might as well use 1 cup.

But then I found an image (via Bob Anstett on Tik­Linked­My­Face­Pin­Insta­Me­Gram­Space­We­In­Tok­Trest­Book) describing the various relationships among Imperial units, and from there, I found a much better way to meaure ⅚ cups—measure out 1 cup, then remove 8 teaspoons; much better than the 2 ⅓ cup measures (or 1 ⅔ if available), a 1½ tablespoon and a ½ teaspoon. And maybe this will help someone else twenty years down the line.

Monday, May 13, 2024

Remembrance of enlightened palms past

The image at the bottom of this page reminds me of the time I used to photograph enlightened palms, but it never occurred to me that one could enlight trees with fireflies (we don't get fireflies down here in Lower Sheol, which may be the reason why). The pictures I took with the Christmas lights used an exposure of a few seconds; I wonder how long an exposure was used for the firefly photo.

Tesla, Edison, and who actually fought the War of Currents?

I used to think Thomas Edison was a self-aggrandizing business man who took the credit for the inventions his employees made, and Nikola Tesla was the real deal—a genius inventor who was actually responsible for most of our technology based on electricity. But now? Having watched the 4½ hour long video “Most Everything You Know About Nikola Tesla and Thomas Edison is Probably Wrong” (and yes, it's four and a half hours long!) I'm not so sure my assessment is correct. The long video goes deep into the history of Tesla, Edison, and the War of the Currents where it wasn't Tesla vs. Edison, but Westinghouse (the company) vs. Edison (the copmany).

Tesla might have been a genius, but not all this theories about physics and electronics were correct and later in life he went a bit … crazy … to say the least (he fell in love with a pidgeon and said he created incredible inventions without having actually … you know … built the incredible inventions). And Edison might have been a self-aggrandizing business man, but he credited his team and oftem times, his team didn't invent the technology, but improved upon existing designs (to the point where he learned 6,000 ways not to build a lightbulb).

And the whole thing about Edison electrocuting an elephant (or at least animals) to show how dangerous alternating current was? Eh … not exactly. And he did not invent the electric chair.

Yes, it's a long video, but if you are interested at all in Tesla and/or Edison, it's worth the time to watch. It got me to rethink how I think about Tesla and Edison.

Wednesday, May 15, 2024

Extreme Monopoly Board Game Knockoff, Boca Raton edition

About two weeks ago I was at a local Walgreens in Boca Raton when I came across something unusual. I meant to blog about it then, but alas, I just now got a round tuit.

Anyway, what I found:

[A picture of a game clearly based on Monopoly] Everglades University? Who ever heard of Everglades University? And in Boca Raton? I've been in Boca Raton for over 30 years and this is the first I've heard of it! [Back of the box showing the board game and pieces] A pretzel? Really? A pretzel? The original Monopoly pieces are better related to Boca Raton than a preztel!

I amazed this even exists! I wonder who's idea this even was? The Boca Raton Chamber of Commerce?

Anyway, it's clearly a knockoff of Monopoly, as you won't find it for sale at Hasbro. It's actually made by Late for the Sky, which seems to make games based off Monopoly, or should I say, The Landlord's Game which is completely in the public domain (wink wink nudge nudge say no more say no more, unlike Monopoly. But Boca Raton Opoly sure looks like Monopoly, walks like Monopoly, and probably quacks like Monopoly, so I wonder how they get away with this?

Perhaps by flying under the radar of Habro?

Update later this day

Apparently, Hasbro doesn't care:

Leaders at Late for the Sky say Monopoly gameplay is not copyrighted, meaning any version of the game can be created as long as the board, pieces and names within the game are different from the original version.

Via my friend Jeff Cuscutis on Linked­Pin­My­Face­Tik­Insta­Me­Trest­We­Gram­Book­In­Tok­Space, Business making Monopoly games based on Carolina towns

Monday, May 27, 2024

How does TLS use less CPU than plain TCP?

I have two services written in Lua—a gopher server and a Gemini server. The both roughly serve the same data (mainly my blog) and yet, the gopher server accumulates more CPU time than the Gemini server, despite that the Gemini server uses TLS and serves more requests. And not by a little bit either:

CPU utilization
gopher 17:26
Gemini 0:45

So I started investigating the issue. It wasn't TCP_NODELAY (via Lobsters) as latency wasn't the issue (but I disabled Nagle's algorithm anyway).

Looking further into the issue, it seemed to be one of buffering. the code was not buffering any data with TCP; furthermore, the code was issuing tons of small writes. My thinking here was—Of course! The TCP code was making tons of system calls, whereas the TLS code (thanks to the library I'm using) must be doing buffering for me.

So I added buffering to the gopher server, and now, after about 12 hours (where I restarted both servers) I have:

new CPU utilization
gopher 2:25
Gemini 2:13

I … I don't know what to make of this. Obviously, things have improved for gopher, but did I somehow make Gemini worse? (I did change some low level code that both TCP and TLS use; I use full buffering for TCP, no buffering for TLS). Is the load more evenly spread?

It's clear that gopher is still accumulating more CPU time, just not as bad as it was. Perhaps more buffering is required? I'll leave this for a few days and see what happens.

Thursday, May 30, 2024

Profile results are as expected as the Spanish Inquisition

I'm not upset at rewriting the code that handles the network buffering as it needed work, but I'm still seeing a disporportionate amount of CPU time accumluate on the supposedly simpler protocol gopher. The most popular requests of both my gopher server and Gemini server are entries from my blog, so I take a look at the code that handles such requests for both servers. Yes, the gohper server has a bit more code dealing with links than the Gemini server (because gopher URLs are almost, but not entirely, like http URLs—and the small differences are annoying), but I'm not seeing anything that stands out. Yes, the code is not quite twice as much, but the CPU utilization is more than three times as much (as of writing this).

I have no other choice at this point and I constantly relearn this lession over and over again: if I'm looking into a performance issue, profile the code under question! Profile profile profile!

The code is in Lua and as it happens, I've profiled Lua code before. First, I want to answer this question: how much code does it take to serve a request? And I figure measuring the lines of code run is a good answer to that. I can get a baseline from that. And the code to answer that is a very easy four line change to each server:

local function main(iostream)
  local count = 0
  debug.sethook(function() count = count + 1 end,'line')

  -- The rest of the main code

  debug.sethook()
  syslog('notice',"count=%d",count)
end

I fire up the servers locally, make a decently sized request to each, and I get my results:

Lines of code to serve a request
gopher 457035
gemini 22661

WHAT THE LITERAL XXXX‽

[Well, there's your problem! —Editor] [Just … gaaaaaaah! —Sean]

I'm constantly surprised at the results of profiling—it's almost never what I think the problem is. And here, it's clear that I messed up pretty bad somewhere in the gopher code.

Now off to more profiling to see where it all goes pear shaped.

Unicode. Why did it have to be Unicode?

Well, I have my answer. I first found a smaller request that exhibits the behavior as to not generate half a million lines of output:

Lines of code executed for a small request
gopher 2549
gemini 564

Good. Two and a half thousand lines of code is tractable. Now, just to show how easy it is to profile Lua code, here's the code I'm using for my gopher server:

local profile = {}

local function doprofile()
  local info = debug.getinfo(2)
  local name = info.name or "..."
  local file = info.source or "@"
  local key  = string.format("%s$%s(%d)",file,name,info.currentline)
  if not profile[key] then
    profile[key] = 1
  else
    profile[key] = profile[key] + 1
  end
end

For each line of code executed, we get the filename, the function name and the line of code that's executing, turn that into a key, and use that to count the number of times that line of code is executed. Easy. And then some code to dump the results:

local function cleanup()
  local results = {}
  for name,value in pairs(profile) do
    results[#results + 1] = { file = name , count = value }
  end
  
  table.sort(results,function(a,b)
    if a.count > b.count then
      return true
    elseif a.count < b.count then
      return false
    else
      return a.file < b.file
    end
  end)
  
  local f = io.open("/tmp/dump.txt","w")
  for i = 1 , #results do
    f:write(string.format("%6d %s\n",results[i].count,results[i].file))
  end
  f:close()  
end

We sort the results based on line count, then alphabetically by key. And like before:

local function main(iostream)
  debug.sethook(doprofile,'line')

  -- The rest of the main code

  debug.sethook()
  cleanup()
end

I make the request and get some results:

   215 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(202)
   211 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(203)
   211 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(204)
   211 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(268)
   210 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(282)
   210 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(283)
   169 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(219)
   169 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(224)
   169 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(239)
   169 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(240)
    42 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(205)
    42 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(206)
    42 @/usr/local/share/lua/5.4/org/conman/string.lua$wrapt(207)
    17 @port70.lua$...(272)
    17 @port70.lua$...(273)
     9 @/usr/local/share/lua/5.4/org/conman/net/ios.lua$write(547)
...

Oh.

Yeah.

That.

Obvious in hindsight.

I completely forgot about that.

Okay. The function in question, wrapt(), wraps text and it's a rather heavy function due to Unicode (and I'm not even following the full specification there). This is the major difference between the gopher and Gemini servers—I don't wrap text for Gemini (the clients handle that). I guess I'll have to drop down to C if I want to speed this up.

Sigh.

Obligatory Picture

An abstract representation of where you're coming from]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

Obligatory AI Disclaimer

No AI was used in the making of this site, unless otherwise noted.

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: https://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

https://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.