The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Sunday, July 01, 2018

Replacing the battery on my iPhone should not require rebooting my Mac

It wasn't as bad as that time 2½ years ago (no, really! you have no idea how bad it was back then) but it still left a bad taste in my mouth. The battery on my iPhone wasn't lasting as long as it used to. I would be lucky get to a day's charge out of it, and I don't really use it all that much (using Shazam was a death sentance for my iPhone—I'd usually get one use out of it before the screen would go dark).

On Thursday, Bunny and I stop by the Apple Store only to find out that a battery replacement would require an appointment, and the first one was for Saturday. Okay, Saturday it is.

At the appointed time on Saturday, we arrive at the store. I will say we didn't have to wait all that long, but on the down side, I would have to leave the phone overnight for the replacement.

Okay, picking it up on Sunday it is.

Today, I got the phone back. It works. I was able to check multiple songs with Shazam with no discernible battery drain, so things are looking up. Until I decide to pull some pictures off the phone.

Usually, I just plug in the phone to the Mac-mini I have, run the Photos app and import them. Only this time I coudn't. What should have been a minute operation turned out to be a grueling half hour torture test trying to find out what happened to the “import function” of Photos. I was able to finally resolve the issue by rebooting the Mac-mini!

Seriously, Apple? You're better than Microsoft! This shouldn't require a reboot! Come on!

I swear, they really need to bring back Steve Jobs.

Wednesday, July 04, 2018

All's quiet on the holiday front

Right now it's all quiet outside, but I suspect that won't last the day.

So today, after you have read The Constitution, but before you hand over your beer and say, “Watch this!” please remember to be careful. I would hate for this to happen to you:

[Fireworks that exploded at ground level, with a capture that says, “Amateurs: There's a Reason Professionals Exist.”]

Notes on an overheard conversation

“So the weather report states thunderstorms for the Fourth.”

“Oh! At night?”

“Mostly the afternoon and ending around 8:00 pm.”

“Darn!”

“Oh, expecting to get out of walking to the Boca Raton fireworks show?”

“No.”

“No?”

“No. I was hoping our neighbor wouldn't blow us all up!”

Neither heat nor rain nor threat of thunderstorms stays the neighbor from the swift lighting of firework rounds

It sounds like Desert Storm outside.

[I guess it beats walking down to DeHoernle Park in a potential thunderstorm to watch fireworks.]

I guess the threat of thunderstorms wasn't enough to deter the neighbor.

Sigh.

Thursday, July 05, 2018

Abandon all hope ye who enter the sucking vortex of ill-marked, multply-named, non-Euclidean roadways known as Orlando

Bunny and I are on the road for a weekend getaway (more on that later). We are headed back into that maelstrom of roads known as Orlando. On the downside, said maelstrom has extended itself a significant distance outside of Orlando and for the first time ever, we missed the turn at Ft. Pierce and ended up on a nearly 20 mile detour.

Then, we nearly ended up missing a service station on the Turnpike—don't ask how that happened. It did. And the maelstrom wrought it's malevolent influence over us to the point where we nearly called the whole weekend off.

And finally, Google Maps directed to the rear of the hotel.

I swear, Orlando is actively trying to kill us.

Singing with scissors

Bunny first proposed this trip several months ago and I was initiallty dubious. I mean, a trip to view barbershop quartets! I was reluctant to go, but Bunny assured me that I was under no threat of losing hair or limbs. No, the only thing under threat was the ears.

We finally arrived at the hotel. As we were walking to our room, it seemed to me that barbershop quartets have shrunk over the years, but the lack of matching outfits, no singing and a definite lack of scissors meant that no, the people I was seeing were in fact Little People milling about. Too many to be conincidental and indeed, there really is a conference of Little People scheduled at the same time and not some Lynchian nightmare.

At least, not yet.

About this hotel

The hotel is quite nice, but it has … a few oddities that I've never seen before. The first, and most innocuous, are all the automatic lights about the place. There's one in the bathroom, one beneath the night stand and the entire garage appears to have motion sensative lights (it certainly got brighter in the garage when I went back this evening to pick up a USB cable I left in the car). It was rather unexpected, but a pleasant unexpectedness.

The next was also unexpected, but to me, not a pleasant unexpectedness. The mirror in the bathroom is nice, large and certainly bright:

[Mirror, mirror, on the wall, who's the geekiest of them all?]

But there's this odd greyed out section near the bottom. Here's the not-so pleasant unexpectedness:

[I am, for I have a built in television set and you are just stuck with a smart phone.]

It's a television screen! (and yes, when I turned it on, it was tuned to Fox News. CNN wouldn't have been any better)

Really?

First off, are people so addicted to television that one is needed in the bathroom? [Yes. —Editor]. And second, you can't even watch it while sitting on the toilet! You can only watch the television while standing in front of the mirror. That's just … just … weirdly random.

And finally there is this:

[You may look. You may even touch. But on penalty of death, not not remove the elevator stick from the elevator.]

A stick. In the elevator. Just hanging there with a sign saying not to remove it. And it's not limited to one elevator. No, each elevator has a stick, hanging on a sign saying not to remove it. They're just there. I don't know.

Groups of people singing four part harmony. Little People milling about. Elevator sticks. Automatic lightbulbs. Television mirrors.

Orlando is weird!

Friday, July 06, 2018

Notes on the epiphany I had on elevator sticks while I slept

It may have been obvious to some, but not to me. It suddenly dawned on me what the elevator sticks were all about. The puzzle pieces were all there, I just hadn't made the connection.

I am not the target audience for using the elevator stick, nor is the staff the target audience. No, the target audience is the Little People, so they can use it to push the elevator button in case they can't reach it.

It's obvious in hindsight.

While I joked about the elevator sticks, it's not something I'm used to seeing. And here they were, just hanging out in elevators with no obvious usage to me, some one who can reach all the buttons. It wasn't until I was sleeping and my subconscience told me an old riddle I heard as a kid: “A man lives in a tall apartment building. On some days, he rides the elevator to his apartment on the 20th floor. On other days, he rides the elevator to the 18th floor and walks up two more flights. Why?”

Oh.

“Yeah. ‘Ooh, ah,’ that's how it always starts. But then later there's running and screaming.”

[The other half is the madness that slowly creeps in as you attempt to make sense of the non-Euclidean nature of space around the Kingdom of the Rat God.]

Don't let the propaganda fool you—Orlando is trying to kill you.

Or at least drive you mad and empty your wallet.

One must always make sure to get the cruelty-free pizza

In the lobby of the hotel is a store by the name of “Market Orlando.” I can only describe it as if hippies-cum-yuppies from Berkeley ran 7-Eleven. Gluten-free, organic, farm-fresh snacks, chips, salads and sandwiches.

Oh, and cruelty-free pizzas. Can't forget those.

It has an extensive wine selection (Napa Valley present!), but the soda selection is pretty much limited to Pepsi and Diet Pepsi (which sucks because I'm a Coca-cola person myself). There's also about a bazillion brands of artisanal water.

Oddly enough, I did not see any granola for sale.

I'm telling you, Orlando is weird! When it's not trying to kill you.

Sunday, July 08, 2018

At the roadways of madness

The sucking vortex of non-Euclidean roadways known as Orlando began it's influence of madness over us hours before we left. We were planning on how best to get back to the Turnpike from the hotel when Bunny and I started arguing yet again. She wanted to follow the torturous path we took to get here, which involved an intricate series of turns and switchbacks while carefully aligning ourselves within certain lanes at certain times. I wanted to follow yet another intricate path of a torturous series of turns and switchbacks while carefully aligning outselves within certain times at certain lanes. Either route lead to madness.

Also at stake was who would be the unfortunate driver to brave the maelstrom of roads until the relative safety of the Turnpike. I don't recall who drove. All I know is we barely made it out alive.

Unrelated, I wish to leave you with a mental image of the most surreal observation I saw on the way out of the hotel: a Little Person and their dog riding a Segway down the hall.

Monday, July 09, 2018

At least now no one has to pay to have their information protected

So I received the following email from my registrar:

From
<XXXXXXXX­XXXXXXXX­XXXXXXX>
To
<sean@conman.org>
Subject
WHOIS Data Confirmation for x-grey.com
Date
Mon, 9 Jul 2018 09:30:23 -0400 (EDT)

Dear Valued Customer,

ICANN, the organization responsible for the stability of the Internet, requires that each domain name registrant be given the opportunity to correct any inaccurate contact data (WHOIS data) associated with a domain name registration. Our records for your domain are as follows:

x-grey.com

Domain Name: X-GREY.COM
Registry Domain ID: 1325416434_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.domain.com
Registrar URL: www.domain.com
Updated Date: 2017-11-08T05:11:47
Creation Date: 2017-11-07T22:32:17
Registrar Registration Expiration Date: 2018-11-08T01:32:21
Reseller: Dotster.com
Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
Domain Status: clientUpdateProhibited https://icann.org/epp#clientUpdateProhibited
Registry Registrant ID:
Registrant Name: Data Protected Data Protected
Registrant Organization: Data Protected
Registrant Street: 123 Data Protected
Registrant City: Toronto
Registrant State/Province: ON
Registrant Postal Code: M6K 3M1
Registrant Country: CA
Registrant Phone: +1.0000000000
Registrant Phone Ext:
Registrant Fax: +1.0000000000
Registrant Fax Ext:
Registrant Email: noreply@data-protected.net
Registry Admin ID:
Admin Name: Data Protected Data Protected
Admin Organization: Data Protected
Admin Street: 123 Data Protected
Admin City: Toronto
Admin State/Province: ON
Admin Postal Code: M6K 3M1
Admin Country: CA
Admin Phone: +1.0000000000
Admin Phone Ext:
Admin Fax: +1.0000000000
Admin Fax Ext:
Admin Email: noreply@data-protected.net
Registry Tech ID:
Tech Name: Data Protected Data Protected
Tech Organization: Data Protected
Tech Street: 123 Data Protected
Tech City: Toronto
Tech State/Province: ON
Tech Postal Code: M6K 3M1
Tech Country: CA
Tech Phone: +1.0000000000
Tech Phone Ext:
Tech Fax: +1.0000000000
Tech Fax Ext:
Tech Email: noreply@data-protected.net
Registry Billing ID:
Billing Name: Data Protected Data Protected
Billing Organization: Data Protected
Billing Street: 123 Data Protected
Billing City: Toronto
Billing State/Province: ON
Billing Postal Code: M6K 3M1
Billing Country: CA
Billing Phone: +1.0000000000
Billing Phone Ext:
Billing Fax: +1.0000000000
Billing Fax Ext:
Billing Email: noreply@data-protected.net
Name Server: ns1.armigeron.com
Name Server: ns2.armigeron.com
URL of the ICANN WHOIS Data Problem Reporting System: http://wdprs.internic.net/
>>> Last update of WHOIS database: 2017-11-08T05:11:47 <<<

“For more information on Whois status codes, please visit https://icann.org/epp”

Registration Service Provider:
    XXXXXXXX­XXXX XXXXXXXX­XXXXXXXX­XXXXXXX
    +1.X­XXXXXXXX­X
    This company may be contacted for domain login/passwords,
    DNS/Nameserver changes, and general domain support questions.

To review and update your WHOIS contact information, please log into our management interface at: XXXXXXXX­XXXXXXXX­XXXXXXXX­XXXXXXXX­XXX

If any of the information above is inaccurate, you should correct it. If all of the information above is accurate, you do not need to take any action.

Please remember that under the terms of your registration agreement, the provision of false WHOIS information can be grounds for cancellation of your domain name registration. If you have any questions or comments regarding ICANN's policy, please visit the following link http://www.icann.org/registrars/wdrp.htm

Thank you for your attention.

Best regards,
XXXXXXX

It's kind of hard to review my information when it's protected from my viewing it for my protection. I think my registrar is going a bit overboard in complying with the GDPR.

Sigh.

Yes. It is music. I identified it. What more do you want?

Bunny and I had dinner at a nearby Chinese restaurant. As we ate, Bunny mentioned that the music we were listening to must be the Chinese equivalent to elevator music. I concurred, and because I was curious to know what it was, I thought I might try to use Shazam to identify it.

It did.

[Of course! It was obvious!]

And now I know.

Monday, July 16, 2018

I don't think this is what McDonald's had in mind when they said “you deserve a break today”

I was at The Scottish Place to pick up some lunch. I placed my order, and while waiting in line, I noticed the telltale signs of Microsoft Windows:

[Image of a crashed Windows box: A stopped clock is right twice a day, unless it's one of those 24-hour clocks which makes it right only once a day, but a crashed Windows box isn't right at any time of day (and some would say that even a functioning Windows box isn't right at any time of the day, but I digress ...)]

Ah, Windows, keep on being your special snowflake self.

But in all seriousness, who set this up? If you are going to run any computer in a public space, I would hope those setting it up would run the bare minimum for the service required (and in this case, just the operating system and whatever program it is that runs the display) and remove everything else. Then, for whatever left is running, is there not some Windows setting to just have it automatically reboot upon an error? Or simply not display warning messages like this on the display meant for public viewing?

I have to wonder how long this message has been here, waiting for someone, anyone, to select an option (or to restart, reboot or reinstall). I wonder if management even cares?

Thursday, July 19, 2018

A sane and easy to use TLS library! Will wonders never cease!

I'm still fighting the stupidity at work, but it's becoming aparent that it's a fait accompli and we're looking at a bunch of REST/HTTPS über alles Kool-Aid™ in an area where time is critical.

Sigh.

So I'm looking around at what I can use to support the “S” in HTTPS that doesn't involve diving into the horror show that is OpenSSL. A library that can still encrypt and decrypt data when it isn't managing the network connections on the program's behalf (because the program is already managing the network connections). It can be complicated, but it must be sane to use.

I was pointed to libtls, which comes with LibreSSL. Not only is this sane, but it's easy to use. I'm simply amazed at how easy.

In just an hour, and only reading the man pages, I was able to write a simple program that fetches a page from a secure website. And most of the code is just there to report any errors that happen. It's a very straight forward program.

Another hour or two, and I had a program where the library does not control the network connection. Which means we can (probably) use this in our existing architecture.

A few more hours, and I was able to replicate the initial C program in Lua:

local tls = require "org.flummux.tls"

-- *****************************************************************

local function okay(v,err)
  if not v then
    print(">>>",err)
    os.exit(1)
  end
  return v
end

-- *****************************************************************

if #arg == 0 then
  io.stderr:write(string.format("usage: %s host resource\n",arg[0]))
  os.exit(1)
end

local config = tls.config()
local ctx    = tls.client()

okay(config:set_protocols "all")
okay(ctx:configure(config))
okay(ctx:connect(arg[1],"https"))
okay(ctx:write(string.format(
     "GET %s HTTP/1.1\r\n"
  .. "Host: %s\r\n"
  .. "User-Agent: TLSTester/1.0 (TLS Testing Program Lua)\r\n"
  .. "Connection: close\r\n"
  .. "Accept: */*\r\n"
  .. "\r\n",
     arg[2],
     arg[1]
)))

while true do
  local bytes = okay(ctx:read(1024))
  if bytes == "" then break end
  io.stdout:write(bytes)
end

I had to write my own Lua wrapper for LibreSSL. The existing ones (and I found only two) weren't up to my standards for use, but it wasn't terribly hard to get the above working.

The next step is expanding the Lua module to see if I can get it working with our networking code we use. I am optimistic about this.

But I am not optimistic about having to use this at work.

Monday, July 23, 2018

Managing TLS connections using Lua and Lua coroutines

Getting libtls working with Lua wasn't as straightforward and I thought it would be. It works (for the most part) but I had to change my entire approach. The code is an ugly mess and there's quite a bit of duplication in several spots.

But! I can request web pages, in Lua, via HTTPS in an event loop based around select() (or poll() or epoll() or whatever is the low level event notification scheme used). Woot! And I'm going into excruciating detail on this.

Back on Friday, when I wrote some “proof-of-concept” code, I had thought I could switch coroutines in the user-supplied I/O callback routines (and if coroutines existed in C, that is where you would yield to another coroutine). It was easy enough to extend the callback to a Lua routine— in the routine that wraps the libtls function tls_connect_cbs():

static int Ltls_connect_cbs(lua_State *L)
{
  struct tls **tls = lua_touserdata(L,1);
  int rc           = tls_connect_cbs(
			*tls,
			Xtls_read,
			Xtls_write,
			L,
			luaL_checkstring(L,2)
		     );
  
  if (rc != 0)
  {
    lua_pushboolean(L,false);
    return 1;
  }
  
  lua_settop(L,5);
  lua_pushlightuserdata(L,*tls);
  lua_getuservalue(L,1);
  lua_pushvalue(L,1);
  lua_setfield(L,-2,"_ctx");
  lua_pushvalue(L,2);
  lua_setfield(L,-2,"_servername");
  lua_pushvalue(L,3);
  lua_setfield(L,-2,"_userdata");
  lua_pushvalue(L,4);
  lua_setfield(L,-2,"_readf");
  lua_pushvalue(L,5);
  lua_setfield(L,-2,"_writef");
  
  lua_settable(L,LUA_REGISTRYINDEX);
  lua_pushboolean(L,true);
  return 1;
}

I pass in the two callback functions, and I'm using the Lua state context as the userdata in the callbacks. I then create a Lua table, populate it with some useful information, such as the Lua functions to call, and associate it in the Lua registry with the value of the libtls context. Then, when libtls calls one of the callbacks:

static ssize_t Xtls_write(struct tls *tls,void const *buf,size_t buflen,void *cb_arg)
{
  lua_State *L = cb_arg;
  ssize_t    len;
  
  lua_pushlightuserdata(L,tls);
  lua_gettable(L,LUA_REGISTRYINDEX);
  lua_getfield(L,-1,"_writef");
  lua_getfield(L,-2,"_ctx");
  lua_pushlstring(L,buf,buflen);
  lua_getfield(L,-4,"_userdata");
  lua_call(L,3,1);
  
  len = lua_tonumber(L,-1);
  lua_pop(L,2);
  return len;
}

I get the Lua state via the user argument. From that, and the libtls context, I obtain the data I cached into the Lua table, which give me the Lua function to call. Said function can then call coroutine.yield().

Straightforward, easy, and wrong! I got the dreaded “attempt to yield across metamethod/C-call boundary” error. Darn.

The attempted flow looks like (yellow boxes are Lua functions; green boxes are C functions):

{data=tls.read()} → [Ltls_read(lua)] → [tls_read(ctx)] → [Xtls_read(ctx,lua)] → [lua_call(lua)] → {my_callback()} → {coroutine.yield()} {}=Lua function []=C function

There are four layers of C functions that can't be yielded through. Lua does have a way of dealing with intervening C functions, but it's somewhat clunky.

{luaf_a()} → [cf_orig(lua)] → [lua_callk(lua,cf_c)] → {luaf_b()} → {coroutine.yield} / {coroutine.resume} → {luaf_b()*} → [cf_c(lua)*] → {luaf_a()}

In this case, the Lua function lua_callk() is handled specially so it doesn't cause an error. The function cf() needs to be split in half—the portion prior to calling into Lua, and the second half to handle things after a potential call to coroutine.yield(). That's represented above by the functions cf_orig() and cf_c(). The “*” represent the functions returning, not calling. coroutine.resume() will restart luaf_b() right after it's call to coroutine.yield(). And when luaf_b() returns, it “returns” to cf_c(), which does whatever and finally returns, which “returns” to luaf_a().

But in the case I'm dealing with just doesn't work with that model. The code calling into Lua doesn't have the signature:

int function(lua_State *lua_State);

but the signature:

ssize_t function(struct tls *ctx,void *buf,size_t buflen,void *cb_arg);

Not only are the return types different, but they have completely different semantics—for libtls, it's the number of bytes transferred, whereas for Lua, it's the number of items being returned to Lua.

No, I had to rethink the entire approach, and do the call to coroutine.yield() a bit higher in the call stack. Which also meant I had to push dealing with TLS_WANT_POLLIN and TLS_WANT_POLLOUT back to the caller. The documentation states:

In the case of blocking file descriptors, the same function call should be repeated immediately. In the case of non-blocking file descriptors, the same function call should be repeated when the required condition has been met.

And here I was, trying to hide such concerns from the user. Ah well.

I eventually got it working, but man, is it ugly. The Lua code wants to read data, so I have to call into libtls. That in turn, calls back into my code, and if I don't have any, I need to return TLS_WANT_POLLIN, which bubbles up through libtls back to my code, which can then yield.

Meanwhile, from the other end, I get data from the network. I can't just feed it into libtls, I have to feed it when libtls calls the callback for the data. But when I get the data, I may need to resume the coroutine, so I have to track that information as well.

I can almost understand the code (and yes, I wrote it; did I mention it's ugly?)

But I'm happy. The following code works in my existing network framework (boy does that sound wierd):

local function request(item)
  syslog('debug',"requesting %s",item.url)
  local u = url:match(item.url)

  -- -------------------------------------------------------
  -- asynchronous DNS lookup---blocks the current coroutine
  -- until a result is returned via the network.
  -- -------------------------------------------------------

  local addr = dns.address(u.host,'ip','tcp',u.port)
  
  if not addr then
    syslog('error',"finished %s---could not look up address",u.host)
    return
  end

  -- ---------------------------------------------------------
  -- This has nothing to do with the iPhone operating system,
  -- but everything to do with "Input/Output Stream"
  -- ---------------------------------------------------------

  local ios
  
  if u.scheme == 'http' then
    ios = tcp.connecta(addr[1]) -- connect via TCP
  else
    ios = tls.connecta(addr[1],u.host) -- connect via TLS
  end
  
  if not ios then
    syslog('error',"could not connect to %s",u.host)
    return
  end
  
  local path    = table.concat(u.path,'/')
  local fhname  = "header/" .. item.hdr
  local fbname  = "body/"   .. item.body
  local fh      = io.open(fhname,"w")
  local fb      = io.open(fbname,"w")
  
  local command = string.format([[
GET /%s HTTP/1.0
Host: %s
Connection: close
User-Agent: TLSTest/2.0 (Lua TLS Testing Program)
Accept: */*

]],path,u.host)

  ios:write(command)
  
  fh:write(ios:read("*h"))
  
  repeat
    local data = ios:read("*a")
    fb:write(data)
  until data == ""
  
  fb:close()
  fh:close()
  ios:close()

  syslog('debug',"finished %s %s",item.url,tostring(addr[1]))
end

Any number of requests can be started and they all run concurrently, which is just what I wanted.

Now, the code I have for the Lua wrapper for libtls covers just what I need to do this. More work is required to finish covering the rest of the API. I also have to clean up the Lua code that backs the above sample code so that I might have a chance of understanding it at some point in the future.

And until I get the working code published, you can look at the “proof-of-concept” Lua coroutine code I worked from (and no, the above code sample will not work as is with this “proof-of-concept” code).

Obligatory Picture

[The future's so bright, I gotta wear shades]

Obligatory Contact Info

Obligatory Feeds

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site: https://boston.conman.org/, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

https://boston.conman.org/2000/08/01

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2024 by Sean Conner. All Rights Reserved.