The Boston Diaries

The ongoing saga of a programmer who doesn't live in Boston, nor does he even like Boston, but yet named his weblog/journal “The Boston Diaries.”

Go figure.

Saturday, May 20, 2017

For those times when downloading over the Internet is too slow

Andrew Tanenbaum has been quoted as saying, “Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.” And yes, if you have a large amount of data to transfer, sometimes it is better to physically move it than transfer it over the Intarwebs.

And if you have a metric buttload of data?

AWS Snowmobile is an Exabyte-scale data transfer service used to move extremely large amounts of data to AWS. You can transfer up to 100PB per Snowmobile, a 45-foot long ruggedized shipping container, pulled by a semi-trailer truck. Snowmobile makes it easy to move massive volumes of data to the cloud, including video libraries, image repositories, or even a complete data center migration. Transferring data with Snowmobile is secure, fast and cost effective.

Via Lobsters, AWS Snowmobile – Massive Exabyte-Scale Data Transfer Service

And for the record, a petabyte (PB) is 1,125,899,906,842,624 bytes which I think is the expected size of the next version of Windows.

Thursday, May 18, 2017

Swimming in gold

Batman has the Batcave, Superman has his Fortress of Solitude, and Scrooge McDuck has his money bin. For 70 years, the maternal uncle of Disney’s Donald Duck has been portrayed as a thrifty—some might say miserly—presence in cartoons and comics, a waterfowl who has such deep affection for his fortune that he enjoys diving into his piles of gold and luxuriating in them.

It’s a rather gross display of money worship, but is it practical? Can anyone, including an anthropomorphic Pekin duck, actually swim in their own money, or would diving headfirst into a pile of metal result only in catastrophic injury?

Via GoogleMyFacePlusSpaceBook, Fact Check: A Physicist Weighs In On Whether Scrooge McDuck Could Actually Swim in a Pool of Gold Coins | Mental Floss

Sadly, no. Even Carl Barks knew this and on several occasions had Scrooge mention it was a trick.

[“Unca Scrooge, how come you dived all the way from that bluff, and you didn't get hurt?”—Huey, Dewey and Louie] [“Well, I'll admit—it's a trick!”—Uncle Scrooge]

But it's still fun to fantasize about swimming in three cubic acres of gold coins though.

Tuesday, April 18, 2017

Welcome back, Bunny

Because it's the anniversary of our first meeting face-to-face (Bunny and I met online), I thought I would take Bunny to Roots Italian Kitchen, the place we stumbled into on St. Valentine's Day. Fortunately for us, we were able to find parking outside the non-Euclidian, Escheresque parking garage from the 8th Dimension sitting smack-dab in downtown Boca Raton.

“Welcome back! We are so glad you have returned to dine with us!” the owner said to us as soon as we entered.

Welcome back? I'm thinking two things at the same time. One, they must not get many customers to recognize us after nine weeks. Two, they must really know how to run a restaurant.

Not only did we end up with the same waiter, he too, remembered us from nine weeks ago!

[I came close to licking the bowl clean.  And this was just the salad!  Yes, the food is that good.  And I'm still hoping pictures of empty dinner plates becomes a thing.]

The Chef even came out and discussed the food he was about to serve us. This place is insanely good.

I'm really glad we found this place.

We're not all dummies at the office

Edvard has some new friends:

[Where are we?  Why is it so hot?  And why are we in this basket?]

They were just hanging out in the conference room at the Ft. Lauderdale Office of the Corporation. On the up side, they're pretty quiet. On the down side, they're no showroom dummies.

Monday, April 17, 2017

Notes on the hundredth overheard conversation about an expired warranty




“Hello there, sir. This is the Chevrolet Division of General Motors about your Chevy Impala. Do you still own it?”

“Sigh. Yes.”

“And about how many miles are on it?”

“About 40,000.”

“Our records indicate that your factory warranty has expired.”

“Yes, you and a dozen other companies call me weekly to remind me of that fact.”

“So you are aware that it's expired, good. Can we transfer you to a specialist to talk about your options?”

“Sigh. Yes.”

“Okay, hold on sir.”

“Holding on by my finger tips.”

“Hello, sir?”

“I'm still here.”

“Could you please hold?”

“Sure, I have a few minutes before my fingers lose their grip and I plummet to my death.”

“Thank you. Um … um … what? Oh … there … but … Can we call you back? I'm currently having issues dealing with this call.”

“Sigh. Okay.”

“It will only be a few minutes.”

“I'm sure it will.”

“Thank you for calling us about your warranty needs.”

“I didn't call—”


Friday, March 17, 2017

Brain Candy

Adam Savage, of Mythbusters fame, is yet again on tour. Only his co-host is not Jamie Hyneman (who no longer wishes to tour and is on to other things), but educator Michael Stevens. Their show—Brain Candy Live.

Bunny and I saw them at The Parker Playhouse in Ft. Lauderdale, and it was an amazing show. Light and breezy, they presented a lot of scientific concepts that one might not even think about, presented in an entertaining and humorous way. So entertaining that it did not feel like two hours.

Monday, February 27, 2017

The Lego Batman Movie

I seriously did not expect “The Lego Batman Movie” to be a romantic comedy. A somewhat twisted romantic comedy, but a romantic comedy none-the-less. At the same time, it's a good Batman movie, really getting the notion of his character—he is the Batman, and Bruce Wayne is the mask he wears. It also showcases the conflict between him and Superman way better than “Batman v Superman: Dawn of Justice (way better—skip that movie and watch this twice).

Plot wise, there isn't much here you already haven't seen before. The Joker wants to take over Gotham and Batman has to stop him, while learning a valuable life lesson. But you aren't going to watch this for the plot, but for the humor and character interactions. Oh, and the meta humor. Alfred: “Sir, I have seen you go through similar phases in 2016 and 2012 and 2008 and 2005 and 1997 and 1995 and 1992 and 1989 and that wierd one in 1966.” Batman: “I have aged phenomenally.” Great stuff.

And lest I forget to mention it, Batman's Rogue Gallery, including Condiment King (yes, Bunny, he really is a Batman villian) show up to help ruin Batman's day.

I was also surprised at the cameos—and no, I don't mean actors who supplied voices, but actual character cameos that showed up for the final act of the movie. That I did not see, and it was most entertaining.

So come for the comedy, stay for the shark repllent Batspray. You won't be disappointed.

An implementation of coroutines for C

I use coroutines in Lua and I do miss having them in C. Sure, there are a number of coroutine implementations for C, but they all generally fall into two camps:

  1. they rely upon setjmp()/longjmp(), which is problematic because it is seriously abusing those two functions (which exist to give C a form of exception) and
  2. they rely upon ucontext.h, which has been deprecated by POSIX (and the API is a bit clunky for what I want to do)

But really, I just wanted to write my own implementation, because.

So the idea is to write some code that works like:

extern int       coroutine_create(coroutine__s **pcp,coroutine_function__f fun);
extern uintptr_t coroutine_yield (coroutine__s *co,uintptr_t value);
extern int       coroutine_free  (coroutine__s *co);

uintptr_t sub_task(coroutine__s *self,uintptr_t value) /* [1] */
  value = coroutine_yield(self,value); /* [2] */
	/* [3] */
  value = coroutine_yield(self,value); /* [4] */
	/* [5] */
  value = coroutine_yield(self,value); /* [6] */
	/* [7] */
  return value;

void main_task(void)
  coroutine__s *co;
  uintptr_t     v;

  coroutine_create(&co,sub_task); /* [8] magic here! */
	/* [ 9] */
  v = coroutine_yield(co,v); /* [10] */
	/* [11] */
  v = coroutine_yield(co,v); /* [12] */
	/* [13] */
  v = coroutine_yield(co,v); /* [14] */
	/* [15] */
  v = coroutine_yield(co,v); /* [16] */
	/* [17] */
  v = coroutine_yield(co,v); /* [18] */
	/* [19] */

It's a contrived example that one would not use coroutines for, but it does serve to illustrate the issue that popped up while developing the code for this. And I"m going to start coroutine_yield(), as that does the actual switching of the stack to another “unit of execution” (note: this code is for the Intel 32-bit x86 architecture):

%assign	P_param		8 + 16
%assign	P_co		4 + 16

		push	ebp			; save callee saved registers
		push	ebx
		push	esi
		push	edi

		mov	eax,[esp + P_param]	; return parameter
		mov	edx,[esp + P_co]	; get stack to yield to
		xchg	esp,[edx]		; YIELD!

		pop	edi			; retore registers
		pop	esi
		pop	ebx
		pop	ebp

Since this is interfacing with C, I have to use the x86 32-bit calling convention (and for the record, I'm using the Intel syntax, not the AT&T syntax). Parameters are passed on the stack, and the callee (in this case, coroutine_yield32()) needs to save certain registers.

Normally, when switching a “unit of execution” such as a thread or process, one needs to save the entire CPU state. But I can cheat here—I'm calling a function, so I can skip saving registers the callee can use (read: trash), which saves a bit of time in the switching. So that's what's going on here. I have the registers that the C calling convention require saving, putting P_param into EAX to return it, get the pointer to the stack we're switching to and at the line that states “YIELD!” we switch the “units of execution.” The final five instructions are running under the coroutine, where we pull the registers saved and return into our now running coroutine.

But now here's the problem—this assumes the stack for the coroutine is properly initialized. Refering back to the C code, line 12 will yield back to line 3 and it works there because everything has been set up. But line 10 is problematic—that's the first switching of execution, and we haven't actually started sub_task(), which is expecting arguments already existing on the stack. Furthermore, for the C calling convention to work, we need to actually call sub_task(). I really don't want to mess up coroutine_yield() with special code to handle that case (that's just … ugly). I want to handle this cleanly.

So the first coroutine_yield() needs to call into (as per our example) sub_task(). The code for that looks like:

		push	eax		; return from coroutine_yield
		push	<coroutine self parameter>
		call	<our function>

Setting aside where we'll get the coroutine self paramter and the address for the function, we just need to ensure that our first call to coroutine_yield() resumes to this code fragment. And we can do that in the coroutine_create()—initialize the stack of the coroutine properly such that that happens. So let's name our fragment:

start_it_up:	push	eax		; return from coroutine_yield
		push	<coroutine self parameter>
		call	<our function>

and we can initialize the coroutine stack:

		mov	dword [ecx + 16],start_it_up
		xor	eax,eax
		mov	[ecx + 8],eax		; "saved" EBX
		mov	[ecx + 4],eax		; "saved" ESI
		mov	[ecx + 0],eax		; "saved" EDI
		mov	[edx],ecx		

For now, just accept that we have the new coroutine stack pointer in ECX (the final version uses ECX but I don't want to spoil things too much at this point). This populates the stack with the values needed for coroutine_yield() to fall into our code fragment, which is techincally a thunk. Now we turn our attention to saving the data required for our new thunk to call our function.

Now, on the 32-bit x86, a classical stack frame will look something like this:

Typical stack frame
offset from EBPcontents
12parameter 2
8parameter 1
4return address
0previous stack frame address (previous EBP)
-4local variable 1
-8local variable 2

The thunk doesn't need paramaters, nor does it need the return address or even a previous stack frame. We just need some local variables. So set up the stack like:

Coroutine stack contents
EBP of coroutine
coroutine pointer
address of sub_task()
address of start_it_up
stack frame for start_it_up
“saved” EBX
“saved” ESI
ESP of coroutine“saved” EDI

We can fix start_it_up:

%assign L_co		-4
%assign L_fun		-8

start_it_up:	push	eax
		push	dword [ebp + L_co]
		call	[ebp + L_fun]

And with our C example, this will get us to through line 15. At line 16 we have an issue, where we resume at line 7 and our coroutine now returns. Well, we did call it, so we get its return value back to our thunk. Well, the easy thing here is to just yield it back. And since we have the stack set for a call, we can save some instructions:

%assign L_co		-4
%assign L_fun		-8
%assign C_param		-12

start_it_up:	push	eax
		push	dword [ebp + L_co]
		call	[ebp + L_fun]

		mov	ebp + C_param],eax
		call	coroutine_yield32

And that will get us to line 18. But now we no longer have a running coroutine and we've run off the bottom of our thunk. There are two options here:

  1. call (or JMP) to abort();
  2. just yield the value back.

Both are valid responses, but I like the second one better as you might not know if a coroutine has finished or not. And that just requires one more instruction to start_it_up:

%assign L_co		-4
%assign L_fun		-8
%assign C_param		-12

start_it_up:	push	eax
		push	dword [ebp + L_co]
		call	[ebp + L_fun]

do_it_again:	mov	ebp + C_param],eax
		call	coroutine_yield32
		jmp	do_it_again

And there you go—coroutines for C.

The 64-bit version is pretty much the same—just that the registers needed to be saves are different, and the parameters are passed in registers instead of the stack, but overall, it's the same approach.

Should this code be used in production? I don't know. It works for Linux (both 32 and 64 bit versions) and for Mac OS-X (64 bit version). And while you can use setjmp()/longjmp(), you CANNOT do so across coroutine stacks (within the same coroutine—fine). And this has only been tested for C, NOT for C++. I don't know enough about C++ (or its calling conventions or exception handling) to recommend this for that.

But really that's all there is to it for coroutines in C.

And the final question—what are coroutines good for? That's for another post.

Saturday, February 18, 2017

I don't think we have to worry about the Vice President killing anyone this time around

To your request of my opinion of the manner in which a newspaper should be conducted, so as to be most useful, I should answer, "by restraining it to true facts & sound principles only." Yet I fear such a paper would find few subscribers. It is a melancholy truth, that a suppression of the press could not more compleatly deprive the nation of it's benefits, than is done by it's abandoned prostitution to falsehood. Nothing can now be believed which is seen in a newspaper.

Via Hacker News, Amendment I (Speech and Press): Thomas Jefferson to John Norvell


We've been worried about “fake news” since 1807!

And yet, somehow, we've survived.

As I've said, history doesn't repeat as much as rhyme

Obligatory Picture

[Don't hate me for my sock monkey headphones.]

Obligatory Links

Obligatory Miscellaneous

You have my permission to link freely to any entry here. Go ahead, I won't bite. I promise.

The dates are the permanent links to that day's entries (or entry, if there is only one entry). The titles are the permanent links to that entry only. The format for the links are simple: Start with the base link for this site:, then add the date you are interested in, say 2000/08/01, so that would make the final URL:

You can also specify the entire month by leaving off the day portion. You can even select an arbitrary portion of time.

You may also note subtle shading of the links and that's intentional: the “closer” the link is (relative to the page) the “brighter” it appears. It's an experiment in using color shading to denote the distance a link is from here. If you don't notice it, don't worry; it's not all that important.

It is assumed that every brand name, slogan, corporate name, symbol, design element, et cetera mentioned in these pages is a protected and/or trademarked entity, the sole property of its owner(s), and acknowledgement of this status is implied.

Copyright © 1999-2017 by Sean Conner. All Rights Reserved.