The Boston Diaries

Wednesday, February 01, 2017

“I am sorry, we cannot allow you to view this innocuous text file. For reasons.”

I found a bug in “Project: Sippy-Cup”. The nature of the bug is not important, but QA and I were able to reproduce locally the issue seen in production. So I ask QA to send a copy of the SIPp script (an XML file) so I could fix the issue.

From

QA <XXXXXXXXXXXXXXXXXXXXXXX>

To

Sean Conner <XXXXXXXXXXXXXXXXXXX>

Subject

Sipp command and xml for CP-209

Date

Wednesday, February 1, 2017 4:27 PM

Access to attachments has been blocked. Blocked attachments: sipptestCp209.xml.

Well then … okay. I have QA resend it, this time renamed to sipptestCp209.txt. This time it shows up. I save it. Then …

GenericUnixPrompt> more sipptestCp209.txt
This attachment was removed because it contains data that could pose a
security risk.
GenericUnixPrmopt>

Seriously? Just to convince myself it wasn't Mac OS-X's version of more that was doing the conversion:

GenericUnixPrompt> cat sipptestCp209.txt
This attachment was removed because it contains data that could pose a
security risk.
GenericUnixPrmopt>

So it's our overly paranoid email system's fault.

Gotcha.

Facebook is not your social life

I went down to my computers to start the day, opened up a tab on Facebook and Twitter, and was faced with pages and pages of wallowing in the horrors of current politics.

All of a sudden I was tremendously conscious of how I had been turning to Twitter and Facebook increasingly for these little bursts of validation, of interaction with other people, and how as social media has evolved to be passing around the same overly amplified horrible news stories, we've lost a lot of the personal connection that led to the friendships and amazing relationships that I have with people I've met online, from the BBS days, to Usenet, to early bloggers back in the late '90s. Friendships and relationships that continue today. My favorite social media these days is email and private slack channels and other side channels that involve those same people.

…

But as I gazed through Twitter and Facebook on Monday morning, what I saw was the same story repeated hundreds of time, the same people linking to different version of the stories, each of which was designed to create more and more anxiety in the reader, more and more clicks, more and more "oh my god, I have to read more".

Flutterby™! : Thoughts on social media and politics 2017-01-31 17:43:23.491303+00

I feel like the rise of FaceGoogleMyBookPlusSpace is one of the reasons I no longer blog as much as I used to. So much is now locked up in the GoogleMyFacePlusSpaceBook silo, so many people think of MyFaceGoogleSpaceBookPlus as the Internet, that it just seems … pointless … to blog on my own little insignificant corner of it. Nearly everybody reads everything on FaceGoogleMyPlusSpaceBook.

I have to shake that out of my system.

It's been in the last year or so that I've realized that some of my friends use FaceGoogleMyBookPlusSpace to rant; to get it all out of their system; to shout at the tide to stop coming in. I just wish others would realize the same thing—that a lot of the craziness and anxiety and “news” being posted is just letting off steam and it doesn't require a reply (not that I'm perfect in this—I too succumb to the occasional craziness on FaceGoogleMyPlusSpaceBook much to my dismay).

Bunny was right—she had a MyFaceGoogleBookPlusSpace account for all of six hours before she realized the horror to come. And I keep having to remind myself—the map is not the territory.

Thursday, February 02, 2017

It only takes 33 bits of information to identify a single person in the world

Speaking of FaceGoogleMyBookPlusSpace, this was quoted in one of the comments on Flutterby:

The surest way to discourage conversations is to enable likes/favorites and retweets/reblogs. These inarticulate gestures barely qualify as social. They are literally the least amount of social interaction you can have with someone. Press a button, done. Much social, wow. They are the form letters of social networks. No personal response is required. To whom it may concern: click. I would've traded one hundred likes for one person taking the small amount of time and effort to personally reply with "I like your tweet." When my tweets got likes but no replies, it was still a very empty experience for me. It still felt like I was tweeting into the void.

What's Wrong With Twitter

While “likes” might be socially meaningless, they might also be quite dangerous as highlighted here:

Kosinski and his team tirelessly refined their models. In 2012, Kosinski proved that on the basis of an average of 68 Facebook "likes" by a user, it was possible to predict their skin color (with 95 percent accuracy), their sexual orientation (88 percent accuracy), and their affiliation to the Democratic or Republican party (85 percent). But it didn't stop there. Intelligence, religious affiliation, as well as alcohol, cigarette and drug use, could all be determined. From the data it was even possible to deduce whether someone's parents were divorced.

The strength of their modeling was illustrated by how well it could predict a subject's answers. Kosinski continued to work on the models incessantly: before long, he was able to evaluate a person better than the average work colleague, merely on the basis of ten Facebook "likes." Seventy "likes" were enough to outdo what a person's friends knew, 150 what their parents knew, and 300 "likes" what their partner knew. More "likes" could even surpass what a person thought they knew about themselves.

Via Hacker News, The Data That Turned the World Upside Down

Oh, and those online quizes that often show up on FaceGoogleMyBookPlusSpace? Yeah, they work just like “likes” in pinning you down (as described in the “World Upside Down” article).

I'm just saying …

Sunday, February 05, 2017

“Ever wonder why the American public got behind the idea of mandatory minimums and stiff sentences? The Seventies. The Seventies are why!”

Clearly, the only logical thing to happen at this point in the story is for Tupac Shakur’s future stepfather to study acupuncture.

Look, I told you today’s installment gets crazy.

It turns out that Marxist education is not actually helpful in curing drug addiction, so clinic staffer Mutulu Shakur learns acupuncture. He learns from a doctor working at Lincoln Detox, but his education is interrupted when the doctor dies of a heroin overdose. IN THE CLINIC.

But he finds a new teacher and he and others eventually get doctor of acupuncture degrees from the Acupuncture Association of Quebec. Naturally, with a cushy city gig and a growing acupuncture practice, Shakur comes to the same decision you would in such a situation: “I should use this place and its connections to start robbing banks so I can raise money to start a revolution.”

“Also,” he doubtless added, “to pay for a cocaine habit that is already considerable *fnorrrrrrrrkkkkkk*”

Reminder: this is all happening at a drug treatment clinic that is fully funded by the tax dollars of the City of New York!

But Shakur has never robbed a bank. He needs an experienced bank robber and oh look here comes Sekou Odinga, formerly of the BLA! Naturally, Shakur and Odinga need some logistical support, and what better place to find this than a bunch of white communist feminists —

Look, I told you this story gets crazy.

Days of Rage | Status 451

When I read this, I thought to myself, how did we ever surive the 60s and 70s? The whole sordid story comes across like a poor Monty Python sketch stretched way too long. Don't believe me?

In 1972, a group called Venceremos, from the Bay Area, literally broke out a black convict named Ronald Beaty during a prison transport so he could train them in guerrilla tactics and lead a revolution.

That was their actual plan. That was their entire actual plan.

Exactly that one bit from South Park, but a bunch of '70s white Bay Area radicals going, 'Token, you're black; you know guerrilla tactics.' (Spoiler: when Beaty got arrested again, he promptly rolled over on the white radicals.)

Days of Rage | Status 451

It goes on and on like that. I found it hilarious, even though it's about a serious topic of political activism gone terrorist. And believe me, the two bits I quoted aren't even all that outrageous. Wait until you read about the Episcopal Church funding a terrorist group out of Puerto Rico.

I wish I was making this up.

It's a long article and it's well worth reading as it points out what might be headed our way. History doesn't repeat, but it does tend to rhyme …

Saturday, February 11, 2017

Exit, stage left. Then right. Then left. Then right …

I was driving south on Dixie Highway when I noticed a slow moving freight train also moving south along the railroad tracks that parallelled the road. I hope I can beat the train to Yamato, I thought. I really don't want to wait for it to pass.

Of course I didn't beat it. Freight trains are freakishly long and it was crossing Yamato by the time I got there. And then it started to slow down. And then it stopped. And sat there. And sat there. Minutes pass. Then it started moving.

North!

WHAT THE XXXX‽ I thought. Sigh. Just listen to the radio until the train passes.

After a few minutes it stopped. And sat there. For many minutes. Then it started moving.

South!

Really?

By this time, the SUV next to me decided it had enough and drove over a three foot median. I thought about it, but I decided against it—the median looked a bit high for my car. Meanhile, the train continued moving south for a few minutes, then it again stopped, sat there for a few minutes, then started moving.

North!

A few more minutes and the whole process repeated itself. The train slowed to a stop, sat there for a few minutes, then started moving. South, if you haven't guessed it by now.

I was amazed at just how fast it got up to speed once it started moving again. A few miles of train pass, and I was able to continue on with my journey.

Total elapsed time: 30 minutes.

Tuesday, February 14, 2017

It was very crowded tonight, as if every couple in Boca Raton decided to eat out because it was a special occasion or even a holiday of some sort

So Bunny wanted to eat out at the Biergarten in downtown Boca Raton for dinner. Hey, I'm always up for German food. Why not? We arrive, and I start driving around the expansive parking lot, looking for a space, any space, to park the car. After driving around for some fifteen minutes or so, Bunny finally saw a sign for public parking leading into a parking garage.

And yes, it had public parking. Two spots, five levels up, marked “Reserved between 8am and 5pm.”

Public parking.

Now all we had to do was find our way back down to ground level. We tried looking for an elevator, but there were no obvious elevatory looking doors to be found. One level down we found an unmarked door in an otherwise featureless wall and I figured why not? It was unlocked.

It lead into an odd shaped corridor. About halfway down was another unmarked door in an otherwise featureless wall in an otherwise featureless odd shaped corridor. It was unlocked. Why not?

I walked through the door and found myself in Biltmore. What the … I thought. This is odd.

“I don't think we want this door,” said Bunny.

“Yeah,” I said, backing through the door and into the oddly shaped corridor. “I don't think we want that door.” I walked down the corridor a bit more, around a three-quarter turn to find yet another unmarked door in an otherwise featureless cul-de-sac. It too, was unlocked.

We walked through, down two and a half steps, along another oddly shaped corridor and found, yes, another unmarked door in an otherwise featureless cul-de-sac. Fortunately for us, this one actually lead us outside the parking garage somewhere in downtown Boca Raton. We started walking and after a bit, found ourselves along Federal Highway a few blocks north of the Biergarten.

Several minutes later, we arrived at our destination. Despite the serious lack of parking, the place itself wasn't crowded and we were able to get seats immediately.

“I smell smoke,” said Bunny almost immediately upon sitting down.

“Smells like pipe smoke. All the windows and doors are open,” I said, looking around the place.

“Must be coming in from the lounge area,” said Bunny. “Pardon me.”

“Yes?”

“Can we sit elsewhere? The smoke is bothering me.”

“You can try over here.”

“Hmmm … no. I can still smell it. I'm sorry, I think we have to go.” Bunny is very sensitive to smoke.

“I'm sorry.”

“Maybe next time,” said Bunny as we left. The Biergarten is in a large shopping center and there were plenty of other restuarants within sight.

“There's an Indian place over there,” I said, pointing to a restaurant a few dozen yards away.

“Let's try it,” said Bunny.

I was a bit fearful as there were throngs of people milling just outside the restaurant. But the place was in a throng of restaurants in this section of the shopping center, so it was hard to tell if the Indian place was crowded or other restaurants were crowded and spilling out. We made our way into the restaurant and it wasn't completely crowded—there were a few empty tables.

And it smelled wonderful!

We stood there, waiting for a maitre d' or a host or a waitron or anybody who worked at the restuarant to notice us. There were plenty of waitrons rushing about, both inside and out, but for some reason, we were invisible. It got rather freaky when I intentionally stepped in front of a passing waitron and he just … appeared to pass through me? Around me? Somehow he ended up on the other side of me and kept on going.

By now, there was a sizable crowd of people waiting just inside the door with us, and just as invisible. None of the staff were paying any of us attention.

After a few minutes of being ignored, Bunny and I left. We decided to try one of the restaurants along Federal Highway that we passed, and that's how we found ourselves walking into Roots Italian Kitchen, a small place nestled between an Irish pub and a Vietnamese grill house. It was crowded, but not so much that we had to wait for a table. The service was professional, friendly and not too slow, not too fast.

And the food?

[A picture of a plate full of food is so cliché. Why not mix it up a bit with a picture of a post dinner plate? Who knows? It could be the next fad!]

Too good to pause long enough to take a picture. Everything was fresh! Even dessert, which we could smell being made. And yes, it was wonderful.

Wow!

And because Bunny and I ended up closing out the place, the owners gave Bunny a long stem rose and a piece of cheesecake, made fresh earlier in the day.

[I swear it felt like a holiday or something.]

It was an amazing find.

All we had to do afterwards was locate the car, parked on the fifth level, in a parking garage, somewhere in downtown Boca Raton.

Friday, February 17, 2017

Bunny and I subject ourselves to the Carbonaro Effect

Bunny and I went to see Michael Carbonaro at the Au-Rene Theater at the Broward Center for the Performing Arts. Carbonaro is a magician known for his television show The Carbonaro Effect, a show in the vein of Candid Camera, where he performs magic tricks on unsuspecting people. We were not entirely sure what to expect for a live performance though.

The opening act was meh (a comedian who laughed at his own jokes, has a website at … um … “drunken … mumble … something dot … com?” Not very memorable) but Carbonaro was great! His magic isn't anything we haven't seen before—mostly standard magical acts (restoring a torn newspaper, some mentalism acts, turning a coin into a gold fish) but it was his presentation that made them work (the torn newspaper done as an example of déjà vu, having a tatoo of an audience member's name on his chest, using a child to color a quarter orange before turning it into a fish).

He also has a wicked, but family friendly, sense of humor that made the two hour show fly by.

And his use of shaving cream was unique. I've never seen that done and frankly, never thought of using it in that way.

Oh, what way? I'm sorry, you'll just have to see the show to see how he transforms shaving cream into something special and unique.

Saturday, February 18, 2017

I don't think we have to worry about the Vice President killing anyone this time around

To your request of my opinion of the manner in which a newspaper should be conducted, so as to be most useful, I should answer, "by restraining it to true facts & sound principles only." Yet I fear such a paper would find few subscribers. It is a melancholy truth, that a suppression of the press could not more compleatly deprive the nation of it's benefits, than is done by it's abandoned prostitution to falsehood. Nothing can now be believed which is seen in a newspaper.

Via Hacker News, Amendment I (Speech and Press): Thomas Jefferson to John Norvell

1807!

We've been worried about “fake news” since 1807!

And yet, somehow, we've survived.

As I've said, history doesn't repeat as much as rhyme …

Monday, February 27, 2017

An implementation of coroutines for C

I use coroutines in Lua and I do miss having them in C. Sure, there are a number of coroutine implementations for C, but they all generally fall into two camps:

they rely upon setjmp()/longjmp(), which is problematic because it is seriously abusing those two functions (which exist to give C a form of exception) and
they rely upon ucontext.h, which has been deprecated by POSIX (and the API is a bit clunky for what I want to do)

But really, I just wanted to write my own implementation, because.

So the idea is to write some code that works like:

extern int       coroutine_create(coroutine__s **pcp,coroutine_function__f fun);
extern uintptr_t coroutine_yield (coroutine__s *co,uintptr_t value);
extern int       coroutine_free  (coroutine__s *co);

uintptr_t sub_task(coroutine__s *self,uintptr_t value) /* [1] */
{
  value = coroutine_yield(self,value); /* [2] */
	/* [3] */
  value = coroutine_yield(self,value); /* [4] */
	/* [5] */
  value = coroutine_yield(self,value); /* [6] */
	/* [7] */
  return value;
}

void main_task(void)
{
  coroutine__s *co;
  uintptr_t     v;

  coroutine_create(&co,sub_task); /* [8] magic here! */
	/* [ 9] */
  v = coroutine_yield(co,v); /* [10] */
	/* [11] */
  v = coroutine_yield(co,v); /* [12] */
	/* [13] */
  v = coroutine_yield(co,v); /* [14] */
	/* [15] */
  v = coroutine_yield(co,v); /* [16] */
	/* [17] */
  v = coroutine_yield(co,v); /* [18] */
	/* [19] */
  coroutine_free(co);
}

It's a contrived example that one would not use coroutines for, but it does serve to illustrate the issue that popped up while developing the code for this. And I"m going to start coroutine_yield(), as that does the actual switching of the stack to another “unit of execution” (note: this code is for the Intel 32-bit x86 architecture):

%assign	P_param		8 + 16
%assign	P_co		4 + 16

coroutine_yield32:
		push	ebp			; save callee saved registers
		push	ebx
		push	esi
		push	edi

		mov	eax,[esp + P_param]	; return parameter
		mov	edx,[esp + P_co]	; get stack to yield to
		xchg	esp,[edx]		; YIELD!

		pop	edi			; retore registers
		pop	esi
		pop	ebx
		pop	ebp
		ret

Since this is interfacing with C, I have to use the x86 32-bit calling convention (and for the record, I'm using the Intel syntax, not the AT&T syntax). Parameters are passed on the stack, and the callee (in this case, coroutine_yield32()) needs to save certain registers.

Normally, when switching a “unit of execution” such as a thread or process, one needs to save the entire CPU state. But I can cheat here—I'm calling a function, so I can skip saving registers the callee can use (read: trash), which saves a bit of time in the switching. So that's what's going on here. I have the registers that the C calling convention require saving, putting P_param into EAX to return it, get the pointer to the stack we're switching to and at the line that states “YIELD!” we switch the “units of execution.” The final five instructions are running under the coroutine, where we pull the registers saved and return into our now running coroutine.

But now here's the problem—this assumes the stack for the coroutine is properly initialized. Refering back to the C code, line 12 will yield back to line 3 and it works there because everything has been set up. But line 10 is problematic—that's the first switching of execution, and we haven't actually started sub_task(), which is expecting arguments already existing on the stack. Furthermore, for the C calling convention to work, we need to actually call sub_task(). I really don't want to mess up coroutine_yield() with special code to handle that case (that's just … ugly). I want to handle this cleanly.

So the first coroutine_yield() needs to call into (as per our example) sub_task(). The code for that looks like:

		push	eax		; return from coroutine_yield
		push	<coroutine self parameter>
		call	<our function>

Setting aside where we'll get the coroutine self paramter and the address for the function, we just need to ensure that our first call to coroutine_yield() resumes to this code fragment. And we can do that in the coroutine_create()—initialize the stack of the coroutine properly such that that happens. So let's name our fragment:

start_it_up:	push	eax		; return from coroutine_yield
		push	<coroutine self parameter>
		call	<our function>

and we can initialize the coroutine stack:

		mov	dword [ecx + 16],start_it_up
		xor	eax,eax
		mov	[ecx + 8],eax		; "saved" EBX
		mov	[ecx + 4],eax		; "saved" ESI
		mov	[ecx + 0],eax		; "saved" EDI
		mov	[edx],ecx

For now, just accept that we have the new coroutine stack pointer in ECX (the final version uses ECX but I don't want to spoil things too much at this point). This populates the stack with the values needed for coroutine_yield() to fall into our code fragment, which is techincally a thunk. Now we turn our attention to saving the data required for our new thunk to call our function.

Now, on the 32-bit x86, a classical stack frame will look something like this:

Typical stack frame
offset from EBP	contents
12	parameter 2
8	parameter 1
4	return address
0	previous stack frame address (previous EBP)
-4	local variable 1
-8	local variable 2

The thunk doesn't need paramaters, nor does it need the return address or even a previous stack frame. We just need some local variables. So set up the stack like:

Coroutine stack contents
EBP of coroutine
	coroutine pointer
	address of `sub_task()`
	address of `start_it_up`
	stack frame for `start_it_up`
	“saved” `EBX`
	“saved” `ESI`
ESP of coroutine	“saved” `EDI`

We can fix start_it_up:

%assign L_co		-4
%assign L_fun		-8

start_it_up:	push	eax
		push	dword [ebp + L_co]
		call	[ebp + L_fun]

And with our C example, this will get us to through line 15. At line 16 we have an issue, where we resume at line 7 and our coroutine now returns. Well, we did call it, so we get its return value back to our thunk. Well, the easy thing here is to just yield it back. And since we have the stack set for a call, we can save some instructions:

%assign L_co		-4
%assign L_fun		-8
%assign C_param		-12

start_it_up:	push	eax
		push	dword [ebp + L_co]
		call	[ebp + L_fun]

		mov	ebp + C_param],eax
		call	coroutine_yield32

And that will get us to line 18. But now we no longer have a running coroutine and we've run off the bottom of our thunk. There are two options here:

call (or JMP) to abort();
just yield the value back.

Both are valid responses, but I like the second one better as you might not know if a coroutine has finished or not. And that just requires one more instruction to start_it_up:

%assign L_co		-4
%assign L_fun		-8
%assign C_param		-12

start_it_up:	push	eax
		push	dword [ebp + L_co]
		call	[ebp + L_fun]

do_it_again:	mov	ebp + C_param],eax
		call	coroutine_yield32
		jmp	do_it_again

And there you go—coroutines for C.

The 64-bit version is pretty much the same—just that the registers needed to be saves are different, and the parameters are passed in registers instead of the stack, but overall, it's the same approach.

Should this code be used in production? I don't know. It works for Linux (both 32 and 64 bit versions) and for Mac OS-X (64 bit version). And while you can use setjmp()/longjmp(), you CANNOT do so across coroutine stacks (within the same coroutine—fine). And this has only been tested for C, NOT for C++. I don't know enough about C++ (or its calling conventions or exception handling) to recommend this for that.

But really that's all there is to it for coroutines in C.

And the final question—what are coroutines good for? That's for another post.

The Lego Batman Movie

I seriously did not expect “The Lego Batman Movie” to be a romantic comedy. A somewhat twisted romantic comedy, but a romantic comedy none-the-less. At the same time, it's a good Batman movie, really getting the notion of his character—he is the Batman, and Bruce Wayne is the mask he wears. It also showcases the conflict between him and Superman way better than “Batman v Superman: Dawn of Justice (way better—skip that movie and watch this twice).

Plot wise, there isn't much here you already haven't seen before. The Joker wants to take over Gotham and Batman has to stop him, while learning a valuable life lesson. But you aren't going to watch this for the plot, but for the humor and character interactions. Oh, and the meta humor. Alfred: “Sir, I have seen you go through similar phases in 2016 and 2012 and 2008 and 2005 and 1997 and 1995 and 1992 and 1989 and that wierd one in 1966.” Batman: “I have aged phenomenally.” Great stuff.

And lest I forget to mention it, Batman's Rogue Gallery, including Condiment King (yes, Bunny, he really is a Batman villian) show up to help ruin Batman's day.

I was also surprised at the cameos—and no, I don't mean actors who supplied voices, but actual character cameos that showed up for the final act of the movie. That I did not see, and it was most entertaining.

So come for the comedy, stay for the shark repllent Batspray. You won't be disappointed.